meta-pytorch / monarchLinks
PyTorch Single Controller
☆438Updated this week
Alternatives and similar repositories for monarch
Users that are interested in monarch are comparing it to the libraries listed below
Sorting:
- Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)☆415Updated last week
- Load compute kernels from the Hub☆299Updated this week
- Dion optimizer algorithm☆366Updated 2 weeks ago
- FlexAttention based, minimal vllm-style inference engine for fast Gemma 2 inference.☆293Updated 2 months ago
- KernelBench: Can LLMs Write GPU Kernels? - Benchmark with Torch -> CUDA problems☆601Updated last week
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆270Updated 2 months ago
- ☆222Updated 2 weeks ago
- A Python-embedded DSL that makes it easy to write fast, scalable ML kernels with minimal boilerplate.☆389Updated this week
- Home for "How To Scale Your Model", a short blog-style textbook about scaling LLMs on TPUs☆655Updated last week
- Scalable and Performant Data Loading☆308Updated this week
- Small scale distributed training of sequential deep learning models, built on Numpy and MPI.☆145Updated last year
- ☆240Updated this week
- LLM KV cache compression made easy☆650Updated this week
- 👷 Build compute kernels☆158Updated this week
- Write a fast kernel and run it on Discord. See how you compare against the best!☆58Updated this week
- A Quirky Assortment of CuTe Kernels☆627Updated last week
- A repository to unravel the language of GPUs, making their kernel conversations easy to understand☆194Updated 4 months ago
- ring-attention experiments☆153Updated last year
- Applied AI experiments and examples for PyTorch☆299Updated last month
- Where GPUs get cooked 👩🍳🔥☆293Updated last month
- Fast low-bit matmul kernels in Triton☆381Updated 2 weeks ago
- A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.☆577Updated 2 months ago
- ☆827Updated this week
- Cataloging released Triton kernels.☆261Updated last month
- kernels, of the mega variety☆586Updated 2 weeks ago
- 🚀 Collection of components for development, training, tuning, and inference of foundation models leveraging PyTorch native components.☆215Updated this week
- A library to analyze PyTorch traces.☆416Updated 2 weeks ago
- Simple MPI implementation for prototyping or learning☆284Updated 2 months ago
- ☆218Updated 8 months ago
- A curated collection of resources, tutorials, and best practices for learning and mastering NVIDIA CUTLASS☆232Updated 5 months ago