meta-pytorch / monarchLinks
PyTorch Single Controller
β374Updated this week
Alternatives and similar repositories for monarch
Users that are interested in monarch are comparing it to the libraries listed below
Sorting:
- Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)β392Updated this week
- π Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flashβ¦β262Updated last month
- β214Updated 6 months ago
- Load compute kernels from the Hubβ258Updated this week
- Small scale distributed training of sequential deep learning models, built on Numpy and MPI.β138Updated last year
- Home for "How To Scale Your Model", a short blog-style textbook about scaling LLMs on TPUsβ562Updated last week
- Dion optimizer algorithmβ318Updated last week
- Scalable and Performant Data Loadingβ295Updated this week
- A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.β571Updated 3 weeks ago
- FlexAttention based, minimal vllm-style inference engine for fast Gemma 2 inference.β261Updated 3 weeks ago
- Where GPUs get cooked π©βπ³π₯β279Updated 3 weeks ago
- KernelBench: Can LLMs Write GPU Kernels? - Benchmark with Torch -> CUDA problemsβ537Updated this week
- β530Updated last year
- A repository to unravel the language of GPUs, making their kernel conversations easy to understandβ191Updated 3 months ago
- Simple MPI implementation for prototyping or learningβ279Updated 3 weeks ago
- ring-attention experimentsβ149Updated 10 months ago
- Write a fast kernel and run it on Discord. See how you compare against the best!β51Updated this week
- β328Updated last week
- Learn CUDA with PyTorchβ67Updated this week
- A JAX-native LLM Post-Training Libraryβ128Updated this week
- Fast low-bit matmul kernels in Tritonβ356Updated last week
- β220Updated 2 months ago
- seqax = sequence modeling + JAXβ166Updated last month
- This repository contains the experimental PyTorch native float8 training UXβ224Updated last year
- π Collection of components for development, training, tuning, and inference of foundation models leveraging PyTorch native components.β209Updated last week
- β234Updated last week
- LLM KV cache compression made easyβ596Updated this week
- π· Build compute kernelsβ119Updated this week
- β217Updated 7 months ago
- Applied AI experiments and examples for PyTorchβ292Updated last week