jax-ml / scaling-bookLinks
Home for "How To Scale Your Model", a short blog-style textbook about scaling LLMs on TPUs
☆523Updated this week
Alternatives and similar repositories for scaling-book
Users that are interested in scaling-book are comparing it to the libraries listed below
Sorting:
- ☆526Updated last year
- ☆275Updated last year
- Minimal yet performant LLM examples in pure JAX☆148Updated this week
- Legible, Scalable, Reproducible Foundation Models with Named Tensors and Jax☆643Updated this week
- ☆444Updated 10 months ago
- PyTorch Single Controller☆361Updated last week
- Simple MPI implementation for prototyping or learning☆278Updated 2 weeks ago
- Dion optimizer algorithm☆291Updated last week
- A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.☆568Updated last week
- ☆380Updated this week
- Implementation of Diffusion Transformer (DiT) in JAX☆291Updated last year
- jax-triton contains integrations between JAX and OpenAI Triton☆413Updated 2 months ago
- 🧱 Modula software package☆222Updated 3 weeks ago
- seqax = sequence modeling + JAX☆166Updated last month
- Building blocks for foundation models.☆532Updated last year
- Small scale distributed training of sequential deep learning models, built on Numpy and MPI.☆137Updated last year
- ☆211Updated 6 months ago
- MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvement…☆389Updated this week
- ☆324Updated 3 weeks ago
- JAX-Toolbox☆329Updated this week
- A repository to unravel the language of GPUs, making their kernel conversations easy to understand☆190Updated 2 months ago
- Puzzles for exploring transformers☆366Updated 2 years ago
- KernelBench: Can LLMs Write GPU Kernels? - Benchmark with Torch -> CUDA problems☆527Updated this week
- For optimization algorithm research and development.☆530Updated this week
- Annotated version of the Mamba paper☆487Updated last year
- Pax is a Jax-based machine learning framework for training large scale models. Pax allows for advanced and fully configurable experimenta…☆526Updated last week
- Best practices & guides on how to write distributed pytorch training code☆467Updated 6 months ago
- Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)☆383Updated last week
- Named Tensors for Legible Deep Learning in JAX☆201Updated this week
- ☆464Updated 2 weeks ago