jax-ml / scaling-bookLinks
Home for "How To Scale Your Model", a short blog-style textbook about scaling LLMs on TPUs
☆440Updated last week
Alternatives and similar repositories for scaling-book
Users that are interested in scaling-book are comparing it to the libraries listed below
Sorting:
- ☆137Updated last week
- ☆274Updated last year
- ☆516Updated last year
- Legible, Scalable, Reproducible Foundation Models with Named Tensors and Jax☆627Updated this week
- ☆336Updated this week
- PyTorch Single Controller☆341Updated this week
- Small scale distributed training of sequential deep learning models, built on Numpy and MPI.☆137Updated last year
- ☆443Updated 9 months ago
- seqax = sequence modeling + JAX☆165Updated last week
- 🧱 Modula software package☆210Updated this week
- A repository to unravel the language of GPUs, making their kernel conversations easy to understand☆188Updated 2 months ago
- ☆203Updated 5 months ago
- JAX-Toolbox☆327Updated this week
- jax-triton contains integrations between JAX and OpenAI Triton☆411Updated last month
- MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvement…☆389Updated this week
- A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.☆563Updated 2 weeks ago
- Building blocks for foundation models.☆519Updated last year
- Implementation of Diffusion Transformer (DiT) in JAX☆280Updated last year
- Simple MPI implementation for prototyping or learning☆269Updated last week
- For optimization algorithm research and development.☆525Updated this week
- A JAX-native LLM Post-Training Library☆76Updated this week
- A simple library for scaling up JAX programs☆140Updated 9 months ago
- Puzzles for exploring transformers☆355Updated 2 years ago
- Named Tensors for Legible Deep Learning in JAX☆194Updated this week
- Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)☆366Updated last week
- Best practices & guides on how to write distributed pytorch training code☆460Updated 5 months ago
- CIFAR-10 speedruns: 94% in 2.6 seconds and 96% in 27 seconds☆274Updated 2 weeks ago
- Annotated version of the Mamba paper☆487Updated last year
- ☆162Updated last year
- Orbax provides common checkpointing and persistence utilities for JAX users☆410Updated this week