jax-ml / scaling-bookLinks
Home for "How To Scale Your Model", a short blog-style textbook about scaling LLMs on TPUs
☆637Updated this week
Alternatives and similar repositories for scaling-book
Users that are interested in scaling-book are comparing it to the libraries listed below
Sorting:
- ☆537Updated last year
- Simple MPI implementation for prototyping or learning☆280Updated last month
- PyTorch Single Controller☆425Updated this week
- Building blocks for foundation models.☆557Updated last year
- Dion optimizer algorithm☆360Updated this week
- ☆281Updated last year
- KernelBench: Can LLMs Write GPU Kernels? - Benchmark with Torch -> CUDA problems☆581Updated 2 weeks ago
- Minimal yet performant LLM examples in pure JAX☆177Updated last week
- Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)☆410Updated last week
- Best practices & guides on how to write distributed pytorch training code☆487Updated 7 months ago
- FlexAttention based, minimal vllm-style inference engine for fast Gemma 2 inference.☆280Updated last month
- A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.☆576Updated last month
- Open-source framework for the research and development of foundation models.☆462Updated this week
- Small scale distributed training of sequential deep learning models, built on Numpy and MPI.☆142Updated last year
- Legible, Scalable, Reproducible Foundation Models with Named Tensors and Jax☆667Updated this week
- What would you do with 1000 H100s...☆1,109Updated last year
- ☆455Updated 11 months ago
- ☆221Updated 7 months ago
- Implementation of Diffusion Transformer (DiT) in JAX☆291Updated last year
- Puzzles for exploring transformers☆371Updated 2 years ago
- ☆773Updated 3 weeks ago
- A curated list of resources for learning and exploring Triton, OpenAI's programming language for writing efficient GPU code.☆417Updated 6 months ago
- 🧱 Modula software package☆277Updated last month
- A repository to unravel the language of GPUs, making their kernel conversations easy to understand☆193Updated 4 months ago
- ☆173Updated last year
- jax-triton contains integrations between JAX and OpenAI Triton☆426Updated last month
- ☆505Updated last month
- For optimization algorithm research and development.☆539Updated last week
- A Quirky Assortment of CuTe Kernels☆603Updated this week
- seqax = sequence modeling + JAX☆167Updated 2 months ago