rwitten / HighPerfLLMs2024Links
☆518Updated last year
Alternatives and similar repositories for HighPerfLLMs2024
Users that are interested in HighPerfLLMs2024 are comparing it to the libraries listed below
Sorting:
- Home for "How To Scale Your Model", a short blog-style textbook about scaling LLMs on TPUs☆440Updated last week
- Building blocks for foundation models.☆525Updated last year
- Legible, Scalable, Reproducible Foundation Models with Named Tensors and Jax☆627Updated this week
- What would you do with 1000 H100s...☆1,079Updated last year
- ☆274Updated last year
- Pax is a Jax-based machine learning framework for training large scale models. Pax allows for advanced and fully configurable experimenta…☆522Updated this week
- Deep learning for dummies. All the practical details and useful utilities that go into working with real models.☆809Updated this week
- A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.☆565Updated this week
- Puzzles for exploring transformers☆355Updated 2 years ago
- ☆206Updated 5 months ago
- PyTorch Single Controller☆341Updated this week
- Best practices & guides on how to write distributed pytorch training code☆460Updated 5 months ago
- ☆443Updated 9 months ago
- Small scale distributed training of sequential deep learning models, built on Numpy and MPI.☆137Updated last year
- ☆347Updated this week
- KernelBench: Can LLMs Write GPU Kernels? - Benchmark with Torch -> CUDA problems☆493Updated last week
- ☆137Updated last week
- seqax = sequence modeling + JAX☆165Updated last week
- Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)☆372Updated this week
- ☆323Updated last month
- A repository to unravel the language of GPUs, making their kernel conversations easy to understand☆188Updated 2 months ago
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs wel…☆364Updated last month
- Accelerate, Optimize performance with streamlined training and serving options with JAX.☆293Updated last week
- A curated list of resources for learning and exploring Triton, OpenAI's programming language for writing efficient GPU code.☆382Updated 4 months ago
- Puzzles for learning Triton☆1,801Updated 8 months ago
- ☆162Updated last year
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆258Updated last week
- jax-triton contains integrations between JAX and OpenAI Triton☆411Updated last month
- ☆227Updated this week
- Simple MPI implementation for prototyping or learning☆269Updated last week