rwitten / HighPerfLLMs2024Links
☆532Updated last year
Alternatives and similar repositories for HighPerfLLMs2024
Users that are interested in HighPerfLLMs2024 are comparing it to the libraries listed below
Sorting:
- Home for "How To Scale Your Model", a short blog-style textbook about scaling LLMs on TPUs☆591Updated last week
- Legible, Scalable, Reproducible Foundation Models with Named Tensors and Jax☆658Updated this week
- Building blocks for foundation models.☆544Updated last year
- Pax is a Jax-based machine learning framework for training large scale models. Pax allows for advanced and fully configurable experimenta…☆532Updated last week
- Deep learning for dummies. All the practical details and useful utilities that go into working with real models.☆814Updated last month
- What would you do with 1000 H100s...☆1,095Updated last year
- ☆279Updated last year
- ☆216Updated 7 months ago
- Puzzles for exploring transformers☆368Updated 2 years ago
- A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.☆575Updated last month
- PyTorch Single Controller☆414Updated this week
- ☆452Updated 10 months ago
- Open-source framework for the research and development of foundation models.☆419Updated this week
- Best practices & guides on how to write distributed pytorch training code☆475Updated 6 months ago
- Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)☆395Updated 2 weeks ago
- Minimal yet performant LLM examples in pure JAX☆152Updated this week
- ☆330Updated this week
- Small scale distributed training of sequential deep learning models, built on Numpy and MPI.☆139Updated last year
- A repository to unravel the language of GPUs, making their kernel conversations easy to understand☆193Updated 3 months ago
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆265Updated last month
- Accelerate, Optimize performance with streamlined training and serving options with JAX.☆308Updated this week
- Annotated version of the Mamba paper☆489Updated last year
- Puzzles for learning Triton☆1,978Updated 9 months ago
- seqax = sequence modeling + JAX☆167Updated last month
- KernelBench: Can LLMs Write GPU Kernels? - Benchmark with Torch -> CUDA problems☆557Updated 2 weeks ago
- JAX-Toolbox☆335Updated this week
- A curated list of resources for learning and exploring Triton, OpenAI's programming language for writing efficient GPU code.☆405Updated 6 months ago
- jax-triton contains integrations between JAX and OpenAI Triton☆416Updated last week
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs wel…☆374Updated 3 months ago
- MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvement…☆392Updated last week