HazyResearch / aisys-building-blocks
Building blocks for foundation models.
☆394Updated 10 months ago
Related projects ⓘ
Alternatives and complementary repositories for aisys-building-blocks
- Helpful tools and examples for working with flex-attention☆469Updated 3 weeks ago
- Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch☆476Updated 3 weeks ago
- A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.☆483Updated 3 weeks ago
- Puzzles for learning Triton☆1,135Updated this week
- A bibliography and survey of the papers surrounding o1☆754Updated this week
- Annotated version of the Mamba paper☆457Updated 8 months ago
- What would you do with 1000 H100s...☆903Updated 10 months ago
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆193Updated this week
- Transformers with Arbitrarily Large Context☆641Updated 3 months ago
- Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton☆1,339Updated this week
- Cataloging released Triton kernels.☆134Updated 2 months ago
- This repository contains the experimental PyTorch native float8 training UX☆211Updated 3 months ago
- Pipeline Parallelism for PyTorch☆726Updated 2 months ago
- ☆224Updated 4 months ago
- Small scale distributed training of sequential deep learning models, built on Numpy and MPI.☆107Updated last year
- ☆197Updated 4 months ago
- 🚀 Collection of components for development, training, tuning, and inference of foundation models leveraging PyTorch native components.☆165Updated this week
- Ring attention implementation with flash attention☆585Updated last week
- Puzzles for exploring transformers☆325Updated last year
- Applied AI experiments and examples for PyTorch☆166Updated 2 weeks ago
- ☆133Updated 9 months ago
- An ML Systems Onboarding list☆545Updated this week
- Legible, Scalable, Reproducible Foundation Models with Named Tensors and Jax☆516Updated this week
- Flash Attention in ~100 lines of CUDA (forward pass only)☆626Updated 7 months ago
- ☆391Updated last month
- Collection of kernels written in Triton language☆68Updated 3 weeks ago
- A repository for research on medium sized language models.☆479Updated this week
- Triton-based implementation of Sparse Mixture of Experts.☆185Updated last month
- GPU programming related news and material links☆1,237Updated last month