BobMcDear / attorch
A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.
☆473Updated 2 weeks ago
Related projects ⓘ
Alternatives and complementary repositories for attorch
- This repository contains the experimental PyTorch native float8 training UX☆211Updated 3 months ago
- ☆140Updated this week
- Helpful tools and examples for working with flex-attention☆460Updated 2 weeks ago
- Cataloging released Triton kernels.☆132Updated 2 months ago
- Puzzles for learning Triton☆1,060Updated last month
- Flash Attention in ~100 lines of CUDA (forward pass only)☆609Updated 7 months ago
- Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch☆474Updated 2 weeks ago
- ☆133Updated 9 months ago
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆190Updated 2 weeks ago
- For optimization algorithm research and development.☆408Updated this week
- Applied AI experiments and examples for PyTorch☆159Updated last week
- Mirage: Automatically Generating Fast GPU Kernels without Programming in Triton/CUDA☆593Updated this week
- Annotated version of the Mamba paper☆455Updated 8 months ago
- ☆262Updated this week
- Legible, Scalable, Reproducible Foundation Models with Named Tensors and Jax☆516Updated this week
- Transformers with Arbitrarily Large Context☆637Updated 2 months ago
- Pipeline Parallelism for PyTorch☆725Updated 2 months ago
- UNet diffusion model in pure CUDA☆567Updated 4 months ago
- Fast Matrix Multiplications for Lookup Table-Quantized LLMs☆183Updated last month
- Implementation of Diffusion Transformer (DiT) in JAX☆252Updated 4 months ago
- Tile primitives for speedy kernels☆1,629Updated this week
- Building blocks for foundation models.☆385Updated 10 months ago
- Ring attention implementation with flash attention☆578Updated this week
- 🚀 Collection of components for development, training, tuning, and inference of foundation models leveraging PyTorch native components.☆163Updated this week
- Simple and fast low-bit matmul kernels in CUDA / Triton☆133Updated this week
- Implementation of a Transformer, but completely in Triton☆248Updated 2 years ago
- An implementation of the transformer architecture onto an Nvidia CUDA kernel☆157Updated last year
- Collection of kernels written in Triton language☆63Updated last week
- ring-attention experiments☆95Updated 3 weeks ago
- Accelerated First Order Parallel Associative Scan☆162Updated 2 months ago