BobMcDear / attorch
A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.
☆483Updated 3 weeks ago
Related projects ⓘ
Alternatives and complementary repositories for attorch
- This repository contains the experimental PyTorch native float8 training UX☆211Updated 3 months ago
- Helpful tools and examples for working with flex-attention☆469Updated 3 weeks ago
- ☆152Updated this week
- Cataloging released Triton kernels.☆134Updated 2 months ago
- Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch☆476Updated 3 weeks ago
- Mirage: Automatically Generating Fast GPU Kernels without Programming in Triton/CUDA☆636Updated this week
- Applied AI experiments and examples for PyTorch☆166Updated 2 weeks ago
- Puzzles for learning Triton☆1,135Updated this week
- Flash Attention in ~100 lines of CUDA (forward pass only)☆626Updated 7 months ago
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆193Updated this week
- ☆133Updated 9 months ago
- ☆267Updated this week
- Pipeline Parallelism for PyTorch☆726Updated 2 months ago
- Simple and fast low-bit matmul kernels in CUDA / Triton☆143Updated this week
- Annotated version of the Mamba paper☆457Updated 8 months ago
- For optimization algorithm research and development.☆449Updated this week
- UNet diffusion model in pure CUDA☆584Updated 4 months ago
- 🚀 Collection of components for development, training, tuning, and inference of foundation models leveraging PyTorch native components.☆165Updated this week
- Transformers with Arbitrarily Large Context☆641Updated 3 months ago
- Triton-based implementation of Sparse Mixture of Experts.☆185Updated last month
- Fast Matrix Multiplications for Lookup Table-Quantized LLMs☆187Updated this week
- Accelerated First Order Parallel Associative Scan☆163Updated 3 months ago
- Ring attention implementation with flash attention☆585Updated last week
- Tile primitives for speedy kernels☆1,658Updated this week
- Implementation of a Transformer, but completely in Triton☆249Updated 2 years ago
- A library to analyze PyTorch traces.☆307Updated this week
- Implementation of Diffusion Transformer (DiT) in JAX☆252Updated 5 months ago
- An implementation of the transformer architecture onto an Nvidia CUDA kernel☆157Updated last year
- Building blocks for foundation models.☆394Updated 10 months ago
- ☆224Updated 4 months ago