A Distributed Attention Towards Linear Scalability for Ultra-Long Context, Heterogeneous Data Training
☆967Oct 30, 2026Updated this week
Alternatives and similar repositories for MagiAttention
Users that are interested in MagiAttention are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A plug-and-play compiler that delivers free-lunch optimizations for both inference and training.☆335Updated this week
- Distributed Compiler and Optimized Parallel Kernels☆1,576Sep 18, 2026Updated last month
- MAGI-1: Autoregressive Video Generation at Scale☆3,798Jun 17, 2026Updated 4 months ago
- Ring attention implementation with flash attention☆1,065Sep 10, 2025Updated last year
- Accelerating MoE with IO and Tile-aware Optimizations☆787Aug 29, 2026Updated 2 months ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo☆2,271Updated this week
- A fast communication-overlapping library for tensor/expert parallelism on GPUs.☆1,375Aug 28, 2025Updated last year
- 🐳 Efficient Triton implementations for "Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention"☆1,026Feb 5, 2026Updated 8 months ago
- USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference☆696Sep 16, 2026Updated last month
- A Quirky Assortment of CuTe Kernels☆1,189Updated this week
- 🚀 Efficient implementations for emerging model architectures☆6,014Updated this week
- [ICML2025, NeurIPS2025 Spotlight] Sparse VideoGen 1 & 2: Accelerating Video Diffusion Transformers with Sparse Attention☆712Jul 4, 2026Updated 3 months ago
- A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on H…☆3,599Updated this week
- ☆37Aug 7, 2025Updated last year
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- ☆255Nov 19, 2025Updated 11 months ago
- xDiT: A Scalable Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism☆2,748Updated this week
- A unified inference and post-training framework for accelerated video generation.☆4,768Updated this week
- MAGI-2-preview: Scaling Video Generation Models Efficiently☆641Aug 6, 2026Updated 2 months ago
- NVSHMEM‑Tutorial: Build a DeepEP‑like GPU Buffer☆208Feb 11, 2026Updated 8 months ago
- [ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.☆1,266Feb 25, 2026Updated 8 months ago
- Helpful tools and examples for working with flex-attention☆1,263Updated this week
- FlashInfer: Kernel Library for LLM Serving☆6,840Updated this week
- Tile primitives for speedy kernels☆3,790Sep 12, 2026Updated last month
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- [ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-t…☆3,985Jan 17, 2026Updated 9 months ago
- A sparse attention kernel supporting mix sparse patterns☆567Jan 18, 2026Updated 9 months ago
- Ulysses sequence-parallel all-to-all as a torch custom op, moved by the GPU copy engines into torch symmetric memory. Zero SM usage; 1.66…☆54Aug 25, 2026Updated 2 months ago
- Perplexity GPU Kernels☆614Nov 7, 2025Updated 11 months ago
- high-performance linear attention kernel library built on TileLang☆745Updated this week
- CUDA kernels for linear attention variants, written in CuTe DSL and CUTLASS C++.☆561Updated this week
- DeepSeek-V3.2-Exp DSA Warmup Lightning Indexer training operator based on tilelang☆52Nov 19, 2025Updated 11 months ago
- A PyTorch native platform for training generative AI models☆5,871Updated this week
- flex-block-attn: an efficient block sparse attention computation library☆133Dec 26, 2025Updated 10 months ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- An efficient implementation of the NSA (Native Sparse Attention) kernel☆135Jun 24, 2025Updated last year
- [NeurIPS 2025] Radial Attention: O(nlogn) Sparse Attention with Energy Decay for Long Video Generation☆609Nov 11, 2025Updated 11 months ago
- Kernel Library for Large Headdim Attention (64~1024, BF16/FP8/FP4), 1.5x~15x↑ vs PyTorch SDPA.☆345Oct 23, 2026Updated last week
- Kernel Design Agents (KDA) is a agent-centric workflow to write high-performance CUDA Kernels.☆1,517Oct 19, 2026Updated 2 weeks ago
- ByteCheckpoint: An Unified Checkpointing Library for LFMs☆290Feb 2, 2026Updated 9 months ago
- Efficient Triton Kernels for LLM Training☆6,683Updated this week
- MoonEP: A Perfectly Balanced Expert Parallelism Library via Dynamic Redundant Experts☆1,197Oct 12, 2026Updated 3 weeks ago