mit-han-lab / Block-Sparse-AttentionLinks
A sparse attention kernel supporting mix sparse patterns
☆436Updated last week
Alternatives and similar repositories for Block-Sparse-Attention
Users that are interested in Block-Sparse-Attention are comparing it to the libraries listed below
Sorting:
- SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse–Linear Attention☆243Updated 2 weeks ago
- [ICML 2025] XAttention: Block Sparse Attention with Antidiagonal Scoring☆263Updated 6 months ago
- ☆189Updated last year
- ☆216Updated last month
- [ICLR'25] ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation☆146Updated 9 months ago
- A Distributed Attention Towards Linear Scalability for Ultra-Long Context, Heterogeneous Data Training☆619Updated this week
- Efficient triton implementation of Native Sparse Attention.☆258Updated 7 months ago
- A Survey of Efficient Attention Methods: Hardware-efficient, Sparse, Compact, and Linear Attention☆271Updated last month
- Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference☆160Updated 3 months ago
- 📚A curated list of Awesome Diffusion Inference Papers with Codes: Sampling, Cache, Quantization, Parallelism, etc.🎉☆490Updated last month
- [ICLR 2025] COAT: Compressing Optimizer States and Activation for Memory-Efficient FP8 Training☆257Updated 5 months ago
- [ICML 2024] Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference☆364Updated 6 months ago
- A parallelism VAE avoids OOM for high resolution image generation☆84Updated 5 months ago
- [ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.☆897Updated 2 weeks ago
- A curated list of recent papers on efficient video attention for video diffusion models, including sparsification, quantization, and cach…☆53Updated 2 months ago
- SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs☆184Updated 3 months ago
- 🎬 3.7× faster video generation E2E 🖼️ 1.6× faster image generation E2E ⚡ ColumnSparseAttn 9.3× vs FlashAttn‑3 💨 ColumnSparseGEMM 2.5× …☆99Updated 4 months ago
- High performance inference engine for diffusion models☆103Updated 4 months ago
- [ICML2025, NeurIPS2025 Spotlight] Sparse VideoGen 1 & 2: Accelerating Video Diffusion Transformers with Sparse Attention☆616Updated last month
- [ICML 2025] SparseLoRA: Accelerating LLM Fine-Tuning with Contextual Sparsity☆68Updated 6 months ago
- 16-fold memory access reduction with nearly no loss☆109Updated 9 months ago
- Accelerating MoE with IO and Tile-aware Optimizations☆542Updated this week
- ☆444Updated 5 months ago
- Code for the paper “Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling”☆110Updated this week
- Code for Draft Attention☆98Updated 7 months ago
- USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference☆626Updated this week
- Official implementation of "Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding"☆790Updated last month
- Distributed MoE in a Single Kernel [NeurIPS '25]☆178Updated this week
- ☆126Updated 4 months ago
- FastCache: Fast Caching for Diffusion Transformer Through Learnable Linear Approximation [Efficient ML Model]☆46Updated last month