Alternatives and similar repositories for native-sparse-attention

Users that are interested in native-sparse-attention are comparing it to the libraries listed below

• lucidrains / native-sparse-attention-pytorch
  Implementation of the sparse attention pattern proposed by the Deepseek team in their "Native Sparse Attention" paper
  ☆765Updated last month
• zhuzilin / ring-flash-attention
  Ring attention implementation with flash attention
  ☆885Updated last month
• MoonshotAI / Moonlight
  Muon is Scalable for LLM Training
  ☆1,321Updated 2 months ago
• feifeibear / long-context-attention
  USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference
  ☆571Updated 3 weeks ago
• fxmeng / TransMLA
  TransMLA: Multi-Head Latent Attention Is All You Need (NeurIPS 2025 Spotlight)
  ☆377Updated 2 weeks ago
• XunhaoLai / native-sparse-attention-triton
  Efficient triton implementation of Native Sparse Attention.
  ☆229Updated 4 months ago
• NVlabs / Fast-dLLM
  Official implementation of "Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding"
  ☆496Updated last week
• meta-pytorch / attention-gym
  Helpful tools and examples for working with flex-attention
  ☆1,015Updated last month
• fla-org / flame
  🔥 A minimal training framework for scaling FLA models
  ☆259Updated 3 weeks ago
• ByteDance-Seed / VeOmni
  VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo
  ☆1,194Updated this week
• QwenLM / ParScale
  Parallel Scaling Law for Language Model — Beyond Parameter and Inference Time Scaling
  ☆443Updated 4 months ago
• THUDM / slime
  slime is an LLM post-training framework for RL Scaling.
  ☆2,091Updated this week
• SandAI-org / MagiAttention
  A Distributed Attention Towards Linear Scalability for Ultra-Long Context, Heterogeneous Data Training
  ☆530Updated last week
• NVIDIA / Star-Attention
  Efficient LLM Inference over Long Sequences
  ☆390Updated 3 months ago
• NVIDIA / kvpress
  LLM KV cache compression made easy
  ☆642Updated this week
• mit-han-lab / duo-attention
  [ICLR 2025] DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
  ☆493Updated 8 months ago
• NVIDIA-NeMo / RL
  Scalable toolkit for efficient model reinforcement
  ☆920Updated this week
• KellerJordan / Muon
  Muon is an optimizer for hidden layers in neural networks
  ☆1,803Updated 2 months ago
• thinking-machines-lab / batch_invariant_ops
  ☆811Updated last week
• stepfun-ai / Step3
  ☆429Updated 2 months ago
• JT-Ushio / MHA2MLA
  Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs
  ☆191Updated last week
• NVlabs / GatedDeltaNet
  [ICLR 2025] Official PyTorch Implementation of Gated Delta Networks: Improving Mamba2 with Delta Rule
  ☆312Updated 3 weeks ago
• lucidrains / ring-attention-pytorch
  Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch
  ☆539Updated 4 months ago
• apple / ml-cross-entropy
  ☆529Updated 2 weeks ago
• sgl-project / SpecForge
  Train speculative decoding models effortlessly and port them smoothly to SGLang serving.
  ☆417Updated this week
• ByteDance-Seed / Triton-distributed
  Distributed Compiler based on Triton for Parallel Systems
  ☆1,152Updated last week
• Dao-AILab / quack
  A Quirky Assortment of CuTe Kernels
  ☆613Updated this week
• haoliuhl / ringattention
  Large Context Attention
  ☆742Updated 8 months ago
• October2001 / Awesome-KV-Cache-Compression
  📰 Must-read papers on KV Cache Compression (constantly updating 🤗).
  ☆552Updated last week
• mit-han-lab / Block-Sparse-Attention
  A sparse attention kernel supporting mix sparse patterns
  ☆311Updated 7 months ago