XunhaoLai / native-sparse-attention-tritonLinks
Efficient triton implementation of Native Sparse Attention.
β208Updated 3 months ago
Alternatives and similar repositories for native-sparse-attention-triton
Users that are interested in native-sparse-attention-triton are comparing it to the libraries listed below
Sorting:
- π₯ A minimal training framework for scaling FLA modelsβ228Updated last week
- [ICML 2025] XAttention: Block Sparse Attention with Antidiagonal Scoringβ219Updated last month
- β237Updated 2 months ago
- Triton implementation of FlashAttention2 that adds Custom Masks.β132Updated last year
- Flash-Muon: An Efficient Implementation of Muon Optimizerβ160Updated 2 months ago
- [ICLR 2025] COAT: Compressing Optimizer States and Activation for Memory-Efficient FP8 Trainingβ227Updated 2 weeks ago
- β80Updated 6 months ago
- An efficient implementation of the NSA (Native Sparse Attention) kernelβ113Updated 2 months ago
- Official implementation of "Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding"β374Updated this week
- β123Updated 2 months ago
- A sparse attention kernel supporting mix sparse patternsβ280Updated 6 months ago
- Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inferenceβ133Updated 3 months ago
- SeerAttention: Learning Intrinsic Sparse Attention in Your LLMsβ147Updated 2 weeks ago
- [ICLR2025] Codebase for "ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing", built on Megatron-LM.β87Updated 8 months ago
- β139Updated 6 months ago
- [CoLM'25] The official implementation of the paper <MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression>β145Updated last month
- [ICLR 2025] DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Headsβ487Updated 6 months ago
- Efficient Attention Methods: A Comprehensive Surveyβ75Updated this week
- [ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"β91Updated 2 months ago
- Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMsβ188Updated 2 months ago
- β117Updated 2 months ago
- Fast and memory-efficient exact attentionβ70Updated 5 months ago
- TransMLA: Multi-Head Latent Attention Is All You Needβ339Updated last month
- The evaluation framework for training-free sparse attention in LLMsβ90Updated 2 months ago
- β55Updated last month
- 16-fold memory access reduction with nearly no lossβ104Updated 4 months ago
- Low-bit optimizers for PyTorchβ130Updated last year
- Implementation for FP8/INT8 Rollout for RL training without performence drop.β124Updated this week
- A curated list of recent papers on efficient video attention for video diffusion models, including sparsification, quantization, and cachβ¦β35Updated last month
- β85Updated 3 weeks ago