Dao-AILab / grouped-latent-attentionLinks
☆130Updated 4 months ago
Alternatives and similar repositories for grouped-latent-attention
Users that are interested in grouped-latent-attention are comparing it to the libraries listed below
Sorting:
- Flash-Muon: An Efficient Implementation of Muon Optimizer☆195Updated 4 months ago
- The evaluation framework for training-free sparse attention in LLMs☆101Updated last week
- Odysseus: Playground of LLM Sequence Parallelism☆78Updated last year
- ☆145Updated 8 months ago
- PyTorch bindings for CUTLASS grouped GEMM.☆125Updated 4 months ago
- GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM☆169Updated last year
- Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference☆147Updated last week
- [ICLR 2025] COAT: Compressing Optimizer States and Activation for Memory-Efficient FP8 Training☆242Updated 2 months ago
- An efficient implementation of the NSA (Native Sparse Attention) kernel☆119Updated 4 months ago
- 16-fold memory access reduction with nearly no loss☆105Updated 6 months ago
- 🔥 A minimal training framework for scaling FLA models☆266Updated last month
- Accelerate LLM preference tuning via prefix sharing with a single line of code☆46Updated 3 months ago
- [ICML 2025] XAttention: Block Sparse Attention with Antidiagonal Scoring☆239Updated 3 months ago
- HALO: Hadamard-Assisted Low-Precision Optimization and Training method for finetuning LLMs. 🚀 The official implementation of https://arx…☆26Updated 8 months ago
- Efficient triton implementation of Native Sparse Attention.☆238Updated 5 months ago
- Implementation for FP8/INT8 Rollout for RL training without performence drop.☆260Updated 3 weeks ago
- ☆101Updated 5 months ago
- Triton implementation of FlashAttention2 that adds Custom Masks.☆141Updated last year
- Boosting 4-bit inference kernels with 2:4 Sparsity☆83Updated last year
- Triton-based implementation of Sparse Mixture of Experts.☆246Updated 3 weeks ago
- [ICLR 2025] Palu: Compressing KV-Cache with Low-Rank Projection☆143Updated 8 months ago
- [CoLM'25] The official implementation of the paper <MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression>☆147Updated 3 months ago
- Official repository for DistFlashAttn: Distributed Memory-efficient Attention for Long-context LLMs Training☆216Updated last year
- ☆50Updated 5 months ago
- ☆93Updated 7 months ago
- Training library for Megatron-based models☆125Updated this week
- Fast and memory-efficient exact attention☆71Updated 7 months ago
- Quantized Attention on GPU☆44Updated 11 months ago
- A curated list of recent papers on efficient video attention for video diffusion models, including sparsification, quantization, and cach…☆41Updated last month
- Low-bit optimizers for PyTorch☆132Updated 2 years ago