Alternatives and similar repositories for MHA2MLA

Users that are interested in MHA2MLA are comparing it to the libraries listed below

• MuLabPKU / TransMLA

  View on GitHub
  TransMLA: Multi-Head Latent Attention Is All You Need (NeurIPS 2025 Spotlight)
  ☆429Sep 23, 2025Updated 4 months ago
• Dao-AILab / grouped-latent-attention

  View on GitHub
  ☆131May 29, 2025Updated 8 months ago
• XunhaoLai / native-sparse-attention-triton

  View on GitHub
  Efficient triton implementation of Native Sparse Attention.
  ☆262May 23, 2025Updated 8 months ago
• thu-ml / SpargeAttn

  View on GitHub
  [ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.
  ☆927Dec 31, 2025Updated last month
• TsinghuaC3I / FS-GEN

  View on GitHub
  Fast and Slow Generating: An Empirical Study on Large and Small Language Models Collaborative Decoding.
  ☆13Nov 19, 2024Updated last year
• MeganTj / multimodal_alignment

  View on GitHub
  ☆19Jun 4, 2025Updated 8 months ago
• itsnamgyu / block-transformer

  View on GitHub
  Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)
  ☆163Apr 13, 2025Updated 10 months ago
• OpenLMLab / scaling-rope

  View on GitHub
  code for Scaling Laws of RoPE-based Extrapolation
  ☆73Oct 16, 2023Updated 2 years ago
• PiotrNawrot / sparse-frontier

  View on GitHub
  The evaluation framework for training-free sparse attention in LLMs
  ☆119Jan 27, 2026Updated 2 weeks ago
• dinobby / MAgICoRE

  View on GitHub
  ☆23Sep 19, 2024Updated last year
• nil0x9 / flash-muon

  View on GitHub
  Flash-Muon: An Efficient Implementation of Muon Optimizer
  ☆235Jun 15, 2025Updated 8 months ago
• mit-han-lab / x-attention

  View on GitHub
  [ICML 2025] XAttention: Block Sparse Attention with Antidiagonal Scoring
  ☆269Jul 6, 2025Updated 7 months ago
• MoonshotAI / Moonlight

  View on GitHub
  Muon is Scalable for LLM Training
  ☆1,426Aug 3, 2025Updated 6 months ago
• MoonshotAI / MoBA

  View on GitHub
  MoBA: Mixture of Block Attention for Long-Context LLMs
  ☆2,044Apr 3, 2025Updated 10 months ago
• SkyworkAI / Skywork-MoE

  View on GitHub
  Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
  ☆139Jun 12, 2024Updated last year
• ZunhaiSu / Super-Experts-Profilling

  View on GitHub
  (ICLR 2026) Unveiling Super Experts in Mixture-of-Experts Large Language Models
  ☆36Sep 25, 2025Updated 4 months ago
• fla-org / native-sparse-attention

  View on GitHub
  🐳 Efficient Triton implementations for "Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention"
  ☆964Feb 5, 2026Updated last week
• feifeibear / ChituAttention

  View on GitHub
  Quantized Attention on GPU
  ☆44Nov 22, 2024Updated last year
• lmbxmu / CutDiffusion

  View on GitHub
  CutDiffusion: A Simple, Fast, Cheap, and Strong Diffusion Extrapolation Method
  ☆27Oct 9, 2025Updated 4 months ago
• SJTU-DENG-Lab / UniCMs

  View on GitHub
  ☆39May 20, 2025Updated 8 months ago
• PKU-SEC-Lab / HybriMoE

  View on GitHub
  [DAC'25] Official implement of "HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference"
  ☆101Dec 15, 2025Updated last month
• NROwind / OpenGPT-4o-Image

  View on GitHub
  A Comprehensive Dataset for Advanced Image Generation and Editing}
  ☆31Oct 2, 2025Updated 4 months ago
• zaydzuhri / flame

  View on GitHub
  Fork of Flame repo for training of some new stuff in development
  ☆19Jan 5, 2026Updated last month
• SalesforceAIResearch / LaTRO

  View on GitHub
  ☆123Feb 21, 2025Updated 11 months ago
• whyNLP / LCKV

  View on GitHub
  Layer-Condensed KV cache w/ 10 times larger batch size, fewer params and less computation. Dramatic speed up with better task performance…
  ☆157Apr 7, 2025Updated 10 months ago
• InternRobotics / OST-Bench

  View on GitHub
  [NeurIPS 2025] OST-Bench: Evaluating the Capabilities of MLLMs in Online Spatio-temporal Scene Understanding
  ☆71Sep 29, 2025Updated 4 months ago
• OpenMOSS / LongLLaDA

  View on GitHub
  [AAAI26] LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs
  ☆52Dec 7, 2025Updated 2 months ago
• GATECH-EIC / Linearized-LLM

  View on GitHub
  [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models
  ☆35Jun 12, 2024Updated last year
• PKU-ML / LongPPL

  View on GitHub
  Code for ICLR 2025 Paper "What is Wrong with Perplexity for Long-context Language Modeling?"
  ☆109Oct 11, 2025Updated 4 months ago
• JoeLi12345 / nGPT

  View on GitHub
  an open source reproduction of NVIDIA's nGPT (Normalized Transformer with Representation Learning on the Hypersphere)
  ☆110Mar 7, 2025Updated 11 months ago
• sail-sg / SkyLadder

  View on GitHub
  The official repository for SkyLadder: Better and Faster Pretraining via Context Window Scheduling
  ☆42Dec 29, 2025Updated last month
• SeunghyunSEO / optimized_hf_llama_class_for_training

  View on GitHub
  ☆48Aug 29, 2024Updated last year
• microsoft / MInference

  View on GitHub
  [NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention…
  ☆1,183Sep 30, 2025Updated 4 months ago
• GeniusHTX / TALE

  View on GitHub
  ☆145Sep 12, 2025Updated 5 months ago
• abdelfattah-lab / TokenButler

  View on GitHub
  ☆27Nov 25, 2025Updated 2 months ago
• sail-sg / SimLayerKV

  View on GitHub
  The official implementation of paper: SimLayerKV: A Simple Framework for Layer-Level KV Cache Reduction.
  ☆52Oct 18, 2024Updated last year
• tval2 / contextual-pruning

  View on GitHub
  Library to facilitate pruning of LLMs based on context
  ☆32Jan 31, 2024Updated 2 years ago
• Infini-AI-Lab / MagicDec

  View on GitHub
  [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆142Dec 4, 2024Updated last year
• zhiyuanhubj / LongRecipe

  View on GitHub
  LongRecipe: Recipe for Efficient Long Context Generalization in Large Language Models
  ☆79Oct 16, 2024Updated last year