Alternatives and similar repositories for Cross-Layer-Attention

Users that are interested in Cross-Layer-Attention are comparing it to the libraries listed below

• JieShibo / MoLE
  [ICML 2025 Oral] Mixture of Lookup Experts
  ☆51Updated 3 months ago
• fla-org / flash-bidirectional-linear-attention
  Triton implement of bi-directional (non-causal) linear attention
  ☆52Updated 6 months ago
• UNITES-Lab / C2R-MoE
  [NAACL 25 SAC Award] Official code for "Advancing MoE Efficiency: A Collaboration-Constrained Routing (C2R) Strategy for Better Expert Pa…
  ☆10Updated 6 months ago
• pprp / Pruner-Zero
  [ICML24] Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for LLMs
  ☆92Updated 9 months ago
• attention-survey / Efficient_Attention_Survey
  A Survey of Efficient Attention Methods: Hardware-efficient, Sparse, Compact, and Linear Attention
  ☆162Updated last week
• sramshetty / mixture-of-depths
  An unofficial implementation of "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆35Updated last year
• hao-ai-lab / Awesome-Video-Attention
  A curated list of recent papers on efficient video attention for video diffusion models, including sparsification, quantization, and cach…
  ☆36Updated last month
• savadikarc / wegeft
  WeGeFT: Weight‑Generative Fine‑Tuning for Multi‑Faceted Efficient Adaptation of Large Models
  ☆21Updated last month
• Aaronhuang-778 / Mixture-Compressor-MoE
  [ICLR 2025] Mixture Compressor for Mixture-of-Experts LLMs Gains More
  ☆50Updated 6 months ago
• OpenSparseLLMs / Open-Pandora
  Open-Pandora: On-the-fly Control Video Generation
  ☆34Updated 9 months ago
• htqin / IR-QLoRA
  [ICML 2024 Oral] This project is the official implementation of our Accurate LoRA-Finetuning Quantization of LLMs via Information Retenti…
  ☆67Updated last year
• sustcsonglin / linear-attention-and-beyond-slides
  ☆80Updated 6 months ago
• OpenSparseLLMs / Linearization
  ☆55Updated last month
• Dao-AILab / grouped-latent-attention
  ☆123Updated 3 months ago
• thu-nics / MBQ
  The code repository of "MBQ: Modality-Balanced Quantization for Large Vision-Language Models"
  ☆55Updated 5 months ago
• OpenSparseLLMs / MoM
  ☆100Updated 4 months ago
• thu-nics / R2R
  The official code implementation for paper "R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large Model Token Routing"
  ☆44Updated 3 weeks ago
• mit-han-lab / x-attention
  [ICML 2025] XAttention: Block Sparse Attention with Antidiagonal Scoring
  ☆222Updated last month
• mdy666 / Qwen-Native-Sparse-Attention
  qwen-nsa
  ☆74Updated 4 months ago
• zhixuan-lin / forgetting-transformer
  [ICLR 2025 & COLM 2025] Official PyTorch implementation of the Forgetting Transformer and Adaptive Computation Pruning
  ☆124Updated 2 weeks ago
• li-plus / flash-preference
  Accelerate LLM preference tuning via prefix sharing with a single line of code
  ☆43Updated last month
• ModelTC / QLLM
  [ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Mod…
  ☆39Updated last year
• feifeibear / ChituAttention
  Quantized Attention on GPU
  ☆44Updated 9 months ago
• leo-yangli / VB-LoRA
  This repo contains the source code for VB-LoRA: Extreme Parameter Efficient Fine-Tuning with Vector Banks (NeurIPS 2024).
  ☆40Updated 10 months ago
• OpenSparseLLMs / Linear-MoE
  ☆117Updated 2 months ago
• bojone / softtopk
  differentiable top-k operator
  ☆22Updated 8 months ago
• ThisisBillhe / ZipCache
  [NeurIPS 2024] The official implementation of ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identification
  ☆25Updated 5 months ago
• tilde-research / nsa-impl
  An efficient implementation of the NSA (Native Sparse Attention) kernel
  ☆114Updated 2 months ago
• BBuf / flash-rwkv
  ☆32Updated last year
• pprp / Awesome-Efficient-MoE
  Efficient Mixture of Experts for LLM Paper List
  ☆118Updated this week