Alternatives and similar repositories for Cross-Layer-Attention

Users that are interested in Cross-Layer-Attention are comparing it to the libraries listed below

• JieShibo / MoLE
  [ICML 2025 Oral] Mixture of Lookup Experts
  ☆61Updated last month
• pprp / Pruner-Zero
  [ICML24] Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for LLMs
  ☆98Updated last year
• OpenSparseLLMs / Open-Pandora
  Open-Pandora: On-the-fly Control Video Generation
  ☆35Updated last year
• VILA-Lab / GBLM-Pruner
  Are gradient information useful for pruning of LLMs?
  ☆47Updated 4 months ago
• htqin / IR-QLoRA
  [ICML 2024 Oral] This project is the official implementation of our Accurate LoRA-Finetuning Quantization of LLMs via Information Retenti…
  ☆67Updated last year
• UNITES-Lab / C2R-MoE
  [NAACL'25 🏆 SAC Award] Official code for "Advancing MoE Efficiency: A Collaboration-Constrained Routing (C2R) Strategy for Better Expert…
  ☆13Updated 11 months ago
• fla-org / flash-bidirectional-linear-attention
  Triton implement of bi-directional (non-causal) linear attention
  ☆60Updated 11 months ago
• ModelTC / QLLM
  [ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Mod…
  ☆39Updated last year
• chuanyang-Zheng / DAPE
  The this is the official implementation of "DAPE: Data-Adaptive Positional Encoding for Length Extrapolation"
  ☆40Updated last year
• sramshetty / mixture-of-depths
  An unofficial implementation of "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆36Updated last year
• thu-ml / ReMoE
  [ICLR2025] Codebase for "ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing", built on Megatron-LM.
  ☆104Updated last year
• Aaronhuang-778 / Mixture-Compressor-MoE
  [ICLR 2025] Mixture Compressor for Mixture-of-Experts LLMs Gains More
  ☆65Updated 10 months ago
• savadikarc / wegeft
  WeGeFT: Weight‑Generative Fine‑Tuning for Multi‑Faceted Efficient Adaptation of Large Models
  ☆22Updated 6 months ago
• leo-yangli / VB-LoRA
  This repo contains the source code for VB-LoRA: Extreme Parameter Efficient Fine-Tuning with Vector Banks (NeurIPS 2024).
  ☆42Updated last year
• shaochenze / PatchTrain
  Code for paper "Patch-Level Training for Large Language Models"
  ☆96Updated 2 months ago
• jxiw / M1
  M1: Towards Scalable Test-Time Compute with Mamba Reasoning Models
  ☆47Updated 5 months ago
• ZhenweiAn / Dynamic_MoE
  Inference Code for Paper "Harder Tasks Need More Experts: Dynamic Routing in MoE Models"
  ☆67Updated last year
• Dao-AILab / grouped-latent-attention
  ☆133Updated 7 months ago
• zhixuan-lin / forgetting-transformer
  [ICLR 2025 & COLM 2025] Official PyTorch implementation of the Forgetting Transformer and Adaptive Computation Pruning
  ☆134Updated 3 weeks ago
• TianjinYellow / StableSPAM
  ☆27Updated 9 months ago
• Doraemonzzz / xmixers
  Xmixers: A collection of SOTA efficient token/channel mixers
  ☆28Updated 4 months ago
• AkideLiu / MiniCache
  ☆10Updated last year
• thu-nics / MoA
  [CoLM'25] The official implementation of the paper <MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression>
  ☆153Updated last month
• OpenSparseLLMs / Linearization
  ☆62Updated 6 months ago
• YuchuanTian / DiJiang
  [ICML'24 Oral] The official code of "DiJiang: Efficient Large Language Models through Compact Kernelization", a novel DCT-based linear at…
  ☆104Updated last year
• Hon-Wong / ByteVideoLLM
  [ICCV 2025] Dynamic-VLM
  ☆28Updated last year
• sustcsonglin / linear-attention-and-beyond-slides
  ☆102Updated 10 months ago
• ZihaoHuang-notabot / Ultra-Sparse-Memory-Network
  ☆38Updated 3 months ago
• ByteDance-Seed / Seed-1.8
  ☆191Updated 3 weeks ago
• UNITES-Lab / MC-SMoE
  [ICLR‘24 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"
  ☆102Updated 6 months ago