Alternatives and similar repositories for Cross-Layer-Attention:

Users that are interested in Cross-Layer-Attention are comparing it to the libraries listed below

• OpenSparseLLMs / Open-Pandora
  Open-Pandora: On-the-fly Control Video Generation
  ☆32Updated 4 months ago
• fla-org / flash-bidirectional-linear-attention
  Triton implement of bi-directional (non-causal) linear attention
  ☆44Updated last month
• waltonfuture / Diff-eRank
  [NeurIPS 2024] A Novel Rank-Based Metric for Evaluating Large Language Models
  ☆43Updated 4 months ago
• sail-sg / LongSpec
  LongSpec: Long-Context Speculative Decoding with Efficient Drafting and Verification
  ☆42Updated 3 weeks ago
• chuanyang-Zheng / DAPE
  The this is the official implementation of "DAPE: Data-Adaptive Positional Encoding for Length Extrapolation"
  ☆37Updated 5 months ago
• yikangshen / MoA
  Mixture of Attention Heads
  ☆43Updated 2 years ago
• Lucky-Lance / SPP
  [ICML 2024] SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Models
  ☆18Updated 10 months ago
• BBuf / flash-rwkv
  ☆30Updated 10 months ago
• fla-org / flame
  🔥 A minimal training framework for scaling FLA models
  ☆82Updated this week
• OpenGVLab / LLMPrune-BESA
  BESA is a differentiable weight pruning technique for large language models.
  ☆14Updated last year
• savadikarc / gift
  GIFT: Generative Interpretable Fine-Tuning
  ☆20Updated 5 months ago
• shaochenze / PatchTrain
  Code for paper "Patch-Level Training for Large Language Models"
  ☆81Updated 4 months ago
• VITA-Group / Ms-PoE
  "Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding" Zhenyu Zhang, Runjin Chen, Shiw…
  ☆28Updated 10 months ago
• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆48Updated 2 years ago
• OpenSparseLLMs / MoM
  ☆71Updated last week
• zhixuan-lin / forgetting-transformer
  [ICLR 2025] Official PyTorch implementation of "Forgetting Transformer: Softmax Attention with a Forget Gate"
  ☆76Updated last week
• rayleizhu / vllm-ra
  [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts
  ☆39Updated last year
• GATECH-EIC / Linearized-LLM
  [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models
  ☆30Updated 9 months ago
• WillDreamer / Awesome-MLLM-Reasoning
  Recent Advances on MLLM's Reasoning Ability
  ☆24Updated this week
• sramshetty / mixture-of-depths
  An unofficial implementation of "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆34Updated 9 months ago
• horseee / CoT-Valve
  CoT-Valve: Length-Compressible Chain-of-Thought Tuning
  ☆55Updated last month
• pprp / Pruner-Zero
  Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for LLMs
  ☆80Updated 4 months ago
• megvii-research / IntLLaMA
  IntLLaMA: A fast and light quantization solution for LLaMA
  ☆18Updated last year
• OpenSparseLLMs / Linear-MoE
  ☆70Updated 2 weeks ago
• MikaStars39 / StableMask
  PyTorch implementation of StableMask (ICML'24)
  ☆12Updated 9 months ago
• hasanar1f / HiRED
  [AAAI 2025] HiRED strategically drops visual tokens in the image encoding stage to improve inference efficiency for High-Resolution Visio…
  ☆26Updated last month
• TIGER-AI-Lab / VISTA
  The code for "VISTA: Enhancing Long-Duration and High-Resolution Video Understanding by VIdeo SpatioTemporal Augmentation" [CVPR2025]
  ☆14Updated last month
• sail-sg / scaling-with-vocab
  [NeurIPS-2024] 📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623
  ☆80Updated 6 months ago
• pprp / Awesome-Efficient-MoE
  Efficient Mixture of Experts for LLM Paper List
  ☆47Updated 3 months ago
• OpenSparseLLMs / Linearization
  ☆36Updated last week