Alternatives and similar repositories for Cross-Layer-Attention:

Users that are interested in Cross-Layer-Attention are comparing it to the libraries listed below

• fla-org / flash-bidirectional-linear-attention
  Triton implement of bi-directional (non-causal) linear attention
  ☆35Updated last week
• OpenSparseLLMs / Open-Pandora
  Open-Pandora: On-the-fly Control Video Generation
  ☆31Updated last month
• BBuf / flash-rwkv
  ☆31Updated 7 months ago
• shaochenze / PatchTrain
  Code for paper "Patch-Level Training for Large Language Models"
  ☆75Updated 2 months ago
• SqueezeAILab / SqueezedAttention
  SQUEEZED ATTENTION: Accelerating Long Prompt LLM Inference
  ☆36Updated last month
• leezythu / FocusLLM
  FocusLLM: Scaling LLM’s Context by Parallel Decoding
  ☆34Updated last month
• Lucky-Lance / SPP
  [ICML 2024] SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Models
  ☆18Updated 7 months ago
• GATECH-EIC / Linearized-LLM
  [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models
  ☆28Updated 7 months ago
• pprp / Pruner-Zero
  Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for LLMs
  ☆76Updated last month
• MikaStars39 / StableMask
  PyTorch implementation of StableMask (ICML'24)
  ☆12Updated 6 months ago
• kamanphoebe / Look-into-MoEs
  A Closer Look into Mixture-of-Experts in Large Language Models
  ☆41Updated 5 months ago
• NVlabs / COAT
  ☆52Updated last week
• sramshetty / mixture-of-depths
  An unofficial implementation of "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆35Updated 7 months ago
• dvlab-research / Q-LLM
  This is the official repo of "QuickLLaMA: Query-aware Inference Acceleration for Large Language Models"
  ☆44Updated 6 months ago
• chuanyang-Zheng / DAPE
  The this is the official implementation of "DAPE: Data-Adaptive Positional Encoding for Length Extrapolation"
  ☆33Updated 3 months ago
• htqin / IR-QLoRA
  [ICML 2024 Oral] This project is the official implementation of our Accurate LoRA-Finetuning Quantization of LLMs via Information Retenti…
  ☆60Updated 9 months ago
• rayleizhu / vllm-ra
  [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts
  ☆38Updated 10 months ago
• fla-org / flame
  🔥 A minimal training framework for scaling FLA models
  ☆24Updated this week
• VITA-Group / Ms-PoE
  "Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding" Zhenyu Zhang, Runjin Chen, Shiw…
  ☆25Updated 8 months ago
• OpenGVLab / LLMPrune-BESA
  BESA is a differentiable weight pruning technique for large language models.
  ☆14Updated 10 months ago
• zyxxmu / cam
  Pytorch implementation of our paper accepted by ICML 2024 -- CaM: Cache Merging for Memory-efficient LLMs Inference
  ☆29Updated 7 months ago
• thu-coai / MiniPLM
  ☆27Updated last month
• ZetangForward / LCM_Stack
  Code for paper: Long cOntext aliGnment via efficient preference Optimization
  ☆13Updated this week
• thu-nics / MBQ
  The code repository of "MBQ: Modality-Balanced Quantization for Large Vision-Language Models"
  ☆27Updated 3 weeks ago
• csarron / PuMer
  [ACL 2023] PuMer: Pruning and Merging Tokens for Efficient Vision Language Models
  ☆28Updated 3 months ago
• Lucky-Lance / Expert_Sparsity
  [ACL 2024] Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
  ☆75Updated 7 months ago
• IST-DASLab / RoSA
  ☆38Updated 11 months ago
• savadikarc / gift
  GIFT: Generative Interpretable Fine-Tuning
  ☆19Updated 3 months ago
• pprp / Awesome-Efficient-MoE
  Efficient Mixture of Experts for LLM Paper List
  ☆26Updated last month
• ModelTC / QLLM
  [ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Mod…
  ☆35Updated 10 months ago