JerryYin777 / Cross-Layer-AttentionLinks
Self Reproduction Code of Paper "Reducing Transformer Key-Value Cache Size with Cross-Layer Attention (MIT CSAIL)
☆16Updated last year
Alternatives and similar repositories for Cross-Layer-Attention
Users that are interested in Cross-Layer-Attention are comparing it to the libraries listed below
Sorting:
- Open-Pandora: On-the-fly Control Video Generation☆34Updated 6 months ago
- ☆31Updated last year
- IntLLaMA: A fast and light quantization solution for LLaMA☆18Updated last year
- [ICML24] Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for LLMs☆84Updated 7 months ago
- [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts☆40Updated last year
- An unofficial implementation of "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"☆35Updated last year
- GIFT: Generative Interpretable Fine-Tuning☆20Updated 8 months ago
- Code for paper "Patch-Level Training for Large Language Models"☆85Updated 7 months ago
- ☆21Updated 2 months ago
- Triton implement of bi-directional (non-causal) linear attention☆50Updated 4 months ago
- "Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding" Zhenyu Zhang, Runjin Chen, Shiw…☆29Updated last year
- Quantized Attention on GPU☆44Updated 7 months ago
- Code for Draft Attention☆72Updated last month
- ☆76Updated 4 months ago
- ☆51Updated 3 months ago
- Official PyTorch implementation of the paper "dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching" (dLLM-Cache…☆113Updated last week
- Efficient Mixture of Experts for LLM Paper List☆77Updated 6 months ago
- ☆85Updated 2 months ago
- Odysseus: Playground of LLM Sequence Parallelism☆70Updated last year
- The this is the official implementation of "DAPE: Data-Adaptive Positional Encoding for Length Extrapolation"☆38Updated 8 months ago
- The official repo of continuous speculative decoding☆27Updated 2 months ago
- [ArXiv] V2PE: Improving Multimodal Long-Context Capability of Vision-Language Models with Variable Visual Position Encoding☆50Updated 6 months ago
- differentiable top-k operator☆21Updated 5 months ago
- Implementation for the paper: CMoE: Fast Carving of Mixture-of-Experts for Efficient LLM Inference☆21Updated 3 months ago
- [ICML 2024 Oral] This project is the official implementation of our Accurate LoRA-Finetuning Quantization of LLMs via Information Retenti…☆65Updated last year
- Official implementation of the paper: "A deeper look at depth pruning of LLMs"☆15Updated 11 months ago
- ☆104Updated 3 weeks ago
- Inference Code for Paper "Harder Tasks Need More Experts: Dynamic Routing in MoE Models"☆52Updated 10 months ago
- The official code implementation for paper "R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large Model Token Routing"☆35Updated last week
- [NAACL 2025] A Closer Look into Mixture-of-Experts in Large Language Models☆52Updated 4 months ago