pprp / Awesome-Efficient-MoELinks
Efficient Mixture of Experts for LLM Paper List
☆140Updated 3 weeks ago
Alternatives and similar repositories for Awesome-Efficient-MoE
Users that are interested in Awesome-Efficient-MoE are comparing it to the libraries listed below
Sorting:
- qwen-nsa☆79Updated last week
- [ICLR 2025] PEARL: Parallel Speculative Decoding with Adaptive Draft Length☆120Updated 6 months ago
- D^2-MoE: Delta Decompression for MoE-based LLMs Compression☆69Updated 7 months ago
- ☆119Updated 4 months ago
- [ICLR 2025] COAT: Compressing Optimizer States and Activation for Memory-Efficient FP8 Training☆242Updated 2 months ago
- The Official Implementation of Ada-KV [NeurIPS 2025]☆106Updated last month
- [NeurIPS 2024] The official implementation of "Kangaroo: Lossless Self-Speculative Decoding for Accelerating LLMs via Double Early Exitin…☆60Updated last year
- [ICML 2025] XAttention: Block Sparse Attention with Antidiagonal Scoring☆239Updated 3 months ago
- SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs☆158Updated last month
- [AAAI 2024] Fluctuation-based Adaptive Structured Pruning for Large Language Models☆61Updated last year
- Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference☆147Updated last week
- QAQ: Quality Adaptive Quantization for LLM KV Cache☆52Updated last year
- [ICML 2025 Oral] Mixture of Lookup Experts☆53Updated 5 months ago
- Due to the huge vocaburary size (151,936) of Qwen models, the Embedding and LM Head weights are excessively heavy. Therefore, this projec…☆27Updated last year
- Code associated with the paper **Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding**☆205Updated 8 months ago
- DeepSeek Native Sparse Attention pytorch implementation☆106Updated 2 weeks ago
- [ACL 2024] A novel QAT with Self-Distillation framework to enhance ultra low-bit LLMs.☆122Updated last year
- ☆148Updated 3 months ago
- ☆284Updated 3 months ago
- Official Repo for SparseLLM: Global Pruning of LLMs (NeurIPS 2024)☆67Updated 6 months ago
- [ICLR 2025] Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models☆137Updated 3 months ago
- Awesome LLM pruning papers all-in-one repository with integrating all useful resources and insights.☆129Updated 2 months ago
- Official implementation of ICML 2024 paper "ExCP: Extreme LLM Checkpoint Compression via Weight-Momentum Joint Shrinking".☆48Updated last year
- ☆46Updated 11 months ago
- Awesome list for LLM pruning.☆267Updated 2 weeks ago
- Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs☆191Updated 3 weeks ago
- GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM☆169Updated last year
- [NeurIPS 2024 Oral🔥] DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMs.☆171Updated last year
- ☆60Updated 10 months ago
- [ACL 2025 main] FR-Spec: Frequency-Ranked Speculative Sampling☆45Updated 3 months ago