pprp / Awesome-Efficient-MoELinks
Efficient Mixture of Experts for LLM Paper List
☆145Updated 2 months ago
Alternatives and similar repositories for Awesome-Efficient-MoE
Users that are interested in Awesome-Efficient-MoE are comparing it to the libraries listed below
Sorting:
- qwen-nsa☆84Updated last month
- [ICLR 2025] PEARL: Parallel Speculative Decoding with Adaptive Draft Length☆132Updated last month
- The Official Implementation of Ada-KV [NeurIPS 2025]☆116Updated last week
- D^2-MoE: Delta Decompression for MoE-based LLMs Compression☆71Updated 8 months ago
- ☆120Updated 6 months ago
- [NeurIPS 2024] The official implementation of "Kangaroo: Lossless Self-Speculative Decoding for Accelerating LLMs via Double Early Exitin…☆63Updated last year
- Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference☆155Updated last month
- [ICML 2025] XAttention: Block Sparse Attention with Antidiagonal Scoring☆256Updated 5 months ago
- [ICLR 2025] Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models☆147Updated 4 months ago
- SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs☆175Updated 2 months ago
- QAQ: Quality Adaptive Quantization for LLM KV Cache☆55Updated last year
- Awesome LLM pruning papers all-in-one repository with integrating all useful resources and insights.☆134Updated 3 months ago
- Tiny-DeepSpeed, a minimalistic re-implementation of the DeepSpeed library☆48Updated 3 months ago
- ☆290Updated 4 months ago
- Implementation for FP8/INT8 Rollout for RL training without performence drop.☆275Updated 3 weeks ago
- [ACL 2024] Official PyTorch implementation of "IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intact"☆48Updated last year
- Official PyTorch implementation of the paper "dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching" (dLLM-Cache…☆186Updated 2 weeks ago
- [ICLR 2025] COAT: Compressing Optimizer States and Activation for Memory-Efficient FP8 Training☆250Updated 3 months ago
- [NeurIPS 2024 Oral🔥] DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMs.☆177Updated last year
- Code associated with the paper **Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding**☆212Updated 9 months ago
- [ICML 2025 Oral] Mixture of Lookup Experts☆55Updated 6 months ago
- DeepSeek Native Sparse Attention pytorch implementation☆108Updated 3 weeks ago
- ☆150Updated 5 months ago
- 16-fold memory access reduction with nearly no loss☆108Updated 8 months ago
- Awesome list for LLM pruning.☆276Updated last month
- Official implementation of ICML 2024 paper "ExCP: Extreme LLM Checkpoint Compression via Weight-Momentum Joint Shrinking".☆48Updated last year
- [AAAI 2024] Fluctuation-based Adaptive Structured Pruning for Large Language Models☆63Updated last year
- ☆49Updated last year
- [ICML 2024 Oral] This project is the official implementation of our Accurate LoRA-Finetuning Quantization of LLMs via Information Retenti…☆67Updated last year
- [ACL 2024] A novel QAT with Self-Distillation framework to enhance ultra low-bit LLMs.☆129Updated last year