ZhenweiAn / Dynamic_MoELinks
Inference Code for Paper "Harder Tasks Need More Experts: Dynamic Routing in MoE Models"
β62Updated last year
Alternatives and similar repositories for Dynamic_MoE
Users that are interested in Dynamic_MoE are comparing it to the libraries listed below
Sorting:
- π LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Trainingβ89Updated 9 months ago
- β119Updated 3 months ago
- [ACL 2024] The official codebase for the paper "Self-Distillation Bridges Distribution Gap in Language Model Fine-tuning".β130Updated 10 months ago
- [SIGIR'24] The official implementation code of MOELoRA.β180Updated last year
- [EMNLP 2025] TokenSkip: Controllable Chain-of-Thought Compression in LLMsβ179Updated 2 months ago
- [ICLR 2025] Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Modelsβ129Updated 2 months ago
- β152Updated last year
- Repo for the EMNLP'24 Paper "Dual-Space Knowledge Distillation for Large Language Models". A general white-box KD framework for both sameβ¦β59Updated 3 weeks ago
- The source code of "Merging Experts into One: Improving Computational Efficiency of Mixture of Experts (EMNLP 2023)":β38Updated last year
- β122Updated 6 months ago
- β152Updated 3 months ago
- [ACL-25] We introduce ScaleQuest, a scalable, novel and cost-effective data synthesis method to unleash the reasoning capability of LLMs.β67Updated 10 months ago
- [ACL 2024] Long-Context Language Modeling with Parallel Encodingsβ162Updated last year
- [NeurIPS 2024] Fast Best-of-N Decoding via Speculative Rejectionβ51Updated 10 months ago
- β100Updated this week
- [ICLR 2025] SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Accelerationβ55Updated 7 months ago
- CoT-Valve: Length-Compressible Chain-of-Thought Tuningβ85Updated 7 months ago
- β65Updated 9 months ago
- [ICLR 2025] 𧬠RegMix: Data Mixture as Regression for Language Model Pre-training (Spotlight)β168Updated 7 months ago
- A block pruning framework for LLMs.β25Updated 4 months ago
- [ICLR 2025] MiniPLM: Knowledge Distillation for Pre-Training Language Modelsβ61Updated 9 months ago
- Model merging is a highly efficient approach for long-to-short reasoning.β82Updated 3 months ago
- β24Updated last year
- [ICLRβ24 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"β93Updated 3 months ago
- [ICML'25] Official code of paper "Fast Large Language Model Collaborative Decoding via Speculation"β25Updated 2 months ago
- [ACL 2024] Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Modelsβ101Updated last year
- Efficient Mixture of Experts for LLM Paper Listβ127Updated this week
- [ICLR 2025] LongPO: Long Context Self-Evolution of Large Language Models through Short-to-Long Preference Optimizationβ40Updated 6 months ago
- this is an implementation for the paper Improve Mathematical Reasoning in Language Models by Automated Process Supervision from google deβ¦β39Updated 2 months ago
- [ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Modelsβ78Updated last year