Inference Code for Paper "Harder Tasks Need More Experts: Dynamic Routing in MoE Models"
☆74Jul 30, 2024Updated 2 years ago
Alternatives and similar repositories for Dynamic_MoE
Users that are interested in Dynamic_MoE are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆15Oct 19, 2024Updated 2 years ago
- Official implementation of RMoE (Layerwise Recurrent Router for Mixture-of-Experts)☆33Aug 4, 2024Updated 2 years ago
- [ICLR2025] Codebase for "ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing", built on Megatron-LM.☆123Dec 20, 2024Updated last year
- ☆17Jun 11, 2025Updated last year
- ☆20Nov 5, 2024Updated 2 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- ☆26Oct 22, 2025Updated last year
- Mixture of Attention Heads☆54Oct 10, 2022Updated 4 years ago
- 🍼 Official implementation of Dynamic Data Mixing Maximizes Instruction Tuning for Mixture-of-Experts☆40Sep 29, 2024Updated 2 years ago
- [ICML 2024] Code for the paper "MoE-RBench: Towards Building Reliable Language Models with Sparse Mixture-of-Experts"☆11Jul 1, 2024Updated 2 years ago
- [ACL 2026 Findings] RACER: Retrieval-Augmented Contextual Rapid Speculative Decoding☆18Sep 23, 2026Updated last month
- Efficient Mixture of Experts for LLM Paper List☆191Jun 16, 2026Updated 4 months ago
- 🚀 LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training☆93Dec 3, 2024Updated last year
- Official code for the ICLR 2025 paper, "Ada-K Routing: Boosting the Efficiency of MoE-based LLMs"☆12Mar 1, 2025Updated last year
- [ACL 2024] Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models☆124May 24, 2024Updated 2 years ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- Self Reproduction Code of Paper "Reducing Transformer Key-Value Cache Size with Cross-Layer Attention (MIT CSAIL)☆17May 24, 2024Updated 2 years ago
- This is code for the EMNLP 2022 Paper "UniRPG: Unified Discrete Reasoning over Table and Text as Program Generation".☆10Apr 30, 2023Updated 3 years ago
- Code for "Routing Manifold Alignment Improves Generalization of Mixture-of-Experts LLMs"☆19Nov 6, 2025Updated last year
- The code for "SE-GSL: A General and Effective Graph Structure Learning Framework through Structural Entropy Optimization"☆15Mar 10, 2023Updated 3 years ago
- Code for ACL 2024 accepted paper titled "SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language …☆40Jan 13, 2025Updated last year
- Source codes for paper "BlockFFN: Towards End-Side Acceleration-Friendly Mixture-of-Experts with Chunk-Level Activation Sparsity".☆19Jan 10, 2026Updated 9 months ago
- ☆29May 24, 2024Updated 2 years ago
- Entropy-Driven GRPO with Guided Error Correction for Advantage Diversity☆23Aug 28, 2025Updated last year
- Official implementation of Dynamic Routing Transformer Network for Multimodal Sarcasm Detection (ACL'23)☆35Jul 9, 2023Updated 3 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- [ICLR 2025] MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts☆281Oct 16, 2024Updated 2 years ago
- Official repository for FLAME-MoE: A Transparent End-to-End Research Platform for Mixture-of-Experts Language Models☆43Sep 19, 2025Updated last year
- Curated collection of papers in MoE model inference☆430Mar 12, 2026Updated 7 months ago
- Awesome Agent Environments☆20Apr 10, 2026Updated 6 months ago
- A family of open-sourced Mixture-of-Experts (MoE) Large Language Models☆1,703Mar 8, 2024Updated 2 years ago
- [NeurIPS 2024] Official code of $\beta$-DPO: Direct Preference Optimization with Dynamic $\beta$☆52Oct 23, 2024Updated 2 years ago
- The code for paper "FloE: On-the-Fly MoE Inference on Memory-constrained GPU“.☆12Apr 1, 2025Updated last year
- The code for the paper "ECR-Chain: Advancing Generative Language Models to Better Emotion Cause Reasoners through Reasoning Chains" (IJCA…☆13May 4, 2024Updated 2 years ago
- Official code for the paper "HEXA-MoE: Efficient and Heterogeneous-Aware MoE Acceleration with Zero Computation Redundancy"☆16Mar 6, 2025Updated last year
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- [ICLR 2025] Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization☆26Oct 5, 2025Updated last year
- Code release for AdapMoE accepted by ICCAD 2024☆40Apr 28, 2025Updated last year
- [ICLR26] AI-based scaling law discovery☆37Jan 30, 2026Updated 9 months ago
- ☆16Aug 14, 2022Updated 4 years ago
- ⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training (EMNLP 2024)☆1,005Dec 6, 2024Updated last year
- 现代汉语词义消歧数据集☆11Dec 4, 2024Updated last year
- DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models☆2,001Jan 16, 2024Updated 2 years ago