Official implementation of RMoE (Layerwise Recurrent Router for Mixture-of-Experts)
☆33Aug 4, 2024Updated 2 years ago
Alternatives and similar repositories for RMoE
Users that are interested in RMoE are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- [COLM 2025] "C3PO: Critical-Layer, Core-Expert, Collaborative Pathway Optimization for Test-Time Expert Re-Mixing"☆21Apr 9, 2025Updated last year
- ☆26Oct 22, 2025Updated 11 months ago
- ☆29May 24, 2024Updated 2 years ago
- [ICLR2025] Codebase for "ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing", built on Megatron-LM.☆123Dec 20, 2024Updated last year
- Inference Code for Paper "Harder Tasks Need More Experts: Dynamic Routing in MoE Models"☆74Jul 30, 2024Updated 2 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆17Jun 11, 2025Updated last year
- ☆15Mar 20, 2025Updated last year
- ☆93Aug 18, 2024Updated 2 years ago
- ☆16Jul 30, 2025Updated last year
- [EMNLP 2024] Quantize LLM to extremely low-bit, and finetune the quantized LLMs☆17Jul 18, 2024Updated 2 years ago
- This repository contains the code for the IEEE Robotics and Automation Letters paper "Open-Set Object Detection Using Classification-Free…☆16Dec 6, 2023Updated 2 years ago
- ✨✨VITA: Towards Open-Source Interactive Omni Multimodal LLM☆11Jun 16, 2025Updated last year
- Offical implementation of "MetaLA: Unified Optimal Linear Approximation to Softmax Attention Map" (NeurIPS2024 Oral)☆36Jan 18, 2025Updated last year
- Solving Token Gradient Conflict in Mixture-of-Experts for Large Vision-Language Model☆13Feb 11, 2025Updated last year
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Dialog2Flow: convert your dialogs to flows. This repository accompanies the paper "Dialog2Flow: Pre-training Soft-Contrastive Sentence Em…☆20Jul 1, 2025Updated last year
- [ICLR 2025] Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models☆164Jul 9, 2025Updated last year
- [ICCV 2025] Dynamic-VLM☆28Dec 16, 2024Updated last year
- Community Implementation of the paper: "Multi-Head Mixture-of-Experts" In PyTorch☆31Oct 2, 2026Updated last week
- Overflow Prevention Enhances Long-Context Recurrent LLMs (COLM 2025)☆18Jul 8, 2025Updated last year
- Official implementation of Vector-ICL: In-context Learning with Continuous Vector Representations (ICLR 2025)☆25Jun 2, 2025Updated last year
- Code for Paper 'DelTA: Discriminative Token Credit Assignment for Reinforcement Learning from Verifiable Rewards'☆18May 21, 2026Updated 4 months ago
- Image Tokenizer Needs Post-Training☆24Oct 4, 2025Updated last year
- Mixture of Lora Experts☆11Apr 7, 2024Updated 2 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- [EMNLP 2025 Outstanding Paper Award] Official repo for DiscoSG: Towards Discourse-Level Text Scene Graph Parsing through Iterative Graph …☆22Nov 16, 2025Updated 10 months ago
- [ICLR 2026] Geometric-Mean Policy Optimization☆106Jan 26, 2026Updated 8 months ago
- ☆15Oct 19, 2024Updated last year
- ☆20May 28, 2025Updated last year
- ☆64Mar 3, 2025Updated last year
- [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models☆35Jun 12, 2024Updated 2 years ago
- Work in progress.☆82Nov 25, 2025Updated 10 months ago
- Official implementation of Scaling Laws in Patchification: An Image Is Worth 50,176 Tokens And More☆26Feb 25, 2025Updated last year
- Non-official implementation of "Attention as an RNN" from https://arxiv.org/pdf/2405.13956, efficient associative parallel prefix scan an…☆28Jul 27, 2024Updated 2 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- The open-source materials for paper "Sparsing Law: Towards Large Language Models with Greater Activation Sparsity".☆32Nov 12, 2024Updated last year
- Localize to Binauralize: Audio Spatialization from Visual Sound Source Localization (ICCV 2021)☆10Oct 11, 2021Updated 5 years ago
- The source code for running LLMs on the AAAR-1.0 benchmark.☆20Apr 5, 2025Updated last year
- Chain of Experts (CoE) enables communication between experts within Mixture-of-Experts (MoE) models☆233Nov 4, 2025Updated 11 months ago
- Efficient PScan implementation in PyTorch☆17Jan 2, 2024Updated 2 years ago
- MoIIE: Mixture of Intra- and Inter-Modality Experts for Large Vision Language Models☆20Aug 14, 2025Updated last year
- A Maximal Mutual Information Criterion for Manipulation Concept Discovery☆15Sep 26, 2024Updated 2 years ago