Official implementation of RMoE (Layerwise Recurrent Router for Mixture-of-Experts)
☆33Aug 4, 2024Updated last year
Alternatives and similar repositories for RMoE
Users that are interested in RMoE are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- [COLM 2025] "C3PO: Critical-Layer, Core-Expert, Collaborative Pathway Optimization for Test-Time Expert Re-Mixing"☆21Apr 9, 2025Updated last year
- ☆23Oct 22, 2025Updated 9 months ago
- ☆29May 24, 2024Updated 2 years ago
- [ICLR2025] Codebase for "ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing", built on Megatron-LM.☆118Dec 20, 2024Updated last year
- ☆16Jun 11, 2025Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Official PyTorch Implementation of EMoE: Unlocking Emergent Modularity in Large Language Models [main conference @ NAACL2024]☆39May 28, 2024Updated 2 years ago
- ☆15Mar 20, 2025Updated last year
- ☆93Aug 18, 2024Updated last year
- ☆15Jul 30, 2025Updated 11 months ago
- [EMNLP 2024] Quantize LLM to extremely low-bit, and finetune the quantized LLMs☆15Jul 18, 2024Updated 2 years ago
- This repository contains the code for the IEEE Robotics and Automation Letters paper "Open-Set Object Detection Using Classification-Free…☆16Dec 6, 2023Updated 2 years ago
- ✨✨VITA: Towards Open-Source Interactive Omni Multimodal LLM☆11Jun 16, 2025Updated last year
- Offical implementation of "MetaLA: Unified Optimal Linear Approximation to Softmax Attention Map" (NeurIPS2024 Oral)☆36Jan 18, 2025Updated last year
- Solving Token Gradient Conflict in Mixture-of-Experts for Large Vision-Language Model☆13Feb 11, 2025Updated last year
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- [ICLR 2025] Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models☆161Jul 9, 2025Updated last year
- Dialog2Flow: convert your dialogs to flows. This repository accompanies the paper "Dialog2Flow: Pre-training Soft-Contrastive Sentence Em…☆20Jul 1, 2025Updated last year
- [ICCV 2025] Dynamic-VLM☆28Dec 16, 2024Updated last year
- Overflow Prevention Enhances Long-Context Recurrent LLMs (COLM 2025)☆18Jul 8, 2025Updated last year
- ☆18Mar 21, 2025Updated last year
- Code for Paper 'DelTA: Discriminative Token Credit Assignment for Reinforcement Learning from Verifiable Rewards'☆17May 21, 2026Updated 2 months ago
- Image Tokenizer Needs Post-Training☆24Oct 4, 2025Updated 9 months ago
- Mixture of Lora Experts☆11Apr 7, 2024Updated 2 years ago
- [NeurIPS 2024] Efficiency for Free: Ideal Data Are Transportable Representations☆19Jan 19, 2025Updated last year
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- [EMNLP 2025 Outstanding Paper Award] Official repo for DiscoSG: Towards Discourse-Level Text Scene Graph Parsing through Iterative Graph …☆22Nov 16, 2025Updated 8 months ago
- [ICLR 2026] Geometric-Mean Policy Optimization☆104Jan 26, 2026Updated 6 months ago
- ☆19Nov 11, 2024Updated last year
- ☆15Oct 19, 2024Updated last year
- ☆20May 28, 2025Updated last year
- ☆62Mar 3, 2025Updated last year
- [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models☆35Jun 12, 2024Updated 2 years ago
- Tensor Switching Networks☆12Nov 2, 2017Updated 8 years ago
- Work in progress.☆80Nov 25, 2025Updated 8 months ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Official implementation of Scaling Laws in Patchification: An Image Is Worth 50,176 Tokens And More☆25Feb 25, 2025Updated last year
- Non-official implementation of "Attention as an RNN" from https://arxiv.org/pdf/2405.13956, efficient associative parallel prefix scan an…☆28Jul 27, 2024Updated last year
- The open-source materials for paper "Sparsing Law: Towards Large Language Models with Greater Activation Sparsity".☆32Nov 12, 2024Updated last year
- The source code for running LLMs on the AAAR-1.0 benchmark.☆20Apr 5, 2025Updated last year
- [ICML 2025] Diff-MoE: Diffusion Transformer with Time-Aware and Space-Adaptive Experts☆34Nov 10, 2025Updated 8 months ago
- Chain of Experts (CoE) enables communication between experts within Mixture-of-Experts (MoE) models☆231Nov 4, 2025Updated 8 months ago
- Efficient PScan implementation in PyTorch☆17Jan 2, 2024Updated 2 years ago