Implementations of a Mixture-of-Experts (MoE) architecture designed for research on large language models (LLMs) and scalable neural network designs. One implementation targets a **single-device/NPU environment** while the other is built for multi-device distributed computing. Both versions showcase the core principles.
☆77Apr 8, 2025Updated last year
Alternatives and similar repositories for MoE-Mixture-of-Experts-in-PyTorch
Users that are interested in MoE-Mixture-of-Experts-in-PyTorch are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Simple MoE - Day 17 of 365 Days of Repos☆20Jun 2, 2026Updated last month
- An efficient and scalable attention module designed to reduce memory usage and improve inference speed in large language models. Designe…☆24Jun 25, 2025Updated last year
- Mamba R1 represents a novel architecture that combines the efficiency of Mamba's state space models with the scalability of Mixture of Ex…☆25Oct 13, 2025Updated 9 months ago
- CRAI is a multimodal large language model based on the Mixture of Experts (MoE) architecture, supporting text and image cross-modal tasks…☆16Apr 29, 2025Updated last year
- The code of 《M4: Multi-Proxy Multi-Gate Mixture of Experts Network for Multiple Instance Learning in Histopathology Image Analysis》☆14Mar 31, 2025Updated last year
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Scaling Laws for Mixture of Experts Models☆15Feb 25, 2025Updated last year
- [ICML 2025] Code for "R2-T2: Re-Routing in Test-Time for Multimodal Mixture-of-Experts"☆19Mar 10, 2025Updated last year
- [ICML 2025] Retraining-Free Merging of Sparse MoE via Hierarchical Clustering☆25Oct 26, 2025Updated 8 months ago
- Spatial Spectral Machine Learning☆14Oct 15, 2025Updated 9 months ago
- Self-made Alfred workflows☆14Dec 9, 2025Updated 7 months ago
- Conditionally Site-Independent Neural Evolution of Antibody Sequences (ICML '26)☆24May 9, 2026Updated 2 months ago
- DUNL - Neuron 2025☆27Jan 18, 2026Updated 6 months ago
- Plasmid.ai is the largest open-source toolkit for developing plasmid foundation models.☆14Jan 22, 2025Updated last year
- A Python toolkit for data-driven HRTF research☆16Feb 6, 2025Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- This repository contains the code for the paper "ST-MoE-BERT: A Spatial-Temporal Mixture-of-Experts Framework for Long-Term Cross-City Mo…☆16Feb 20, 2025Updated last year
- Code for "Discovering Non-monotonic Autoregressive Orderings with Variational Inference" (paper and code updated from ICLR 2021)☆12Mar 7, 2024Updated 2 years ago
- Code for "Improving Robustness of Vision Transformers by Reducing Sensitivity to Patch Corruptions"☆14Sep 3, 2023Updated 2 years ago
- Chain of Experts (CoE) enables communication between experts within Mixture-of-Experts (MoE) models☆231Nov 4, 2025Updated 8 months ago
- Multi-encoder segmentation for contrail detection in satellite imagery | Google Researc☆12Jan 28, 2026Updated 5 months ago
- Code for Remember and Reuse: Cross-Task Blind Image Quality Assessment via Relevance-aware Incremental Learning (ACM Multimedia 2021)☆13Sep 16, 2021Updated 4 years ago
- ☆30Jul 6, 2026Updated 2 weeks ago
- ☆12Apr 18, 2025Updated last year
- Data simulation scripts for paper "Target Sound Extraction with Variable Cross-modality Clues"☆17May 19, 2023Updated 3 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆15Jan 9, 2026Updated 6 months ago
- Federated learning used to detect credit card fraud☆13Jun 23, 2020Updated 6 years ago
- ☆15Dec 14, 2025Updated 7 months ago
- Demo showing how to package and execute a Federated Learning jobs on Nevermined.☆12Dec 6, 2022Updated 3 years ago
- ☆16Mar 31, 2025Updated last year
- This is a simple torch implementation of the high performance Multi-Query Attention☆16Aug 23, 2023Updated 2 years ago
- Neural IIR Filter Field for HRTF Upsampling and Personalization☆29Feb 26, 2024Updated 2 years ago
- Code repository for the manuscript: Nucleotide dependency analysis of DNA language models reveals genomic functional elements☆30Mar 13, 2025Updated last year
- A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models☆866Sep 13, 2023Updated 2 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- ☆11Nov 13, 2022Updated 3 years ago
- [CVPR 2025] Lifelong Knowledge Editing for Vision Language Models with Low-Rank Mixture-of-Experts☆24Jun 22, 2025Updated last year
- ☆10Jun 26, 2015Updated 11 years ago
- A curated biochemical database that integrates and refines data from KEGG and ATLAS databases to support precise analyses of biochemical …☆16May 13, 2026Updated 2 months ago
- ☆12Mar 12, 2026Updated 4 months ago
- Official implementation of Neuronal Time-Invariant Representations (NeuPRINT), NeurIPS 2023☆10Mar 10, 2026Updated 4 months ago
- ☆25Mar 3, 2025Updated last year