Implementations of a Mixture-of-Experts (MoE) architecture designed for research on large language models (LLMs) and scalable neural network designs. One implementation targets a **single-device/NPU environment** while the other is built for multi-device distributed computing. Both versions showcase the core principles.
☆84Apr 8, 2025Updated last year
Alternatives and similar repositories for MoE-Mixture-of-Experts-in-PyTorch
Users that are interested in MoE-Mixture-of-Experts-in-PyTorch are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Simple MoE - Day 17 of 365 Days of Repos☆20Jun 2, 2026Updated 3 months ago
- Mamba R1 represents a novel architecture that combines the efficiency of Mamba's state space models with the scalability of Mixture of Ex…☆26Oct 13, 2025Updated 11 months ago
- Awesome Mixture of Experts (MoE): A Curated List of Mixture of Experts (MoE) and Mixture of Multimodal Experts (MoME)☆74Jul 13, 2026Updated 2 months ago
- CRAI is a multimodal large language model based on the Mixture of Experts (MoE) architecture, supporting text and image cross-modal tasks…☆16Apr 29, 2025Updated last year
- Solving Token Gradient Conflict in Mixture-of-Experts for Large Vision-Language Model☆13Feb 11, 2025Updated last year
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- Scaling Laws for Mixture of Experts Models☆16Feb 25, 2025Updated last year
- [ICML 2025] Retraining-Free Merging of Sparse MoE via Hierarchical Clustering☆25Oct 26, 2025Updated 10 months ago
- Mixture-of-Experts Multimodal Variational Autoencoder☆15Jul 3, 2025Updated last year
- Spatial Spectral Machine Learning☆14Oct 15, 2025Updated 11 months ago
- Tools and Scripts for running Isaac Sim workloads on Omniverse Farm☆12Jun 15, 2025Updated last year
- Jeroen Cottaar's work for the Kaggle Geophysical Waveform Inversion competition (2nd place)☆14Aug 11, 2025Updated last year
- Conditionally Site-Independent Neural Evolution of Antibody Sequences (ICML '26)☆24Aug 30, 2026Updated 3 weeks ago
- Easy & Pretrained SOTA Deep Learning for RNA strings☆12Apr 15, 2022Updated 4 years ago
- This repository contains the code for the paper "ST-MoE-BERT: A Spatial-Temporal Mixture-of-Experts Framework for Long-Term Cross-City Mo…☆16Feb 20, 2025Updated last year
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Code for "Discovering Non-monotonic Autoregressive Orderings with Variational Inference" (paper and code updated from ICLR 2021)☆12Mar 7, 2024Updated 2 years ago
- A scalable, clean architecture, ready-to-use NestJs boilerplate☆11May 9, 2024Updated 2 years ago
- Multi-encoder segmentation for contrail detection in satellite imagery | Google Researc☆12Jan 28, 2026Updated 7 months ago
- MoE-Visualizer is a tool designed to visualize the selection of experts in Mixture-of-Experts (MoE) models.☆16Apr 8, 2025Updated last year
- [ICLR 2025] Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization☆25Oct 5, 2025Updated 11 months ago
- [ICLR 2022] Denoising Likelihood Score Matching for Conditional Score-based Data Generation☆12Jun 15, 2026Updated 3 months ago
- Transformer + GAT for RNA chemical reactivity prediction| Stanford Ribonanza☆11Jan 28, 2026Updated 7 months ago
- Large-scale Exploration of Neural Relation Classification Architectures☆11Nov 15, 2018Updated 7 years ago
- Insular is a software architecture to build scalable Laravel projects in a modern, clear and modular way.☆11Nov 4, 2022Updated 3 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- A Scalable Chat Application, built using reactjs and express, and the architecture consists of 5 services. Used Redis and kafka for event…☆10Feb 20, 2024Updated 2 years ago
- [ICML 2025] Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions☆16Mar 7, 2026Updated 6 months ago
- This is a simple torch implementation of the high performance Multi-Query Attention☆16Aug 23, 2023Updated 3 years ago
- Code repository for the manuscript: Nucleotide dependency analysis of DNA language models reveals genomic functional elements☆30Aug 3, 2026Updated last month
- Implementation of the SOTA Transformer architecture from PaLM - Scaling Language Modeling with Pathways in JAX/Flax☆13Jun 22, 2022Updated 4 years ago
- This repository features a modular HMVC (Hierarchical Model-View-Controller) architecture for CodeIgniter 4, integrated with the official…☆15Mar 20, 2026Updated 6 months ago
- A powerful framework for decentralized federated learning with user-defined communication topology☆12Dec 14, 2021Updated 4 years ago
- NodeJS Clean Architecture. Clean Architecture is an opinionated boilerplate for Node web APIs focused on separation of concerns and scal…☆11Jul 1, 2019Updated 7 years ago
- A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models☆873Sep 13, 2023Updated 3 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- Generating Sentences from Disentangled Syntactic and Semantic Spaces☆11Jun 24, 2019Updated 7 years ago
- ☆11Jun 26, 2015Updated 11 years ago
- A curated biochemical database that integrates and refines data from KEGG and ATLAS databases to support precise analyses of biochemical …☆16May 13, 2026Updated 4 months ago
- Official implementation of Neuronal Time-Invariant Representations (NeuPRINT), NeurIPS 2023☆10Mar 10, 2026Updated 6 months ago
- This repository contains the jupyter and mathematica notebooks of the Transfer Learning Model on VGG16, ResNet 50 architecture with Image…☆10Jun 3, 2019Updated 7 years ago
- A modular, scalable, and maintainable Spring Boot microservices application demonstrating a HexaLayered Architecture for phone directory …☆12Feb 3, 2025Updated last year
- Post-Trained MoE Can Skip Half Experts via Self-Distillation☆41Sep 6, 2026Updated 2 weeks ago