Implementations of a Mixture-of-Experts (MoE) architecture designed for research on large language models (LLMs) and scalable neural network designs. One implementation targets a **single-device/NPU environment** while the other is built for multi-device distributed computing. Both versions showcase the core principles.
☆83Apr 8, 2025Updated last year
Alternatives and similar repositories for MoE-Mixture-of-Experts-in-PyTorch
Users that are interested in MoE-Mixture-of-Experts-in-PyTorch are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Simple MoE - Day 17 of 365 Days of Repos☆20Jun 2, 2026Updated 2 months ago
- An efficient and scalable attention module designed to reduce memory usage and improve inference speed in large language models. Designe…☆26Jun 25, 2025Updated last year
- Mamba R1 represents a novel architecture that combines the efficiency of Mamba's state space models with the scalability of Mixture of Ex…☆25Oct 13, 2025Updated 10 months ago
- Awesome Mixture of Experts (MoE): A Curated List of Mixture of Experts (MoE) and Mixture of Multimodal Experts (MoME)☆73Jul 13, 2026Updated last month
- Collection of useful kernels☆20Oct 3, 2025Updated 10 months ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- This repository enables inference and sampling for Dyno Psi-1, a de novo miniprotein binder design model.☆24Updated this week
- [ICML 2025] Retraining-Free Merging of Sparse MoE via Hierarchical Clustering☆25Oct 26, 2025Updated 10 months ago
- Mixture-of-Experts Multimodal Variational Autoencoder☆15Jul 3, 2025Updated last year
- Jeroen Cottaar's work for the Kaggle Geophysical Waveform Inversion competition (2nd place)☆14Aug 11, 2025Updated last year
- Conditionally Site-Independent Neural Evolution of Antibody Sequences (ICML '26)☆24Updated this week
- DUNL - Neuron 2025☆27Jan 18, 2026Updated 7 months ago
- Plasmid.ai is the largest open-source toolkit for developing plasmid foundation models.☆14Jan 22, 2025Updated last year
- Easy & Pretrained SOTA Deep Learning for RNA strings☆12Apr 15, 2022Updated 4 years ago
- Code for "Improving Robustness of Vision Transformers by Reducing Sensitivity to Patch Corruptions"☆14Sep 3, 2023Updated 2 years ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- Chain of Experts (CoE) enables communication between experts within Mixture-of-Experts (MoE) models☆231Nov 4, 2025Updated 9 months ago
- Multi-encoder segmentation for contrail detection in satellite imagery | Google Researc☆12Jan 28, 2026Updated 7 months ago
- 2nd Place Solution for the Google Research - Identify Contrails to Reduce Global Warming Competition☆14Aug 15, 2023Updated 3 years ago
- ☆35Aug 15, 2026Updated 2 weeks ago
- [ICLR 2022] Denoising Likelihood Score Matching for Conditional Score-based Data Generation☆11Jun 15, 2026Updated 2 months ago
- An awesome list of Causality and Machine Learning related papers, books and other resources.☆12Nov 13, 2023Updated 2 years ago
- [ICML 2025] Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions☆16Mar 7, 2026Updated 5 months ago
- Graph-based representation learning method for protein function prediction☆24Aug 25, 2025Updated last year
- This is a simple torch implementation of the high performance Multi-Query Attention☆16Aug 23, 2023Updated 3 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models☆869Sep 13, 2023Updated 2 years ago
- Generating Sentences from Disentangled Syntactic and Semantic Spaces☆11Jun 24, 2019Updated 7 years ago
- A curated biochemical database that integrates and refines data from KEGG and ATLAS databases to support precise analyses of biochemical …☆16May 13, 2026Updated 3 months ago
- CompBioBench: A benchmark of 100 diverse, verifiable questions for agents for computational biology (scripts for running agents)☆51Apr 6, 2026Updated 4 months ago
- Neural ODE Transformers (ICLR 2025)☆21Sep 6, 2025Updated 11 months ago
- ☆12Mar 12, 2026Updated 5 months ago
- ☆30Oct 14, 2024Updated last year
- ☆26Mar 3, 2025Updated last year
- My scalable microservice architecture☆15Jul 10, 2021Updated 5 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- BioDiscoveryAgent is an LLM-based AI agent for closed-loop design of genetic perturbation experiments☆124Jul 6, 2025Updated last year
- Post-Trained MoE Can Skip Half Experts via Self-Distillation☆40May 19, 2026Updated 3 months ago
- ☆17Jul 6, 2026Updated last month
- This repository serves as an illustrative example for developing a scalable chat application using NestJS. The architecture of this appli…☆15Feb 25, 2024Updated 2 years ago
- Python library for serverless Federated Learning experiments.☆16Mar 17, 2023Updated 3 years ago
- ☆31May 5, 2024Updated 2 years ago
- ☆16Mar 24, 2023Updated 3 years ago