Implementations of a Mixture-of-Experts (MoE) architecture designed for research on large language models (LLMs) and scalable neural network designs. One implementation targets a **single-device/NPU environment** while the other is built for multi-device distributed computing. Both versions showcase the core principles.
☆85Apr 8, 2025Updated last year
Alternatives and similar repositories for MoE-Mixture-of-Experts-in-PyTorch
Users that are interested in MoE-Mixture-of-Experts-in-PyTorch are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆11Jun 16, 2026Updated 4 months ago
- An efficient and scalable attention module designed to reduce memory usage and improve inference speed in large language models. Designe…☆29Jun 25, 2025Updated last year
- HydraNet is a state-of-the-art transformer architecture that combines Multi-Query Attention (MQA), Mixture of Experts (MoE), and continuo…☆14Oct 26, 2026Updated last week
- Solving Token Gradient Conflict in Mixture-of-Experts for Large Vision-Language Model☆13Feb 11, 2025Updated last year
- The papers of point cloud.☆14Jan 20, 2022Updated 4 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- [ICML 2025] Retraining-Free Merging of Sparse MoE via Hierarchical Clustering☆24Oct 26, 2025Updated last year
- A high-performance PyTorch toolkit for pre-training modern, Llama-style language models. Based on nanoGPT with significant architectural …☆14Jan 30, 2026Updated 9 months ago
- The source code for WebConf 2025 paper "Multimodal Graph-Based Variational Mixture of Experts Network for Zero-Shot Multimodal Informatio…☆11Feb 17, 2025Updated last year
- GhostType is a sleek and minimalistic typing test app designed to enhance your typing speed and accuracy. It features real-time performan…☆12Mar 18, 2025Updated last year
- A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models☆877Sep 13, 2023Updated 3 years ago
- Identifying amino acids important to AlphaFold!☆11Oct 20, 2026Updated 2 weeks ago
- Baseline systems of the VoiceMOS Challenge 2026☆15May 18, 2026Updated 5 months ago
- SWIFT: Shared WaIt Free Transmission☆10May 3, 2023Updated 3 years ago
- The code of 《M4: Multi-Proxy Multi-Gate Mixture of Experts Network for Multiple Instance Learning in Histopathology Image Analysis》☆14Mar 31, 2025Updated last year
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Code for the paper “RelaySum for Decentralized Deep Learning on Heterogeneous Data”☆10Apr 21, 2023Updated 3 years ago
- Tools and Scripts for running Isaac Sim workloads on Omniverse Farm☆12Jun 15, 2025Updated last year
- Code for "Discovering Non-monotonic Autoregressive Orderings with Variational Inference" (paper and code updated from ICLR 2021)☆12Mar 7, 2024Updated 2 years ago
- ☆13May 8, 2024Updated 2 years ago
- Conditionally Site-Independent Neural Evolution of Antibody Sequences (ICML '26)☆26Updated this week
- unofficial go sdk for typesafe ai. not affiliated with or endorsed by typesafe ai. a side project built to fill the missing go sdk gap, f…☆13Sep 21, 2026Updated last month
- Rapidly simulate realistic cryo-EM particles☆19May 18, 2023Updated 3 years ago
- Robust Inference-Time Steering of Protein Diffusion Models via Embedding Optimization☆20Apr 27, 2026Updated 6 months ago
- Chain of Experts (CoE) enables communication between experts within Mixture-of-Experts (MoE) models☆233Nov 4, 2025Updated 11 months ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- DUNL - Neuron 2025☆27Jan 18, 2026Updated 9 months ago
- Multi-encoder segmentation for contrail detection in satellite imagery | Google Researc☆11Jan 28, 2026Updated 9 months ago
- ☆21Sep 21, 2026Updated last month
- Fine-tuning Vision Language Model (Qwen2.5-VL-7B) with Hugging Face TRL - Tutorial and implementation for visual question-answering on Ch…☆10Updated this week
- How Post-Training Shapes Biological Reasoning Models☆17Jun 16, 2026Updated 4 months ago
- Personal solutions to the Triton Puzzles☆22Jul 18, 2024Updated 2 years ago
- A powerful AI-powered development assistant CLI that brings intelligent agents and tools directly to your terminal. Build, plan, and exec…☆16Oct 19, 2026Updated 2 weeks ago
- Notes on the Mistral AI model☆22Dec 27, 2023Updated 2 years ago
- Code for Remember and Reuse: Cross-Task Blind Image Quality Assessment via Relevance-aware Incremental Learning (ACM Multimedia 2021)☆13Sep 16, 2021Updated 5 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Algorithms described in the paper Hindsight Credit Assignment (NeurIPS 2019).☆11Oct 27, 2019Updated 7 years ago
- The code for "MoPE: Mixture of Prefix Experts for Zero-Shot Dialogue State Tracking"☆19Jan 25, 2025Updated last year
- Graph-based representation learning method for protein function prediction☆24Aug 25, 2025Updated last year
- ☆17Oct 18, 2023Updated 3 years ago
- Demo showing how to package and execute a Federated Learning jobs on Nevermined.☆12Dec 6, 2022Updated 3 years ago
- Spatial Spectral Machine Learning☆14Oct 15, 2025Updated last year
- [Nature Machine Intelligence 2026] Beyond representational alignment with brain-guided language models for robust reasoning☆15Aug 4, 2026Updated 3 months ago