PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)
☆83Oct 5, 2023Updated 2 years ago
Alternatives and similar repositories for soft-mixture-of-experts
Users that are interested in soft-mixture-of-experts are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- PyTorch implementation of "From Sparse to Soft Mixtures of Experts"☆72Aug 22, 2023Updated 2 years ago
- Implementation of Soft MoE, proposed by Brain's Vision team, in Pytorch☆347Apr 2, 2025Updated last year
- Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch☆385Jun 17, 2024Updated 2 years ago
- ☆725Jul 2, 2026Updated last week
- AdaMoLE: Adaptive Mixture of LoRA Experts☆38Oct 11, 2024Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Unofficial Implementation of Selective Attention Transformer☆20Oct 31, 2024Updated last year
- A collection of AWESOME things about mixture-of-experts☆1,281Dec 8, 2024Updated last year
- Implementation of MixCE method described in ACL 2023 paper by Zhang et al.☆20May 29, 2023Updated 3 years ago
- Federated Learning - PyTorch☆15Jun 27, 2021Updated 5 years ago
- A fork from the official Madrona. The fork will be used for Genesis's own Madrona integration.☆26Updated this week
- Dynamic Neural Representational Decoders for High-Resolution Semantic Segmentation☆19Nov 28, 2022Updated 3 years ago
- ☆25Aug 2, 2024Updated last year
- sigma-MoE layer☆21Jan 5, 2024Updated 2 years ago
- GPT-J 6B inference on TensorRT with INT-8 precision☆11Apr 5, 2023Updated 3 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Implementation of "Towards Understanding Mixture of Experts in Deep Learning", NeurIPS 2022☆10Jan 6, 2023Updated 3 years ago
- RS Generate dataset☆18Jan 2, 2025Updated last year
- Reimplementation of https://github.com/montemac/algebraic_value_editing in pure PyTorch for efficiency on large models☆11Jun 28, 2023Updated 3 years ago
- Provably (and non-vacuously) bounding test error of deep neural networks under distribution shift with unlabeled test data.☆10Feb 27, 2024Updated 2 years ago
- Towards Understanding the Mixture-of-Experts Layer in Deep Learning☆35Dec 12, 2023Updated 2 years ago
- UWSOD: Toward Fully-Supervised-Level Capacity Weakly Supervised Object Detection☆23Mar 30, 2021Updated 5 years ago
- Mixture of Attention Heads☆53Oct 10, 2022Updated 3 years ago
- ☆24Dec 18, 2025Updated 6 months ago
- Pytorch-based adaptive deformable convolution☆17Jun 26, 2021Updated 5 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Links to publications that focus on the interpretation and analysis of in-context learning☆14Oct 17, 2024Updated last year
- A curated reading list of research in Mixture-of-Experts(MoE).☆669Oct 30, 2024Updated last year
- ☆95Apr 3, 2023Updated 3 years ago
- optimize neuro-centric parameters instead of weights to solve RL tasks☆14Oct 2, 2023Updated 2 years ago
- ☆14Jun 11, 2024Updated 2 years ago
- Machine learning project using federated learning for text generation☆11May 5, 2024Updated 2 years ago
- [CVPR 2025] Official implementation of paper "MoVE-KD: Knowledge Distillation for VLMs with Mixture of Visual Encoders".☆52Jun 7, 2025Updated last year
- [NeurIPS 2024 Spotlight] code for "Diffusion Model with Cross Attention as an Inductive Bias for Disentanglement"☆20Jan 26, 2025Updated last year
- [ICCV' 23] MRM: Masked Relation Modeling for Medical Image Pre-Training with Genetics☆10Oct 28, 2024Updated last year
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- This repository hosts code for converting the original MLP Mixer models (JAX) to TensorFlow.☆15Sep 29, 2021Updated 4 years ago
- ☆19Jun 10, 2024Updated 2 years ago
- ICCV2021 (poster)☆74Oct 27, 2021Updated 4 years ago
- Integrates Imbue's Cost Aware pareto-Region Bayesian Search (CARBS) with Weights and Biases (WanDB)☆12Mar 17, 2025Updated last year
- ☆10Mar 4, 2024Updated 2 years ago
- some mixture of experts architecture implementations☆27Mar 22, 2024Updated 2 years ago
- PyTorch implementation of 'Learning from Simulated and Unsupervised Images through Adversarial Training'☆16Jun 16, 2020Updated 6 years ago