Implementation of Switch Transformers from the paper: "Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity"
☆143Jun 22, 2026Updated 2 weeks ago
Alternatives and similar repositories for SwitchTransformers
Users that are interested in SwitchTransformers are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- [ECCV 2024] Official pytorch implementation of "Switch Diffusion Transformer: Synergizing Denoising Tasks with Sparse Mixture-of-Experts"☆47Jul 4, 2024Updated 2 years ago
- A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models☆862Sep 13, 2023Updated 2 years ago
- Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch☆385Jun 17, 2024Updated 2 years ago
- ☆15Oct 19, 2024Updated last year
- PyTorch Re-Implementation of "The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer et al. https://arxiv.org/abs/1701.06538☆1,248Apr 19, 2024Updated 2 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Code for the paper "Stack Attention: Improving the Ability of Transformers to Model Hierarchical Patterns"☆18Mar 15, 2024Updated 2 years ago
- [ICLR'25] "Understanding Bottlenecks of State Space Models through the Lens of Recency and Over-smoothing" by Peihao Wang, Ruisi Cai, Yue…☆18Mar 21, 2025Updated last year
- ☆17Mar 18, 2026Updated 3 months ago
- PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)☆83Oct 5, 2023Updated 2 years ago
- Deft: A Scalable Tree Index for Disaggregated Memory☆22Apr 23, 2025Updated last year
- Code from paper "Natural language supervision with a large and diverse dataset builds better models of human high-level visual cortex"☆25Feb 6, 2024Updated 2 years ago
- [ICML 2024] Official Repository for the paper "Transformers Get Stable: An End-to-End Signal Propagation Theory for Language Models"☆10Jul 19, 2024Updated last year
- Sample implementations of the Strassen algorithm for matrix multiplication☆12Nov 26, 2023Updated 2 years ago
- A collection of AWESOME things about mixture-of-experts☆1,281Dec 8, 2024Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- This is the implementation of the paper "Pre-training Time Series Models with Stock Data Customization"☆48May 30, 2025Updated last year
- ☆725Jul 2, 2026Updated last week
- ☆56Dec 31, 2025Updated 6 months ago
- Reinforcement Learning from Text Feedback☆48Feb 17, 2026Updated 4 months ago
- This repo has moved to https://github.com/haosulab/ManiSkill☆18May 28, 2025Updated last year
- Source code for "To Adapt or Not to Adapt? Real-Time Adaptation for Semantic Segmentation", ICCV 2023☆49Jun 20, 2024Updated 2 years ago
- ☆30Sep 28, 2023Updated 2 years ago
- Open-sourcing code associated with the AAAI-25 paper "On the Expressiveness and Length Generalization of Selective State-Space Models on …☆16Sep 18, 2025Updated 9 months ago
- PyTorch Implementation of DCENet for Trajectory Forecasting☆13Jun 5, 2021Updated 5 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Automate hyper-parameters tuning for NNs (learning rate, number of dense layers and nodes and activation function)☆14Aug 9, 2020Updated 5 years ago
- Deferred Continuous Batching in Resource-Efficient Large Language Model Serving (EuroMLSys 2024)☆19May 28, 2024Updated 2 years ago
- A simple, lightweight, and efficient solution for multi-object trajectory prediction☆15Apr 28, 2026Updated 2 months ago
- Official PyTorch implementation of The Linear Attention Resurrection in Vision Transformer☆15Sep 7, 2024Updated last year
- Implementation of Soft MoE, proposed by Brain's Vision team, in Pytorch☆347Apr 2, 2025Updated last year
- [NeurIPS 2024 Spotlight] code for "Diffusion Model with Cross Attention as an Inductive Bias for Disentanglement"☆20Jan 26, 2025Updated last year
- A pipeline for the automatic construction of geometry problems along with step-by-step solutions.☆17Aug 27, 2025Updated 10 months ago
- Artifacts for our ASPLOS'23 paper ElasticFlow☆56May 10, 2024Updated 2 years ago
- [ICCV 2025] CoDa-4DGS: Dynamic Gaussian Splatting with Context and Deformation Awareness for Autonomous Driving☆39Aug 28, 2025Updated 10 months ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- [IROS 2021] ADD: A Fine-grained Dynamic Inference Architecture for Semantic Image Segmentation☆10May 3, 2022Updated 4 years ago
- ☆14Nov 13, 2022Updated 3 years ago
- An automated data pipeline scaling RL to pretraining levels☆76Jun 2, 2026Updated last month
- Towards Understanding the Mixture-of-Experts Layer in Deep Learning☆35Dec 12, 2023Updated 2 years ago
- Implementation of MoE Mamba from the paper: "MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts" in Pytorch and Ze…☆132Jun 22, 2026Updated 2 weeks ago
- Implementation of the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"☆122Jun 22, 2026Updated 2 weeks ago
- Learning to Skip the Middle Layers of Transformers☆17Aug 7, 2025Updated 11 months ago