Implementation of Switch Transformers from the paper: "Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity"
☆148Oct 26, 2026Updated last week
Alternatives and similar repositories for SwitchTransformers
Users that are interested in SwitchTransformers are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models☆877Sep 13, 2023Updated 3 years ago
- Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch☆390Jun 17, 2024Updated 2 years ago
- PyTorch Re-Implementation of "The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer et al. https://arxiv.org/abs/1701.06538☆1,257Apr 19, 2024Updated 2 years ago
- Community Implementation of the paper: "Multi-Head Mixture-of-Experts" In PyTorch☆31Oct 26, 2026Updated last week
- A collection of AWESOME things about mixture-of-experts☆1,287Dec 8, 2024Updated last year
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- ☆728Oct 28, 2026Updated last week
- PyTorch Implementation of Jamba: "Jamba: A Hybrid Transformer-Mamba Language Model"☆223Oct 26, 2026Updated last week
- Simple reimplementation of Flow Matching for Generative Modeling (https://arxiv.org/abs/2210.02747) paper in PyTorch☆24Aug 10, 2024Updated 2 years ago
- [TKDE'25] The official GitHub page for the survey paper "A Survey on Mixture of Experts in Large Language Models".☆512Aug 18, 2026Updated 2 months ago
- Proteus: A High-Throughput Inference-Serving System with Accuracy Scaling☆14Mar 7, 2024Updated 2 years ago
- DepthBench: Measuring How Residual Connections Enable More Computational Depth☆20Updated this week
- [ACCV'26] Revisiting Multi-Task Visual Representation Learning☆23Jan 21, 2026Updated 9 months ago
- Implementation of Soft MoE, proposed by Brain's Vision team, in Pytorch☆349Apr 2, 2025Updated last year
- [ECCV 2024] SparseRefine: Sparse Refinement for Efficient High-Resolution Semantic Segmentation☆17Jan 10, 2025Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- [ICLR'25] "Understanding Bottlenecks of State Space Models through the Lens of Recency and Over-smoothing" by Peihao Wang, Ruisi Cai, Yue…☆18Mar 21, 2025Updated last year
- ☆18Sep 22, 2024Updated 2 years ago
- FlashLoop: Fast and Memory-Efficient Looped Transformers via Lazy Updates☆25Sep 28, 2026Updated last month
- This repository implements the paper "Effective Training of Convolutional Neural Networks with Low-bitwidth Weights and Activations"☆20Aug 30, 2021Updated 5 years ago
- PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)☆83Oct 5, 2023Updated 3 years ago
- The open source implementation of "Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers"☆18Mar 11, 2024Updated 2 years ago
- This is the offical repository for "Multi-modal Gated Mixture of Local-to-Global Experts for Dynamic Image Fusion" (ICCV 2023).☆76Apr 30, 2024Updated 2 years ago
- [ACL 2023 Findings] Emergent Modularity in Pre-trained Transformers☆26Jun 7, 2023Updated 3 years ago
- ☆22Dec 15, 2023Updated 2 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆13Sep 6, 2026Updated 2 months ago
- Official implementation of the paper "AWM: Accurate Weight-Matrix Fingerprint for Large Language Models"☆13Feb 20, 2026Updated 8 months ago
- Contains from-scratch implementation of the MobileNetV1, V2 and V3 paper with PyTorch. Each model architecture is contained in a single f…☆19Aug 10, 2022Updated 4 years ago
- Code for the paper "Stack Attention: Improving the Ability of Transformers to Model Hierarchical Patterns"☆19Mar 15, 2024Updated 2 years ago
- ☆11May 4, 2023Updated 3 years ago
- ☆20Apr 17, 2023Updated 3 years ago
- ☆10Apr 8, 2025Updated last year
- This is an read-only mirror of the gem5 simulator. The upstream repository is stored in https://gem5.googlesource.com, code reviews shoul…☆13May 15, 2020Updated 6 years ago
- Amortized inference in sparse factor graphs☆10Aug 30, 2026Updated 2 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- A simple implementation of LoRA+: Efficient Low Rank Adaptation of Large Models☆10Mar 20, 2024Updated 2 years ago
- Code for ICML25 Paper "Overcoming Non-monotonicity in Transducer-based Streaming Generation"☆12May 19, 2025Updated last year
- ☆11May 12, 2026Updated 5 months ago
- This is the official code for ViTE: Virtual Graph Trajectory Expert Router for Pedestrian Trajectory Prediction☆13Apr 3, 2026Updated 7 months ago
- Multiscale Score Matching Analysis☆12Jan 19, 2023Updated 3 years ago
- [WACV 2026] Beyond Paired Data: Self-Supervised UAV Geo-Localization from Reference Imagery Alone☆15Mar 31, 2026Updated 7 months ago
- [AAAI 2026]Fine-Grained Representation for Lane Topology Reasoning☆15Nov 29, 2025Updated 11 months ago