Sparse Backpropagation for Mixture-of-Expert Training
☆30Jul 2, 2024Updated 2 years ago
Alternatives and similar repositories for SparseMixer
Users that are interested in SparseMixer are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A Vector Caching Scheme for Streaming FPGA SpMV Accelerators☆10Sep 7, 2015Updated 11 years ago
- Official Implementation of "RTop-K: Ultra-Fast Row-Wise Top-K Selection for Neural Network Acceleration on GPUs"☆30Jul 23, 2025Updated last year
- ☆17May 19, 2026Updated 6 months ago
- [ACL 2023 Findings] Emergent Modularity in Pre-trained Transformers☆26Jun 7, 2023Updated 3 years ago
- GRadient-INformed MoE☆263Sep 25, 2024Updated 2 years ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- Facebook AI Research Sequence-to-Sequence Toolkit written in Python.☆22Jan 25, 2023Updated 3 years ago
- MATLAB code for performing the coalescent embedding☆13May 23, 2023Updated 3 years ago
- ☆13Oct 13, 2025Updated last year
- Official implementation for DenseMixer: Improving MoE Post-Training with Precise Router Gradient☆68Aug 3, 2025Updated last year
- Official implementation of Vector-ICL: In-context Learning with Continuous Vector Representations (ICLR 2025)☆25Jun 2, 2025Updated last year
- large language model internal-medicine monitor toolbox☆17Oct 28, 2026Updated 3 weeks ago
- ☆20Mar 25, 2025Updated last year
- Mamba support for transformer lens☆20Sep 17, 2024Updated 2 years ago
- ☆21May 5, 2020Updated 6 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- ☆13Apr 13, 2026Updated 7 months ago
- Official PyTorch Implementation for Paper "No More Adam: Learning Rate Scaling at Initialization is All You Need"☆54Jan 27, 2025Updated last year
- ☆12Apr 27, 2013Updated 13 years ago
- ☆20Nov 5, 2024Updated 2 years ago
- Triton-based implementation of Sparse Mixture of Experts.☆282Oct 3, 2025Updated last year
- The source code of "Merging Experts into One: Improving Computational Efficiency of Mixture of Experts (EMNLP 2023)":☆49Oct 2, 2026Updated last month
- [ICML 2024] Code for the paper "MoE-RBench: Towards Building Reliable Language Models with Sparse Mixture-of-Experts"☆11Jul 1, 2024Updated 2 years ago
- kubernetes client to☆11May 27, 2022Updated 4 years ago
- [COLM'25] A Controlled Study on Long Context Extension and Generalization in LLMs☆65Mar 9, 2026Updated 8 months ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- ☆21Apr 16, 2025Updated last year
- The open-source materials for paper "Sparsing Law: Towards Large Language Models with Greater Activation Sparsity".☆32Nov 12, 2024Updated 2 years ago
- Transformers components but in Triton☆35May 9, 2025Updated last year
- Exploring context retrieval strategies for code editing☆13Sep 23, 2024Updated 2 years ago
- Code for "Fine-Tuning by Curriculum Learning for Non-Autoregressive Neural Machine Translation"☆13Jul 10, 2020Updated 6 years ago
- High-performance RDMA for Python and PyTorch (ibverbs, efa, NVMe-oF)☆14Sep 2, 2026Updated 2 months ago
- ☆22Apr 17, 2025Updated last year
- The official repo of our research work "Interactive Editing for Text Summarization".☆23Jun 3, 2023Updated 3 years ago
- ☆21Dec 14, 2024Updated last year
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- [ICLR 2025] Monet: Mixture of Monosemantic Experts for Transformers☆79Jun 23, 2025Updated last year
- Open-Pandora: On-the-fly Control Video Generation☆35Nov 28, 2024Updated last year
- Contrast Subgraph Mining from Coherent Cores☆13Feb 20, 2018Updated 8 years ago
- Codes for "Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspective" (AAAI 2025)☆12Dec 5, 2023Updated 2 years ago
- ☆13Apr 3, 2024Updated 2 years ago
- ☆146Jul 21, 2024Updated 2 years ago
- Official repo for "Cardiverse: Harnessing LLMs for Novel Card Game Prototyping" at EMNLP 2025☆16Apr 18, 2026Updated 7 months ago
Are these results useful?