uclaml / MoELinks
Towards Understanding the Mixture-of-Experts Layer in Deep Learning
☆31Updated last year
Alternatives and similar repositories for MoE
Users that are interested in MoE are comparing it to the libraries listed below
Sorting:
- Official repository of "LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging"☆30Updated 9 months ago
- PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)☆77Updated last year
- State Space Models☆70Updated last year
- Decomposing and Editing Predictions by Modeling Model Computation☆138Updated last year
- Integrating Mamba/SSMs with Transformer for Enhanced Long Context and High-Quality Sequence Modeling☆204Updated 3 weeks ago
- ☆49Updated 7 months ago
- Recycling diverse models☆45Updated 2 years ago
- HGRN2: Gated Linear RNNs with State Expansion☆54Updated last year
- Official repo of Progressive Data Expansion: data, code and evaluation☆29Updated last year
- One Initialization to Rule them All: Fine-tuning via Explained Variance Adaptation☆41Updated 10 months ago
- Official Code for ICLR 2024 Paper: Non-negative Contrastive Learning☆45Updated last year
- MultiModN – Multimodal, Multi-Task, Interpretable Modular Networks (NeurIPS 2023)☆33Updated last year
- The official GitHub page for paper "NegativePrompt: Leveraging Psychology for Large Language Models Enhancement via Negative Emotional St…☆23Updated last year
- [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…☆54Updated 2 years ago
- A regression-alike loss to improve numerical reasoning in language models - ICML 2025☆24Updated 2 weeks ago
- Code and benchmark for the paper: "A Practitioner's Guide to Continual Multimodal Pretraining" [NeurIPS'24]☆57Updated 8 months ago
- Code for PHATGOOSE introduced in "Learning to Route Among Specialized Experts for Zero-Shot Generalization"☆88Updated last year
- Official implementation for Equivariant Architectures for Learning in Deep Weight Spaces [ICML 2023]☆89Updated 2 years ago
- ☆188Updated last year
- ☆148Updated 11 months ago
- Data Valuation without Training of a Model, submitted to ICLR'23☆22Updated 2 years ago
- ☆31Updated 6 months ago
- ☆34Updated 7 months ago
- A repository for DenseSSMs☆88Updated last year
- Optimal Transport in the Big Data Era☆110Updated 9 months ago
- Model Merging with SVD to Tie the KnOTS [ICLR 2025]☆63Updated 4 months ago
- [NAACL 2025] A Closer Look into Mixture-of-Experts in Large Language Models☆54Updated 6 months ago
- Official Code for Paper: Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representation☆121Updated 2 months ago
- Repository for research works and resources related to model reprogramming <https://arxiv.org/abs/2202.10629>☆61Updated last year
- Official implementation of ORCA proposed in the paper "Cross-Modal Fine-Tuning: Align then Refine"☆72Updated last year