Hunter-DDM / stablemoe
Code for the ACL-2022 paper "StableMoE: Stable Routing Strategy for Mixture of Experts"
☆44Updated 2 years ago
Alternatives and similar repositories for stablemoe:
Users that are interested in stablemoe are comparing it to the libraries listed below
- [NeurIPS 2022] "A Win-win Deal: Towards Sparse and Robust Pre-trained Language Models", Yuanxin Liu, Fandong Meng, Zheng Lin, Jiangnan Li…☆21Updated last year
- One Network, Many Masks: Towards More Parameter-Efficient Transfer Learning☆38Updated last year
- [NAACL 2022] "Learning to Win Lottery Tickets in BERT Transfer via Task-agnostic Mask Training", Yuanxin Liu, Fandong Meng, Zheng Lin, Pe…☆15Updated 2 years ago
- ☆33Updated 2 years ago
- Visual and Embodied Concepts evaluation benchmark☆21Updated last year
- The source code of "Merging Experts into One: Improving Computational Efficiency of Mixture of Experts (EMNLP 2023)":☆35Updated 10 months ago
- ☆32Updated 3 years ago
- This package implements THOR: Transformer with Stochastic Experts.☆62Updated 3 years ago
- Code for EMNLP 2021 main conference paper "Dynamic Knowledge Distillation for Pre-trained Language Models"☆40Updated 2 years ago
- Methods and evaluation for aligning language models temporally☆27Updated 11 months ago
- ☆30Updated last year
- DEMix Layers for Modular Language Modeling☆53Updated 3 years ago
- EMNLP'2022: BERTScore is Unfair: On Social Bias in Language Model-Based Metrics for Text Generation☆40Updated 2 years ago
- VaLM: Visually-augmented Language Modeling. ICLR 2023.☆56Updated last year
- ☆47Updated last month
- [NeurIPS 2022] Non-Linguistic Supervision for Contrastive Learning of Sentence Embeddings☆21Updated 2 years ago
- ☆18Updated 2 years ago
- [Findings of EMNLP22] From Mimicking to Integrating: Knowledge Integration for Pre-Trained Language Models☆19Updated last year
- ☆14Updated last year
- The official repository for the paper "From Zero to Hero: Examining the Power of Symbolic Tasks in Instruction Tuning".☆63Updated last year
- A Kernel-Based View of Language Model Fine-Tuning https://arxiv.org/abs/2210.05643☆74Updated last year
- ReCross: Unsupervised Cross-Task Generalization via Retrieval Augmentation☆24Updated 2 years ago
- ACL'23: Unified Demonstration Retriever for In-Context Learning☆36Updated last year
- ☆85Updated 2 years ago
- Official repository for MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models [NeurIPS 2024]☆58Updated 3 months ago
- ☆9Updated last week
- ☆20Updated 2 years ago
- Momentum Decoding: Open-ended Text Generation as Graph Exploration☆19Updated 2 years ago
- Source code for paper "On the Pareto Front of Multilingual Neural Machine Translation" @ NeurIPS 2023☆16Updated last year
- ☆25Updated last year