maidacundo / MoE-LoRALinks
Adapt an LLM model to a Mixture-of-Experts model using Parameter Efficient finetuning (LoRA), injecting the LoRAs in the FFN.
☆50Updated 9 months ago
Alternatives and similar repositories for MoE-LoRA
Users that are interested in MoE-LoRA are comparing it to the libraries listed below
Sorting:
- [SIGIR'24] The official implementation code of MOELoRA.☆175Updated last year
- ☆148Updated last year
- [ACL 2024] The official codebase for the paper "Self-Distillation Bridges Distribution Gap in Language Model Fine-tuning".☆125Updated 9 months ago
- ☆83Updated last year
- ☆107Updated last year
- ☆183Updated last year
- [ICML'2024] Can AI Assistants Know What They Don't Know?☆81Updated last year
- ☆140Updated 2 months ago
- this is an implementation for the paper Improve Mathematical Reasoning in Language Models by Automated Process Supervision from google de…☆36Updated last month
- The code and data of DPA-RAG, accepted by WWW 2025 main conference.☆61Updated 6 months ago
- LoRAMoE: Revolutionizing Mixture of Experts for Maintaining World Knowledge in Language Model Alignment☆360Updated last year
- Official code implementation for the ACL 2025 paper: 'CoT-based Synthesizer: Enhancing LLM Performance through Answer Synthesis'☆27Updated 2 months ago
- Code and Data for EMNLP 2024 Paper "Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agent"☆130Updated 2 weeks ago
- ☆49Updated last year
- [NAACL'24] Self-data filtering of LLM instruction-tuning data using a novel perplexity-based difficulty score, without using any other mo…☆381Updated last month
- [ACL'24] Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning☆166Updated last month
- The demo, code and data of FollowRAG☆74Updated last month
- ☆38Updated 3 weeks ago
- MoCLE (First MLLM with MoE for instruction customization and generalization!) (https://arxiv.org/abs/2312.12379)☆42Updated last month
- Awesome papers for role-playing with language models☆196Updated 9 months ago
- (ICLR'25) A Comprehensive Framework for Developing and Evaluating Multimodal Role-Playing Agents☆81Updated 6 months ago
- Instruct Once, Chat Consistently in Multiple Rounds: An Efficient Tuning Framework for Dialogue (ACL 2024)☆23Updated last year
- Official completion of “Training on the Benchmark Is Not All You Need”.☆35Updated 7 months ago
- Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation☆86Updated 8 months ago
- LongQLoRA: Extent Context Length of LLMs Efficiently☆166Updated last year
- [NeurIPS'24] Weak-to-Strong Search: Align Large Language Models via Searching over Small Language Models☆62Updated 7 months ago
- Benchmarking Complex Instruction-Following with Multiple Constraints Composition (NeurIPS 2024 Datasets and Benchmarks Track)☆90Updated 5 months ago
- The implement of paper:"ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretability"☆30Updated 2 months ago
- [ICLR'25] DataGen: Unified Synthetic Dataset Generation via Large Language Models☆60Updated 5 months ago
- [ACL 2024] This is the code repo for our ACL‘24 paper "MARVEL: Unlocking the Multi-Modal Capability of Dense Retrieval via Visual Module …☆36Updated last year