State-of-the-art Parameter-Efficient MoE Fine-tuning Method
☆203Aug 22, 2024Updated last year
Alternatives and similar repositories for MixLoRA
Users that are interested in MixLoRA are comparing it to the libraries listed below
Sorting:
- An Efficient LLM Fine-Tuning Factory Optimized for MoE PEFT☆133Mar 11, 2025Updated 11 months ago
- This repository has transferred to https://github.com/TUDB-Labs/MoE-PEFT☆22Aug 16, 2024Updated last year
- LoRAMoE: Revolutionizing Mixture of Experts for Maintaining World Knowledge in Language Model Alignment☆401Apr 29, 2024Updated last year
- Multimodal Instruction Tuning with Conditional Mixture of LoRA (ACL 2024)☆31Aug 9, 2024Updated last year
- [SIGIR'24] The official implementation code of MOELoRA.☆188Jul 22, 2024Updated last year
- An Efficient "Factory" to Build Multiple LoRA Adapters☆372Feb 13, 2025Updated last year
- ☆176Jul 22, 2024Updated last year
- ☆65Dec 2, 2024Updated last year
- Adapt an LLM model to a Mixture-of-Experts model using Parameter Efficient finetuning (LoRA), injecting the LoRAs in the FFN.☆84Oct 21, 2025Updated 4 months ago
- [ ICLR 2025 ] Making LLMs More Effective with Hierarchical Mixture of LoRA Experts☆28Oct 9, 2025Updated 4 months ago
- Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuning☆36Nov 17, 2024Updated last year
- [NeurIPS 2024] MoME: Mixture of Multimodal Experts for Generalist Multimodal Large Language Models☆79Dec 27, 2025Updated 2 months ago
- [COLM 2024] LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition☆669Jul 22, 2024Updated last year
- ☆17May 2, 2024Updated last year
- AdaMoLE: Adaptive Mixture of LoRA Experts☆38Oct 11, 2024Updated last year
- Load multiple LoRA modules simultaneously and automatically switch the appropriate combination of LoRA modules to generate the best answe…☆159Feb 9, 2024Updated 2 years ago
- ☆14Jun 6, 2023Updated 2 years ago
- ☆21Jul 25, 2025Updated 7 months ago
- Source code of paper: A Stronger Mixture of Low-Rank Experts for Fine-Tuning Foundation Models. (ICML 2025)☆36Apr 2, 2025Updated 11 months ago
- ☆274Oct 31, 2023Updated 2 years ago
- Official PyTorch implementation of RACRO (https://www.arxiv.org/abs/2506.04559)☆19Jul 1, 2025Updated 8 months ago
- official implementation of "CLIP-VQDiffusion : Langauge Free Training of Text To Image generation using CLIP and vector quantized diffusi…☆18Sep 5, 2024Updated last year
- Official repository for MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models [NeurIPS 2024]☆79Nov 14, 2024Updated last year
- [ICML‘2024] "LoCoCo: Dropping In Convolutions for Long Context Compression", Ruisi Cai, Yuandong Tian, Zhangyang Wang, Beidi Chen☆17Sep 7, 2024Updated last year
- Code for http://proceedings.mlr.press/v80/dvurechensky18a.html☆17Jul 31, 2018Updated 7 years ago
- AdaMerging: Adaptive Model Merging for Multi-Task Learning. ICLR, 2024.☆99Oct 28, 2024Updated last year
- ☆22Nov 25, 2025Updated 3 months ago
- [CVPR2025] Official Repository for IMMUNE: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignment☆27Jun 11, 2025Updated 8 months ago
- Compiler-R1: Towards Agentic Compiler Auto-tuning with Reinforcement Learning☆28Jul 14, 2025Updated 7 months ago
- [SIGIR'24] The official implementation code of MOELoRA.☆36Aug 3, 2024Updated last year
- Parameter-Efficient Sparsity Crafting From Dense to Mixture-of-Experts for Instruction Tuning on General Tasks (EMNLP'24)☆145Sep 20, 2024Updated last year
- Xmixers: A collection of SOTA efficient token/channel mixers☆28Sep 4, 2025Updated 6 months ago
- AdaSplash: Adaptive Sparse Flash Attention (aka Flash Entmax Attention)☆33Sep 30, 2025Updated 5 months ago
- A list of papers for group meeting☆19Jan 5, 2026Updated 2 months ago
- A holistic benchmark for LLM abstention☆71Aug 27, 2025Updated 6 months ago
- ☆126Jul 6, 2024Updated last year
- [NeurIPS 2025] Elevating Visual Perception in Multimodal LLMs with Visual Embedding Distillation☆71Oct 17, 2025Updated 4 months ago
- Official repository for the paper Local Linear Attention: An Optimal Interpolation of Linear and Softmax Attention For Test-Time Regressi…☆23Oct 1, 2025Updated 5 months ago
- Official Implementation of HIMA (COLM'25)☆19Nov 25, 2025Updated 3 months ago