TUDB-Labs / MoE-PEFT
An Efficient LLM Fine-Tuning Factory Optimized for MoE PEFT
☆85Updated 3 weeks ago
Alternatives and similar repositories for MoE-PEFT:
Users that are interested in MoE-PEFT are comparing it to the libraries listed below
- State-of-the-art Parameter-Efficient MoE Fine-tuning Method☆153Updated 7 months ago
- ☆131Updated 8 months ago
- TokenSkip: Controllable Chain-of-Thought Compression in LLMs☆107Updated 3 weeks ago
- [ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"☆77Updated 9 months ago
- [SIGIR'24] The official implementation code of MOELoRA.☆155Updated 8 months ago
- CoT-Valve: Length-Compressible Chain-of-Thought Tuning☆58Updated last month
- ☆85Updated 3 weeks ago
- ☆99Updated 8 months ago
- [ICLR 2025] SuperCorrect: Advancing Small LLM Reasoning with Thought Template Distillation and Self-Correction☆66Updated last week
- ☆73Updated 2 weeks ago
- ☆82Updated 3 months ago
- code for ACL24 "MELoRA: Mini-Ensemble Low-Rank Adapter for Parameter-Efficient Fine-Tuning"☆19Updated last month
- The source code of "Merging Experts into One: Improving Computational Efficiency of Mixture of Experts (EMNLP 2023)":☆36Updated 11 months ago
- ☆26Updated last month
- [ICLR 2025] Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models☆82Updated last month
- [ICLR 2024] This is the repository for the paper titled "DePT: Decomposed Prompt Tuning for Parameter-Efficient Fine-tuning"☆96Updated 11 months ago
- Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models☆44Updated this week
- [ACL 2024] Long-Context Language Modeling with Parallel Encodings☆153Updated 9 months ago
- This repository contains a regularly updated paper list for LLMs-reasoning-in-latent-space.☆67Updated this week
- ☆50Updated last month
- ☆73Updated last week
- TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Models☆67Updated last year
- [ICLR 2025] When Attention Sink Emerges in Language Models: An Empirical View (Spotlight)☆55Updated 5 months ago
- A curated list of awesome resources dedicated to Scaling Laws for LLMs☆71Updated last year
- [NAACL 2025] A Closer Look into Mixture-of-Experts in Large Language Models☆47Updated last month
- [ICLR 2025] SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration☆45Updated last month
- ☆82Updated 2 weeks ago
- L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning☆165Updated 2 weeks ago
- [ICML 2024] Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibrati…☆33Updated 9 months ago
- ☆170Updated 8 months ago