Outsider565 / LoRA-GA
☆186Updated 5 months ago
Alternatives and similar repositories for LoRA-GA:
Users that are interested in LoRA-GA are comparing it to the libraries listed below
- A generalized framework for subspace tuning methods in parameter efficient fine-tuning.☆132Updated last month
- ☆170Updated 8 months ago
- Official code for our paper, "LoRA-Pro: Are Low-Rank Adapters Properly Optimized? "☆108Updated 3 weeks ago
- PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models(NeurIPS 2024 Spotlight)☆337Updated 2 months ago
- LoRAMoE: Revolutionizing Mixture of Experts for Maintaining World Knowledge in Language Model Alignment☆314Updated 11 months ago
- ☆131Updated 8 months ago
- ☆99Updated 8 months ago
- AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning (ICLR 2023).☆311Updated last year
- [NeurIPS'24 Oral] HydraLoRA: An Asymmetric LoRA Architecture for Efficient Fine-Tuning☆181Updated 4 months ago
- ☆146Updated 3 weeks ago
- [SIGIR'24] The official implementation code of MOELoRA.☆155Updated 8 months ago
- ☆178Updated last month
- [ACL 2024] Long-Context Language Modeling with Parallel Encodings☆153Updated 9 months ago
- Rectified Rotary Position Embeddings☆362Updated 10 months ago
- ☆142Updated 6 months ago
- State-of-the-art Parameter-Efficient MoE Fine-tuning Method☆153Updated 7 months ago
- Implementation for "Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs"☆357Updated 2 months ago
- A prototype repo for hybrid training of pipeline parallel and distributed data parallel with comments on core code snippets. Feel free to…☆55Updated last year
- An Easy-to-use, Scalable and High-performance RLHF Framework designed for Multimodal Models.☆103Updated 3 weeks ago
- CoT-Valve: Length-Compressible Chain-of-Thought Tuning☆58Updated last month
- [ICLR 2024]EMO: Earth Mover Distance Optimization for Auto-Regressive Language Modeling(https://arxiv.org/abs/2310.04691)☆120Updated last year
- [EMNLP 2024 Findings🔥] Official implementation of ": LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context In…☆92Updated 4 months ago
- Official implementation of "DoRA: Weight-Decomposed Low-Rank Adaptation"☆123Updated 11 months ago
- ☆28Updated 4 months ago
- [ICLR 2025] 🧬 RegMix: Data Mixture as Regression for Language Model Pre-training (Spotlight)☆117Updated last month
- TokenSkip: Controllable Chain-of-Thought Compression in LLMs☆107Updated 3 weeks ago
- An Efficient LLM Fine-Tuning Factory Optimized for MoE PEFT☆85Updated 3 weeks ago
- [ACL 2024] The official codebase for the paper "Self-Distillation Bridges Distribution Gap in Language Model Fine-tuning".☆115Updated 5 months ago
- Exploring the Limit of Outcome Reward for Learning Mathematical Reasoning☆167Updated 2 weeks ago
- qwen-nsa☆46Updated this week