GraphPKU / PiSSALinks
PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models(NeurIPS 2024 Spotlight)
☆367Updated last month
Alternatives and similar repositories for PiSSA
Users that are interested in PiSSA are comparing it to the libraries listed below
Sorting:
- ☆204Updated 9 months ago
- AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning (ICLR 2023).☆338Updated 2 years ago
- Repo for Rho-1: Token-level Data Selection & Selective Pretraining of LLMs.☆428Updated last year
- [ICML2024 (Oral)] Official PyTorch implementation of DoRA: Weight-Decomposed Low-Rank Adaptation☆818Updated 10 months ago
- ☆148Updated last year
- ☆220Updated last year
- ☆269Updated last year
- State-of-the-art Parameter-Efficient MoE Fine-tuning Method☆175Updated 11 months ago
- Official implementation of "DoRA: Weight-Decomposed Low-Rank Adaptation"☆124Updated last year
- ☆183Updated last year
- A generalized framework for subspace tuning methods in parameter efficient fine-tuning.☆153Updated last month
- LoRAMoE: Revolutionizing Mixture of Experts for Maintaining World Knowledge in Language Model Alignment☆360Updated last year
- An Efficient LLM Fine-Tuning Factory Optimized for MoE PEFT☆108Updated 4 months ago
- [NeurIPS'24 Oral] HydraLoRA: An Asymmetric LoRA Architecture for Efficient Fine-Tuning☆220Updated 8 months ago
- Official PyTorch implementation of DistiLLM: Towards Streamlined Distillation for Large Language Models (ICML 2024)☆225Updated 4 months ago
- [ICML 2024] LESS: Selecting Influential Data for Targeted Instruction Tuning☆474Updated 9 months ago
- TokenSkip: Controllable Chain-of-Thought Compression in LLMs☆171Updated last month
- Official code for our paper, "LoRA-Pro: Are Low-Rank Adapters Properly Optimized? "☆127Updated 3 months ago
- ☆223Updated last year
- ☆205Updated 5 months ago
- A simple toolkit for benchmarking LLMs on mathematical reasoning tasks. 🧮✨☆238Updated last year
- [ICLR 2024] Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning☆626Updated last year
- Unofficial implementation for the paper "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"☆167Updated last year
- [NeurIPS 2024] SimPO: Simple Preference Optimization with a Reference-Free Reward☆912Updated 5 months ago
- ☆56Updated 7 months ago
- [ICLR2025] DiffuGPT and DiffuLLaMA: Scaling Diffusion Language Models via Adaptation from Autoregressive Models☆253Updated 2 months ago
- Code accompanying the paper "Massive Activations in Large Language Models"☆173Updated last year
- ⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training (EMNLP 2024)☆977Updated 7 months ago
- Implementation of DoRA☆299Updated last year
- ☆198Updated 3 months ago