[EMNLP 2023, Main Conference] Sparse Low-rank Adaptation of Pre-trained Language Models
☆87Mar 5, 2024Updated 2 years ago
Alternatives and similar repositories for SoRA
Users that are interested in SoRA are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆36Aug 23, 2023Updated 3 years ago
- AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning (ICLR 2023).☆399Jun 1, 2023Updated 3 years ago
- ☆156Sep 9, 2024Updated 2 years ago
- ☆126Jul 6, 2024Updated 2 years ago
- ☆235Jun 24, 2024Updated 2 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- [NAACL 24 Oral] LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models☆40Jan 9, 2025Updated last year
- Official implementation of "DoRA: Weight-Decomposed Low-Rank Adaptation"☆122Apr 28, 2024Updated 2 years ago
- ☆15Nov 7, 2024Updated last year
- A generalized framework for subspace tuning methods in parameter efficient fine-tuning.☆183Jan 29, 2026Updated 9 months ago
- ☆216Nov 25, 2025Updated 11 months ago
- PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models(NeurIPS 2024 Spotlight)☆432Jun 30, 2025Updated last year
- ☆44Jul 22, 2024Updated 2 years ago
- ☆26Nov 23, 2023Updated 2 years ago
- code for ACL24 "MELoRA: Mini-Ensemble Low-Rank Adapter for Parameter-Efficient Fine-Tuning"☆33Feb 19, 2025Updated last year
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Awesome-Low-Rank-Adaptation☆134Oct 13, 2024Updated 2 years ago
- Official PyTorch implementation of QA-LoRA☆149Mar 13, 2024Updated 2 years ago
- [ICML2024 (Oral)] Official PyTorch implementation of DoRA: Weight-Decomposed Low-Rank Adaptation☆1,006Mar 24, 2026Updated 7 months ago
- ☆14Nov 2, 2025Updated last year
- ☆179Jul 22, 2024Updated 2 years ago
- One Network, Many Masks: Towards More Parameter-Efficient Transfer Learning☆40Jul 1, 2023Updated 3 years ago
- GPTQLoRA: Efficient Finetuning of Quantized LLMs with GPTQ☆100May 30, 2023Updated 3 years ago
- Codes for Merging Large Language Models☆37Aug 7, 2024Updated 2 years ago
- ☆22Jun 11, 2024Updated 2 years ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- This is the official repository for the paper "Flora: Low-Rank Adapters Are Secretly Gradient Compressors" in ICML 2024.☆106Jul 1, 2024Updated 2 years ago
- MiSS is a novel PEFT method that features a low-rank structure but introduces a new update mechanism distinct from LoRA, achieving an exc…☆37Mar 9, 2026Updated 7 months ago
- ☆27Jun 29, 2025Updated last year
- Supporting code for ReCEval paper☆33Sep 14, 2024Updated 2 years ago
- GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection☆22Mar 7, 2024Updated 2 years ago
- ☆129Jan 22, 2024Updated 2 years ago
- ☆71Jul 11, 2024Updated 2 years ago
- ☆236Jun 11, 2024Updated 2 years ago
- ☆99Jun 4, 2024Updated 2 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- ☆12Mar 6, 2024Updated 2 years ago
- Text as a Universal User Representation for Transferable Personalization☆13Mar 4, 2026Updated 7 months ago
- The GitHub repository for the paper "Self-prompted Chain-of-Thought on Large Language Models for Open-domain Multi-hop Reasoning" accepte…☆33Mar 8, 2024Updated 2 years ago
- AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuning (Zhou et al.; TACL 2024)☆50Mar 17, 2024Updated 2 years ago
- GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection☆1,712Oct 28, 2024Updated 2 years ago
- Code for NOLA, an implementation of "nola: Compressing LoRA using Linear Combination of Random Basis"☆61Aug 25, 2024Updated 2 years ago
- Task Compass: Scaling Multi-task Pre-training with Task Prefix (EMNLP 2022: Findings) (stay tuned & more will be updated)☆22Oct 17, 2022Updated 4 years ago