Official implementation of "DoRA: Weight-Decomposed Low-Rank Adaptation"
☆121Apr 28, 2024Updated 2 years ago
Alternatives and similar repositories for DoRA
Users that are interested in DoRA are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Implementation of DoRA☆310Oct 16, 2026Updated 2 weeks ago
- ☆236Jun 11, 2024Updated 2 years ago
- Official Code For Dual Grained Quantization: Efficient Fine-Grained Quantization for LLM☆14Dec 27, 2023Updated 2 years ago
- [NAACL 24 Oral] LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models☆40Jan 9, 2025Updated last year
- [EMNLP 2023, Main Conference] Sparse Low-rank Adaptation of Pre-trained Language Models☆87Mar 5, 2024Updated 2 years ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- [ICML2024 (Oral)] Official PyTorch implementation of DoRA: Weight-Decomposed Low-Rank Adaptation☆1,005Mar 24, 2026Updated 7 months ago
- ☆235Jun 24, 2024Updated 2 years ago
- AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning (ICLR 2023).☆398Jun 1, 2023Updated 3 years ago
- The code and data for the GPT-4 based benchmark in the vicuna blog post☆42Aug 2, 2023Updated 3 years ago
- GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection☆1,710Oct 28, 2024Updated 2 years ago
- ☆207Dec 5, 2024Updated last year
- ☆35Feb 10, 2025Updated last year
- Code for our EMNLP 2023 Paper: "LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models"☆1,237Mar 10, 2024Updated 2 years ago
- [TMLR 2025] When Attention Collapses: How Degenerate Layers in LLMs Enable Smaller, Stronger Models☆126Mar 6, 2026Updated 7 months ago
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- fixed official code for paper "A Closer Look at Parameter-Efficient Tuning in Diffusion Models".☆43Jun 5, 2023Updated 3 years ago
- ☆44Jul 22, 2024Updated 2 years ago
- [EMNLP 2024] RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantization☆40Sep 24, 2024Updated 2 years ago
- [ICLR 2024] This is the repository for the paper titled "DePT: Decomposed Prompt Tuning for Parameter-Efficient Fine-tuning"☆101Apr 10, 2024Updated 2 years ago
- Official code for ReLoRA from the paper Stack More Layers Differently: High-Rank Training Through Low-Rank Updates☆477Apr 21, 2024Updated 2 years ago
- This is a repository for "PMET: Precise Model Editing in a Transformer"☆58Sep 28, 2023Updated 3 years ago
- code for ACL24 "MELoRA: Mini-Ensemble Low-Rank Adapter for Parameter-Efficient Fine-Tuning"☆33Feb 19, 2025Updated last year
- ☆279Oct 31, 2023Updated 2 years ago
- ☆216Nov 25, 2025Updated 11 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Code Repository for the NeurIPS 2024 Paper "Toward Efficient Inference for Mixture of Experts".☆19Oct 30, 2024Updated 2 years ago
- [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models☆35Jun 12, 2024Updated 2 years ago
- PB-LLM: Partially Binarized Large Language Models☆159Nov 20, 2023Updated 2 years ago
- A fork of the PEFT library, supporting Robust Adaptation (RoSA)☆16Aug 16, 2024Updated 2 years ago
- [EMNLP 2024] LongAlign: A Recipe for Long Context Alignment of LLMs☆265Dec 16, 2024Updated last year
- An unofficial implementation of "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"☆35Jun 7, 2024Updated 2 years ago
- Repo for Rho-1: Token-level Data Selection & Selective Pretraining of LLMs.☆474Apr 18, 2024Updated 2 years ago
- [SIGIR'24] The official implementation code of MOELoRA.☆197Jul 22, 2024Updated 2 years ago
- (ICLR 2025) BinaryDM: Accurate Weight Binarization for Efficient Diffusion Models☆25Oct 4, 2024Updated 2 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Official repository for ACL 2025 paper "Model Extrapolation Expedites Alignment"☆75May 20, 2025Updated last year
- Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint?☆117Oct 21, 2024Updated 2 years ago
- [NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization☆436Aug 13, 2024Updated 2 years ago
- Official implementation of "Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization"☆82Apr 12, 2024Updated 2 years ago
- The repository for our paper: Neighboring Perturbations of Knowledge Editing on Large Language Models☆16May 4, 2024Updated 2 years ago
- [ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Mod…☆39Mar 11, 2024Updated 2 years ago
- For releasing code related to compression methods for transformers, accompanying our publications☆464Sep 10, 2026Updated last month