Official implementation of "DoRA: Weight-Decomposed Low-Rank Adaptation"
☆122Apr 28, 2024Updated 2 years ago
Alternatives and similar repositories for DoRA
Users that are interested in DoRA are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Implementation of DoRA☆309Oct 16, 2026Updated last month
- ☆236Jun 11, 2024Updated 2 years ago
- ☆235Jun 24, 2024Updated 2 years ago
- [NAACL 24 Oral] LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models☆40Jan 9, 2025Updated last year
- [ICML2024 (Oral)] Official PyTorch implementation of DoRA: Weight-Decomposed Low-Rank Adaptation☆1,007Mar 24, 2026Updated 7 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- [EMNLP 2023, Main Conference] Sparse Low-rank Adaptation of Pre-trained Language Models☆87Mar 5, 2024Updated 2 years ago
- Official Code For Dual Grained Quantization: Efficient Fine-Grained Quantization for LLM☆14Dec 27, 2023Updated 2 years ago
- AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning (ICLR 2023).☆397Jun 1, 2023Updated 3 years ago
- The code and data for the GPT-4 based benchmark in the vicuna blog post☆42Aug 2, 2023Updated 3 years ago
- GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection☆1,712Oct 28, 2024Updated 2 years ago
- ☆35Feb 10, 2025Updated last year
- ☆207Dec 5, 2024Updated last year
- [TMLR 2025] When Attention Collapses: How Degenerate Layers in LLMs Enable Smaller, Stronger Models☆126Mar 6, 2026Updated 8 months ago
- Code for our EMNLP 2023 Paper: "LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models"☆1,238Mar 10, 2024Updated 2 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- fixed official code for paper "A Closer Look at Parameter-Efficient Tuning in Diffusion Models".☆43Jun 5, 2023Updated 3 years ago
- ☆44Jul 22, 2024Updated 2 years ago
- A collection of RL baselines in Jax.☆10Nov 10, 2024Updated 2 years ago
- [EMNLP 2024] RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantization☆40Sep 24, 2024Updated 2 years ago
- This is a repository for "PMET: Precise Model Editing in a Transformer"☆58Sep 28, 2023Updated 3 years ago
- code for ACL24 "MELoRA: Mini-Ensemble Low-Rank Adapter for Parameter-Efficient Fine-Tuning"☆32Feb 19, 2025Updated last year
- [ICLR 2024] This is the repository for the paper titled "DePT: Decomposed Prompt Tuning for Parameter-Efficient Fine-tuning"☆101Apr 10, 2024Updated 2 years ago
- ☆279Oct 31, 2023Updated 3 years ago
- Official code for ReLoRA from the paper Stack More Layers Differently: High-Rank Training Through Low-Rank Updates☆478Apr 21, 2024Updated 2 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- A fork of the PEFT library, supporting Robust Adaptation (RoSA)☆16Aug 16, 2024Updated 2 years ago
- [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models☆35Jun 12, 2024Updated 2 years ago
- ☆156Sep 9, 2024Updated 2 years ago
- ☆215Nov 25, 2025Updated 11 months ago
- PB-LLM: Partially Binarized Large Language Models☆159Nov 20, 2023Updated 3 years ago
- [EMNLP 2024] LongAlign: A Recipe for Long Context Alignment of LLMs☆266Dec 16, 2024Updated last year
- Repo for Rho-1: Token-level Data Selection & Selective Pretraining of LLMs.☆475Apr 18, 2024Updated 2 years ago
- An unofficial implementation of "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"☆35Jun 7, 2024Updated 2 years ago
- (ICLR 2025) BinaryDM: Accurate Weight Binarization for Efficient Diffusion Models☆25Oct 4, 2024Updated 2 years ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- LoRA and DoRA from Scratch Implementations☆232Mar 5, 2024Updated 2 years ago
- Official repository for ACL 2025 paper "Model Extrapolation Expedites Alignment"☆75May 20, 2025Updated last year
- [SIGIR'24] The official implementation code of MOELoRA.☆196Jul 22, 2024Updated 2 years ago
- Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint?☆117Oct 21, 2024Updated 2 years ago
- [NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization☆440Aug 13, 2024Updated 2 years ago
- The repository for our paper: Neighboring Perturbations of Knowledge Editing on Large Language Models☆16May 4, 2024Updated 2 years ago
- Official implementation of "Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization"☆82Apr 12, 2024Updated 2 years ago
Are these results useful?