roymiles / VeLoRALinks
[NeurIPS 2024] VeLoRA : Memory Efficient Training using Rank-1 Sub-Token Projections
☆21Updated 9 months ago
Alternatives and similar repositories for VeLoRA
Users that are interested in VeLoRA are comparing it to the libraries listed below
Sorting:
- Model Merging with SVD to Tie the KnOTS [ICLR 2025]☆62Updated 4 months ago
- Code for NOLA, an implementation of "nola: Compressing LoRA using Linear Combination of Random Basis"☆55Updated 11 months ago
- ☆21Updated 2 years ago
- Towards Meta-Pruning via Optimal Transport, ICLR 2024 (Spotlight)☆16Updated 8 months ago
- Metrics for "Beyond neural scaling laws: beating power law scaling via data pruning " (NeurIPS 2022 Outstanding Paper Award)☆56Updated 2 years ago
- This is a PyTorch implementation of the paperViP A Differentially Private Foundation Model for Computer Vision☆36Updated 2 years ago
- [NeurIPS'24] Multilinear Mixture of Experts: Scalable Expert Specialization through Factorization☆33Updated 10 months ago
- ☆14Updated 6 months ago
- This repository is the implementation of the paper Training Free Pretrained Model Merging (CVPR2024).☆31Updated last year
- Code for T-MARS data filtering☆35Updated last year
- SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining (NeurIPS 2024)☆32Updated 9 months ago
- ☆23Updated 2 months ago
- ☆30Updated 6 months ago
- Official implementation of "Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization"☆79Updated last year
- Code and benchmark for the paper: "A Practitioner's Guide to Continual Multimodal Pretraining" [NeurIPS'24]☆57Updated 8 months ago
- AdaSplash: Adaptive Sparse Flash Attention (aka Flash Entmax Attention)☆19Updated 3 weeks ago
- ☆38Updated last year
- Official PyTorch implementation of DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMs (ICML 2025 Oral)☆34Updated last month
- Official implementation of AAAI 2023 paper "Parameter-efficient Model Adaptation for Vision Transformers"☆105Updated 2 years ago
- Official Pytorch Implementation of Our Paper Accepted at ICLR 2024-- Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLM…☆49Updated last year
- Data distillation benchmark☆67Updated last month
- ☆51Updated last year
- [ICML24] Official Implementation of "ETHER: Efficient Finetuning of Large-Scale Models with Hyperplane Reflections"☆16Updated last year
- ☆34Updated 2 months ago
- Source code of "Task arithmetic in the tangent space: Improved editing of pre-trained models".☆103Updated 2 years ago
- Official code for our paper, "LoRA-Pro: Are Low-Rank Adapters Properly Optimized? "☆127Updated 4 months ago
- [NAACL 2025 Oral] Multimodal Needle in a Haystack (MMNeedle): Benchmarking Long-Context Capability of Multimodal Large Language Models☆48Updated 3 months ago
- [BMVC 2022] Information Theoretic Representation Distillation☆18Updated last year
- ☆43Updated 9 months ago
- [ICLR 2025] Official Pytorch Implementation of "Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN" by Pengxia…☆25Updated 2 weeks ago