AntoAndGar / task_singular_vectors
Task Singular Vectors: Reducing Task Interference in Model Merging. Merge models avoiding task interference through separable models.
☆10Updated 2 months ago
Alternatives and similar repositories for task_singular_vectors:
Users that are interested in task_singular_vectors are comparing it to the libraries listed below
- Official repository of "Localizing Task Information for Improved Model Merging and Compression" [ICML 2024]☆42Updated 5 months ago
- [NeurIPS 2024 Spotlight] EMR-Merging: Tuning-Free High-Performance Model Merging☆53Updated last month
- AdaMerging: Adaptive Model Merging for Multi-Task Learning. ICLR, 2024.☆72Updated 5 months ago
- Code for paper "Parameter Efficient Multi-task Model Fusion with Partial Linearization"☆20Updated 6 months ago
- Code for paper "Merging Multi-Task Models via Weight-Ensembling Mixture of Experts"☆23Updated 9 months ago
- Representation Surgery for Multi-Task Model Merging. ICML, 2024.☆42Updated 5 months ago
- [NeurIPS 2024] For paper Parameter Competition Balancing for Model Merging☆37Updated 5 months ago
- ☆27Updated 11 months ago
- Structured Pruning Adapters in PyTorch☆16Updated last year
- ☆10Updated 2 months ago
- Codes for Merging Large Language Models☆29Updated 7 months ago
- [NeurIPS 2023] Make Your Pre-trained Model Reversible: From Parameter to Memory Efficient Fine-Tuning☆31Updated last year
- Less is More: Task-aware Layer-wise Distillation for Language Model Compression (ICML2023)☆34Updated last year
- ☆48Updated 4 months ago
- Code for "Merging Text Transformers from Different Initializations"☆20Updated 2 months ago
- A curated list of Model Merging methods.☆91Updated 6 months ago
- Code and benchmark for the paper: "A Practitioner's Guide to Continual Multimodal Pretraining" [NeurIPS'24]☆54Updated 3 months ago
- [NeurIPS 2024] AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Models☆22Updated 2 weeks ago
- ☆28Updated 8 months ago
- Localize-and-Stitch: Efficient Model Merging via Sparse Task Arithmetic☆23Updated 2 months ago
- CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for task-aware parameter-efficient fine-tuning(NeurIPS 2024)☆46Updated 2 months ago
- Official Pytorch Implementation of Our Paper Accepted at ICLR 2024-- Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLM…☆45Updated 11 months ago
- This is the repository for "Model Merging by Uncertainty-Based Gradient Matching", ICLR 2024.☆27Updated 10 months ago
- Data distillation benchmark☆58Updated this week
- [ICML 2024] Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibrati…☆33Updated 9 months ago
- Source code of "Task arithmetic in the tangent space: Improved editing of pre-trained models".☆99Updated last year
- Metrics for "Beyond neural scaling laws: beating power law scaling via data pruning " (NeurIPS 2022 Outstanding Paper Award)☆55Updated last year
- Code for NOLA, an implementation of "nola: Compressing LoRA using Linear Combination of Random Basis"☆53Updated 7 months ago
- LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning☆29Updated 11 months ago
- [EMNLP 2023 Main] Sparse Low-rank Adaptation of Pre-trained Language Models☆72Updated last year