ycjing / Awesome-Model-Merging
A curated list of Model Merging methods.
☆71Updated this week
Related projects: ⓘ
- FusionBench: A Comprehensive Benchmark of Deep Model Fusion☆42Updated 2 weeks ago
- The source code of the EMNLP 2023 main conference paper: Sparse Low-rank Adaptation of Pre-trained Language Models.☆62Updated 6 months ago
- Code accompanying the paper "Massive Activations in Large Language Models"☆104Updated 6 months ago
- ☆54Updated 2 months ago
- AdaMerging: Adaptive Model Merging for Multi-Task Learning. ICLR, 2024.☆40Updated 2 weeks ago
- source code of (quasi-)Givens Orthogonal Fine Tuning integrated to peft lib☆12Updated this week
- [ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"☆63Updated 3 months ago
- ☆40Updated 5 months ago
- Official repository of "Localizing Task Information for Improved Model Merging and Compression" [ICML 2024]☆27Updated 3 months ago
- Source code of "Task arithmetic in the tangent space: Improved editing of pre-trained models".☆79Updated last year
- Official implementation of "DoRA: Weight-Decomposed Low-Rank Adaptation"☆120Updated 4 months ago
- ☆119Updated last week
- ☆94Updated 6 months ago
- Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities. arXiv:2408.07666.☆107Updated this week
- ☆27Updated last month
- [ICML 2024] Official code for the paper "Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark ".☆62Updated 2 months ago
- ☆97Updated last month
- ☆25Updated 11 months ago
- ☆136Updated 7 months ago
- ICLR 2024, Towards Lossless Dataset Distillation via Difficulty-Aligned Trajectory Matching☆87Updated 3 months ago
- PyTorch implementation of "From Sparse to Soft Mixtures of Experts"☆38Updated last year
- [ICLR 2024] This is the repository for the paper titled "DePT: Decomposed Prompt Tuning for Parameter-Efficient Fine-tuning"☆90Updated 5 months ago
- Awesome Learn From Model Beyond Fine-Tuning: A Survey☆44Updated 9 months ago
- [ICML 2024 Oral] Official code repository for MLLM-as-a-Judge.☆47Updated last month
- A repository for DenseSSMs☆86Updated 5 months ago
- LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning☆15Updated 5 months ago
- ☆24Updated 11 months ago
- [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…☆42Updated last year
- [ACL 2023] Code for paper “Tailoring Instructions to Student’s Learning Levels Boosts Knowledge Distillation”(https://arxiv.org/abs/2305.…☆38Updated last year
- ☆16Updated last month