Alternatives and similar repositories for transformer-fusion

Users that are interested in transformer-fusion are comparing it to the libraries listed below

• tanganke / peta
  Code for paper "Parameter Efficient Multi-task Model Fusion with Partial Linearization"
  ☆21Updated 8 months ago
• TsinghuaC3I / SoRA
  [EMNLP 2023, Main Conference] Sparse Low-rank Adaptation of Pre-trained Language Models
  ☆76Updated last year
• nik-dim / tall_masks
  Official repository of "Localizing Task Information for Improved Model Merging and Compression" [ICML 2024]
  ☆44Updated 6 months ago
• harveyhuang18 / EMR_Merging
  [NeurIPS 2024 Spotlight] EMR-Merging: Tuning-Free High-Performance Model Merging
  ☆59Updated 2 months ago
• UKPLab / iclr2024-model-merging
  This is the repository for "Model Merging by Uncertainty-Based Gradient Matching", ICLR 2024.
  ☆27Updated last year
• ArthurLeoM / peft-givens
  source code of (quasi-)Givens Orthogonal Fine Tuning integrated to peft lib
  ☆16Updated 2 months ago
• chuanyang-Zheng / DAPE
  The this is the official implementation of "DAPE: Data-Adaptive Positional Encoding for Length Extrapolation"
  ☆38Updated 7 months ago
• uiuctml / Localize-and-Stitch
  Localize-and-Stitch: Efficient Model Merging via Sparse Task Arithmetic
  ☆24Updated 4 months ago
• EnnengYang / RepresentationSurgery
  Representation Surgery for Multi-Task Model Merging. ICML, 2024.
  ☆45Updated 7 months ago
• tanganke / opcm
  ☆10Updated 3 months ago
• EnnengYang / Efficient-WEMoE
  Efficient and Effective Weight-Ensembling Mixture of Experts for Multi-Task Model Merging. Arxiv, 2024.
  ☆12Updated 6 months ago
• yule-BUAA / MergeLLM
  Codes for Merging Large Language Models
  ☆29Updated 9 months ago
• DaShenZi721 / HRA
  ☆28Updated 2 months ago
• PKU-ML / non_neg
  Official Code for ICLR 2024 Paper: Non-negative Contrastive Learning
  ☆45Updated last year
• AntoAndGar / task_singular_vectors
  Task Singular Vectors: Reducing Task Interference in Model Merging. Merge models avoiding task interference through separable models.
  ☆13Updated 2 weeks ago
• ElvishElvis / LCA-on-the-line
  LCA-on-the-line (ICML 2024 Oral)
  ☆11Updated 3 months ago
• EnnengYang / AdaMerging
  AdaMerging: Adaptive Model Merging for Multi-Task Learning. ICLR, 2024.
  ☆80Updated 6 months ago
• Shwai-He / SparseAdapter
  Source code of EMNLP 2022 Findings paper "SparseAdapter: An Easy Approach for Improving the Parameter-Efficiency of Adapters"
  ☆18Updated last year
• MohammadrezaBanaei / LoRA-XS
  LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters
  ☆33Updated 2 months ago
• VijayLingam95 / SVFT
  ☆28Updated 3 months ago
• haiquanlu / AlphaPruning
  [NeurIPS 2024] AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Models
  ☆23Updated last month
• ycjing / Awesome-Model-Merging
  A curated list of Model Merging methods.
  ☆92Updated 8 months ago
• sail-sg / Attention-Sink
  [ICLR 2025] When Attention Sink Emerges in Language Models: An Empirical View (Spotlight)
  ☆72Updated 7 months ago
• DavidFanzz / SCMoE
  ☆24Updated 11 months ago
• cjyaras / deep-lora-transformers
  Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation (ICML'24 Oral)
  ☆14Updated 9 months ago
• GATECH-EIC / ACT
  [ICML 2024] Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibrati…
  ☆38Updated 10 months ago
• pilancilab / Riemannian_Preconditioned_LoRA
  source code for paper "Riemannian Preconditioned LoRA for Fine-Tuning Foundation Models"
  ☆24Updated 10 months ago
• gortizji / tangent_task_arithmetic
  Source code of "Task arithmetic in the tangent space: Improved editing of pre-trained models".
  ☆100Updated last year
• arumaekawa / DiLM
  Implementaiton of "DiLM: Distilling Dataset into Language Model for Text-level Dataset Distillation" (accepted by NAACL2024 Findings)".
  ☆18Updated 3 months ago
• FeiyuZhang98 / IncreLoRA
  ☆31Updated last year