Alternatives and similar repositories for MergeBench

Users that are interested in MergeBench are comparing it to the libraries listed below

• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆56Updated 2 years ago
• mmatena / model_merging
  ☆80Updated 3 years ago
• nverma1 / merging-text-transformers
  Code for "Merging Text Transformers from Different Initializations"
  ☆20Updated 11 months ago
• ShiZhengyan / DePT
  [ICLR 2024] This is the repository for the paper titled "DePT: Decomposed Prompt Tuning for Parameter-Efficient Fine-tuning"
  ☆101Updated last year
• Thartvigsen / GRACE
  [NeurIPS'23] Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors
  ☆83Updated last year
• ShiZhengyan / InstructionModelling
  [NeurIPS 2024 Main Track] Code for the paper titled "Instruction Tuning With Loss Over Instructions"
  ☆38Updated last year
• princeton-nlp / Edge-Pruning
  [NeurIPS 2024 Spotlight] Code and data for the paper "Finding Transformer Circuits with Edge Pruning".
  ☆64Updated 5 months ago
• kamanphoebe / Look-into-MoEs
  [NAACL 2025] A Closer Look into Mixture-of-Experts in Large Language Models
  ☆58Updated 11 months ago
• rabeehk / compacter
  ☆130Updated 3 years ago
• facebookresearch / RLCD
  Reproduction of "RLCD Reinforcement Learning from Contrast Distillation for Language Model Alignment
  ☆69Updated 2 years ago
• cxcscmu / MATES
  Official repository for MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models [NeurIPS 2024]
  ☆79Updated last year
• BeyonderXX / TRACE
  TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Models
  ☆86Updated 2 years ago
• yule-BUAA / MergeLLM
  Codes for Merging Large Language Models
  ☆35Updated last year
• song-wx / SIFT
  [ICML2024 Spotlight] Fine-Tuning Pre-trained Large Language Models Sparsely
  ☆24Updated last year
• allenai / hyperdecoders
  Codebase for Hyperdecoders https://arxiv.org/abs/2203.08304
  ☆14Updated 3 years ago
• wang-kee / LiNeS
  Official repository of "LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging"
  ☆31Updated last year
• bloomberg / dataless-model-merging
  Code release for Dataless Knowledge Fusion by Merging Weights of Language Models (https://openreview.net/forum?id=FCnohuR6AnM)
  ☆92Updated 2 years ago
• zjunlp / PitfallsKnowledgeEditing
  [ICLR 2024] Unveiling the Pitfalls of Knowledge Editing for Large Language Models
  ☆22Updated last year
• pietrolesci / memorisation-profiles
  This is the official implementation for our ACL 2024 paper: "Causal Estimation of Memorisation Profiles".
  ☆24Updated 10 months ago
• sail-sg / dice
  Official implementation of Bootstrapping Language Models via DPO Implicit Rewards
  ☆47Updated 9 months ago
• adymaharana / d2pruning
  ☆43Updated 2 years ago
• benzakenelad / BitFit
  Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
  ☆143Updated 3 years ago
• BaohaoLiao / mefts
  [NeurIPS 2023] Make Your Pre-trained Model Reversible: From Parameter to Memory Efficient Fine-Tuning
  ☆33Updated 2 years ago
• hamishivi / automated-instruction-selection
  Exploration of automated dataset selection approaches at large scales.
  ☆52Updated 10 months ago
• RLHFlow / Directional-Preference-Alignment
  Directional Preference Alignment
  ☆58Updated last year
• tatsu-lab / test_set_contamination
  ☆41Updated 2 years ago
• avalonstrel / Mitigating-the-Alignment-Tax-of-RLHF
  ☆16Updated last year
• r-three / smear
  ☆30Updated 2 years ago
• TianduoWang / DPO-ST
  [ACL 2024] Self-Training with Direct Preference Optimization Improves Chain-of-Thought Reasoning
  ☆53Updated last year
• zjysteven / mink-plus-plus
  [ICLR'25 Spotlight] Min-K%++: Improved baseline for detecting pre-training data of LLMs
  ☆52Updated 8 months ago