Alternatives and similar repositories for MergeBench

Users that are interested in MergeBench are comparing it to the libraries listed below

• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆56Updated 2 years ago
• allenai / hyperdecoders
  Codebase for Hyperdecoders https://arxiv.org/abs/2203.08304
  ☆14Updated 3 years ago
• BaohaoLiao / mefts
  [NeurIPS 2023] Make Your Pre-trained Model Reversible: From Parameter to Memory Efficient Fine-Tuning
  ☆33Updated 2 years ago
• princeton-nlp / Edge-Pruning
  [NeurIPS 2024 Spotlight] Code and data for the paper "Finding Transformer Circuits with Edge Pruning".
  ☆64Updated 5 months ago
• nverma1 / merging-text-transformers
  Code for "Merging Text Transformers from Different Initializations"
  ☆20Updated last year
• ShiZhengyan / InstructionModelling
  [NeurIPS 2024 Main Track] Code for the paper titled "Instruction Tuning With Loss Over Instructions"
  ☆38Updated last year
• facebookresearch / RLCD
  Reproduction of "RLCD Reinforcement Learning from Contrast Distillation for Language Model Alignment
  ☆69Updated 2 years ago
• ShiZhengyan / DePT
  [ICLR 2024] This is the repository for the paper titled "DePT: Decomposed Prompt Tuning for Parameter-Efficient Fine-tuning"
  ☆101Updated last year
• maszhongming / ParaKnowTransfer
  Code for "Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective"
  ☆33Updated last year
• BeyonderXX / TRACE
  TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Models
  ☆87Updated 2 years ago
• kamanphoebe / Look-into-MoEs
  [NAACL 2025] A Closer Look into Mixture-of-Experts in Large Language Models
  ☆58Updated last year
• tianyi-lab / Mosaic-IT
  [ACL'25] Mosaic-IT: Cost-Free Compositional Data Synthesis for Instruction Tuning
  ☆20Updated 4 months ago
• yule-BUAA / MergeLLM
  Codes for Merging Large Language Models
  ☆35Updated last year
• RLHFlow / Directional-Preference-Alignment
  Directional Preference Alignment
  ☆58Updated last year
• allenai / hyper-task-descriptions
  Learning adapter weights from task descriptions
  ☆19Updated 2 years ago
• cxcscmu / MATES
  Official repository for MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models [NeurIPS 2024]
  ☆79Updated last year
• benzakenelad / BitFit
  Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
  ☆143Updated 3 years ago
• sail-sg / dice
  Official implementation of Bootstrapping Language Models via DPO Implicit Rewards
  ☆47Updated 9 months ago
• Leooyii / LCEG
  Long Context Extension and Generalization in LLMs
  ☆62Updated last year
• kaistAI / factual-knowledge-acquisition
  ☆24Updated last month
• adymaharana / d2pruning
  ☆43Updated 2 years ago
• jongwooko / NASH-Pruning-Official
  Code Implementation for "NASH: A Simple Unified Framework of Structured Pruning for Accelerating Encoder-Decoder Language Models" (EMNLP …
  ☆17Updated 2 years ago
• r-three / smear
  ☆30Updated 2 years ago
• yuzhaouoe / pretraining-data-packing
  [ACL'24 Oral] Analysing The Impact of Sequence Composition on Language Model Pre-Training
  ☆23Updated last year
• bloomberg / dataless-model-merging
  Code release for Dataless Knowledge Fusion by Merging Weights of Language Models (https://openreview.net/forum?id=FCnohuR6AnM)
  ☆93Updated 2 years ago
• rabeehk / compacter
  ☆130Updated 3 years ago
• deeplearning-wisc / args
  ☆46Updated last year
• shankarp8 / knowledge_distillation
  Repository for "Propagating Knowledge Updates to LMs Through Distillation" (NeurIPS 2023).
  ☆26Updated last year
• wang-kee / LiNeS
  Official repository of "LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging"
  ☆31Updated last year
• mmatena / model_merging
  ☆80Updated 3 years ago