Alternatives and similar repositories for MergeBench

Users that are interested in MergeBench are comparing it to the libraries listed below

• kamanphoebe / Look-into-MoEs
  [NAACL 2025] A Closer Look into Mixture-of-Experts in Large Language Models
  ☆55Updated 10 months ago
• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆56Updated 2 years ago
• allenai / hyperdecoders
  Codebase for Hyperdecoders https://arxiv.org/abs/2203.08304
  ☆13Updated 3 years ago
• pietrolesci / memorisation-profiles
  This is the official implementation for our ACL 2024 paper: "Causal Estimation of Memorisation Profiles".
  ☆23Updated 8 months ago
• ShiZhengyan / InstructionModelling
  [NeurIPS 2024 Main Track] Code for the paper titled "Instruction Tuning With Loss Over Instructions"
  ☆38Updated last year
• maszhongming / ParaKnowTransfer
  Code for "Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective"
  ☆33Updated last year
• sail-sg / dice
  Official implementation of Bootstrapping Language Models via DPO Implicit Rewards
  ☆45Updated 7 months ago
• BaohaoLiao / mefts
  [NeurIPS 2023] Make Your Pre-trained Model Reversible: From Parameter to Memory Efficient Fine-Tuning
  ☆33Updated 2 years ago
• ShiZhengyan / DePT
  [ICLR 2024] This is the repository for the paper titled "DePT: Decomposed Prompt Tuning for Parameter-Efficient Fine-tuning"
  ☆99Updated last year
• princeton-nlp / Edge-Pruning
  [NeurIPS 2024 Spotlight] Code and data for the paper "Finding Transformer Circuits with Edge Pruning".
  ☆62Updated 3 months ago
• nverma1 / merging-text-transformers
  Code for "Merging Text Transformers from Different Initializations"
  ☆20Updated 10 months ago
• ZHZisZZ / weak-to-strong-search
  [NeurIPS'24] Weak-to-Strong Search: Align Large Language Models via Searching over Small Language Models
  ☆64Updated last year
• facebookresearch / RLCD
  Reproduction of "RLCD Reinforcement Learning from Contrast Distillation for Language Model Alignment
  ☆69Updated 2 years ago
• RLHFlow / Directional-Preference-Alignment
  Directional Preference Alignment
  ☆58Updated last year
• kaistAI / factual-knowledge-acquisition
  ☆23Updated 3 weeks ago
• john-hewitt / implicit-ins
  Codebase for Instruction Following without Instruction Tuning
  ☆36Updated last year
• janphilippfranken / sami
  Self-Supervised Alignment with Mutual Information
  ☆20Updated last year
• hamishivi / automated-instruction-selection
  Exploration of automated dataset selection approaches at large scales.
  ☆50Updated 9 months ago
• matchten / LoRA-Models-for-SAEs
  Code for reproducing our paper "Low Rank Adapting Models for Sparse Autoencoder Features"
  ☆17Updated 8 months ago
• yale-nlp / refdpo
  ☆16Updated last year
• PKU-ML / LongPPL
  Code for ICLR 2025 Paper "What is Wrong with Perplexity for Long-context Language Modeling?"
  ☆105Updated 2 months ago
• Hritikbansal / jpo
  ☆13Updated 5 months ago
• yule-BUAA / MergeLLM
  Codes for Merging Large Language Models
  ☆34Updated last year
• r-three / smear
  ☆30Updated 2 years ago
• jongwooko / NASH-Pruning-Official
  Code Implementation for "NASH: A Simple Unified Framework of Structured Pruning for Accelerating Encoder-Decoder Language Models" (EMNLP …
  ☆16Updated 2 years ago
• bloomberg / dataless-model-merging
  Code release for Dataless Knowledge Fusion by Merging Weights of Language Models (https://openreview.net/forum?id=FCnohuR6AnM)
  ☆92Updated 2 years ago
• princeton-pli / MeCo
  Code for ICML 25 paper "Metadata Conditioning Accelerates Language Model Pre-training (MeCo)"
  ☆48Updated 5 months ago
• Infini-AI-Lab / S2FT
  ☆19Updated 11 months ago
• cxcscmu / MATES
  Official repository for MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models [NeurIPS 2024]
  ☆76Updated last year
• yuzhaouoe / pretraining-data-packing
  [ACL'24 Oral] Analysing The Impact of Sequence Composition on Language Model Pre-Training
  ☆22Updated last year