Alternatives and similar repositories for Pruning-via-Merging

Users that are interested in Pruning-via-Merging are comparing it to the libraries listed below

• zyxxmu / DSnoT
  Official Pytorch Implementation of Our Paper Accepted at ICLR 2024-- Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLM…
  ☆49Updated last year
• ROIM1998 / APT
  [ICML'24 Oral] APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference
  ☆43Updated last year
• jiwonsong-dev / SLEB
  Official Implementation of SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blocks
  ☆36Updated 5 months ago
• CASIA-IVA-Lab / FLAP
  [AAAI 2024] Fluctuation-based Adaptive Structured Pruning for Large Language Models
  ☆56Updated last year
• yangyifei729 / LaCo
  Official implementation for LaCo (EMNLP 2024 Findings)
  ☆17Updated 9 months ago
• MrGGLS / BlockPruner
  A block pruning framework for LLMs.
  ☆24Updated last month
• hahnyuan / ASVD4LLM
  Activation-aware Singular Value Decomposition for Compressing Large Language Models
  ☆74Updated 8 months ago
• wzhuang-xmu / LoSA
  [ICLR 2025] The official pytorch implement of "Dynamic Low-Rank Sparse Adaptation for Large Language Models".
  ☆19Updated 4 months ago
• ruipeterpan / specreason
  PoC for "SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning" [arXiv '25]
  ☆41Updated 2 months ago
• BaiTheBest / SparseLLM
  Official Repo for SparseLLM: Global Pruning of LLMs (NeurIPS 2024)
  ☆64Updated 3 months ago
• UNITES-Lab / MC-SMoE
  [ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"
  ☆86Updated 3 weeks ago
• locuslab / massive-activations
  Code accompanying the paper "Massive Activations in Large Language Models"
  ☆169Updated last year
• Lucky-Lance / Expert_Sparsity
  [ACL 2024] Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
  ☆94Updated last year
• htqin / IR-QLoRA
  [ICML 2024 Oral] This project is the official implementation of our Accurate LoRA-Finetuning Quantization of LLMs via Information Retenti…
  ☆65Updated last year
• horseee / CoT-Valve
  CoT-Valve: Length-Compressible Chain-of-Thought Tuning
  ☆76Updated 5 months ago
• aim-uofa / LoRAPrune
  ☆56Updated 7 months ago
• zyxxmu / cam
  Pytorch implementation of our paper accepted by ICML 2024 -- CaM: Cache Merging for Memory-efficient LLMs Inference
  ☆41Updated last year
• hemingkx / TokenSkip
  TokenSkip: Controllable Chain-of-Thought Compression in LLMs
  ☆164Updated 2 weeks ago
• osehmathias / lisa
  LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning
  ☆31Updated last year
• PositionalHidden / PositionalHidden
  To mitigate position bias in LLMs, especially in long-context scenarios, we scale only one dimension of LLMs, reducing position bias and …
  ☆11Updated last year
• machilusZ / FastGen
  This repo contains the source code for: Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs
  ☆37Updated 11 months ago
• UCSB-NLP-Chang / ThinkPrune
  ☆36Updated 3 months ago
• LINs-lab / DynMoE
  [ICLR 2025] Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models
  ☆116Updated last week
• harveyhuang18 / EMR_Merging
  [NeurIPS 2024 Spotlight] EMR-Merging: Tuning-Free High-Performance Model Merging
  ☆59Updated 4 months ago
• OpenSparseLLMs / MoM
  ☆90Updated 2 months ago
• yxli2123 / LoSparse
  ☆58Updated last year
• SqueezeAILab / SqueezedAttention
  SQUEEZED ATTENTION: Accelerating Long Prompt LLM Inference
  ☆49Updated 7 months ago
• sramshetty / ShortGPT
  Unofficial implementations of block/layer-wise pruning methods for LLMs.
  ☆72Updated last year
• yule-BUAA / MergeLLM
  Codes for Merging Large Language Models
  ☆32Updated 11 months ago
• OpenSparseLLMs / Linear-MoE
  ☆109Updated last month