Alternatives and similar repositories for peft

Users that are interested in peft are comparing it to the libraries listed below

• sjelassi / transformers_ssm_copy
  ☆35Updated last year
• yuzhaouoe / pretraining-data-packing
  [ACL'24 Oral] Analysing The Impact of Sequence Composition on Language Model Pre-Training
  ☆23Updated last year
• Leooyii / LCEG
  Long Context Extension and Generalization in LLMs
  ☆62Updated last year
• john-hewitt / backpacks-flash-attn
  The original Backpack Language Model implementation, a fork of FlashAttention
  ☆71Updated 2 years ago
• HanGuo97 / lq-lora
  ☆127Updated last year
• roeehendel / icl_task_vectors
  ☆103Updated 2 years ago
• PKU-ML / LongPPL
  Code for ICLR 2025 Paper "What is Wrong with Perplexity for Long-context Language Modeling?"
  ☆107Updated 3 months ago
• benzakenelad / BitFit
  Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
  ☆143Updated 3 years ago
• qiuzh20 / EMoE
  Official PyTorch Implementation of EMoE: Unlocking Emergent Modularity in Large Language Models [main conference @ NAACL2024]
  ☆38Updated last year
• MadryLab / DsDm
  ☆51Updated last year
• GSYfate / knnlm-limits
  Official code repo for paper "Great Memory, Shallow Reasoning: Limits of kNN-LMs"
  ☆23Updated 8 months ago
• locuslab / massive-activations
  Code accompanying the paper "Massive Activations in Large Language Models"
  ☆191Updated last year
• FeiyuZhang98 / IncreLoRA
  ☆34Updated 2 years ago
• ducdauge / sft-llm
  Scaling Sparse Fine-Tuning to Large Language Models
  ☆18Updated last year
• bloomberg / dataless-model-merging
  Code release for Dataless Knowledge Fusion by Merging Weights of Language Models (https://openreview.net/forum?id=FCnohuR6AnM)
  ☆92Updated 2 years ago
• ShiZhengyan / InstructionModelling
  [NeurIPS 2024 Main Track] Code for the paper titled "Instruction Tuning With Loss Over Instructions"
  ☆38Updated last year
• HKUNLP / STRING
  [ICLR'25] Data and code for our paper "Why Does the Effective Context Length of LLMs Fall Short?"
  ☆78Updated last year
• RobertCsordas / moeut
  ☆91Updated last year
• mlfoundations / scaling
  Language models scale reliably with over-training and on downstream tasks
  ☆100Updated last year
• JeanKaddour / NoTrainNoGain
  Revisiting Efficient Training Algorithms For Transformer-based Language Models (NeurIPS 2023)
  ☆81Updated 2 years ago
• haonan3 / AnchorContext
  AnchorAttention: Improved attention for LLMs long-context training
  ☆213Updated 11 months ago
• thunlp / Modularity-Analysis
  [ACL 2023 Findings] Emergent Modularity in Pre-trained Transformers
  ☆26Updated 2 years ago
• snu-mllab / Context-Memory
  Pytorch implementation for "Compressed Context Memory For Online Language Model Interaction" (ICLR'24)
  ☆62Updated last year
• HazyResearch / skill-it
  Skill-It! A Data-Driven Skills Framework for Understanding and Training Language Models
  ☆48Updated 2 years ago
• whyNLP / LCKV
  Layer-Condensed KV cache w/ 10 times larger batch size, fewer params and less computation. Dramatic speed up with better task performance…
  ☆157Updated 9 months ago
• rabeehk / compacter
  ☆130Updated 3 years ago
• kyleliang919 / Online-Subspace-Descent
  [NeurIPS 2024] Low rank memory efficient optimizer without SVD
  ☆32Updated 6 months ago
• prateeky2806 / ties-merging
  ☆203Updated last year
• princeton-nlp / HELMET
  The HELMET Benchmark
  ☆197Updated last month
• JacobPfau / fillerTokens
  ☆75Updated last year