Alternatives and similar repositories for MeZO

Users that are interested in MeZO are comparing it to the libraries listed below

• OpenLMLab / LOMO
  LOMO: LOw-Memory Optimization
  ☆988Updated last year
• locuslab / wanda
  A simple and effective LLM pruning approach.
  ☆810Updated last year
• IST-DASLab / sparsegpt
  Code for the ICML 2023 paper "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot".
  ☆843Updated last year
• XueFuzhao / OpenMoE
  A family of open-sourced Mixture-of-Experts (MoE) Large Language Models
  ☆1,614Updated last year
• princeton-nlp / LLM-Shearing
  [ICLR 2024] Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
  ☆631Updated last year
• datamllab / LongLM
  [ICML'24 Spotlight] LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning
  ☆662Updated last year
• yule-BUAA / MergeLM
  Codebase for Merging Language Models (ICML 2024)
  ☆853Updated last year
• lucidrains / self-rewarding-lm-pytorch
  Implementation of the training framework proposed in Self-Rewarding Language Model, from MetaAI
  ☆1,399Updated last year
• jquesnelle / yarn
  YaRN: Efficient Context Window Extension of Large Language Models
  ☆1,619Updated last year
• Liuhong99 / Sophia
  The official implementation of “Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training”
  ☆976Updated last year
• tatsu-lab / alpaca_farm
  A simulation framework for RLHF and alternatives. Develop your RLHF method without collecting human data.
  ☆825Updated last year
• abertsch72 / unlimiformer
  Public repo for the NeurIPS 2023 paper "Unlimiformer: Long-Range Transformers with Unlimited Length Input"
  ☆1,062Updated last year
• sail-sg / lorahub
  [COLM 2024] LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition
  ☆655Updated last year
• hao-ai-lab / LookaheadDecoding
  [ICML 2024] Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
  ☆1,288Updated 7 months ago
• Guitaricet / relora
  Official code for ReLoRA from the paper Stack More Layers Differently: High-Rank Training Through Low-Rank Updates
  ☆465Updated last year
• declare-lab / instruct-eval
  This repository contains code to quantitatively evaluate instruction-tuned models such as Alpaca and Flan-T5 on held-out tasks.
  ☆548Updated last year
• lucidrains / MEGABYTE-pytorch
  Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch
  ☆653Updated 9 months ago
• Vahe1994 / SpQR
  ☆546Updated 10 months ago
• tomaarsen / attention_sinks
  Extend existing LLMs way beyond the original training length with constant memory usage, without retraining
  ☆722Updated last year
• arielnlee / Platypus
  Code for fine-tuning Platypus fam LLMs using LoRA
  ☆628Updated last year
• AGI-Edgerunners / LLM-Adapters
  Code for our EMNLP 2023 Paper: "LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models"
  ☆1,201Updated last year
• horseee / LLM-Pruner
  [NeurIPS 2023] LLM-Pruner: On the Structural Pruning of Large Language Models. Support Llama-3/3.1, Llama-2, LLaMA, BLOOM, Vicuna, Baich…
  ☆1,072Updated last year
• kmeng01 / memit
  Mass-editing thousands of facts into a transformer memory (ICLR 2023)
  ☆519Updated last year
• pratyushasharma / laser
  The Truth Is In There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction
  ☆388Updated last year
• IBM / Dromedary
  Dromedary: towards helpful, ethical and reliable LLMs.
  ☆1,142Updated last month
• PiotrNawrot / nanoT5
  Fast & Simple repository for pre-training and fine-tuning T5-style models
  ☆1,011Updated last year
• epfml / landmark-attention
  Landmark Attention: Random-Access Infinite Context Length for Transformers
  ☆426Updated last year
• ContextualAI / HALOs
  A library with extensible implementations of DPO, KTO, PPO, ORPO, and other human-aware loss functions (HALOs).
  ☆890Updated 3 weeks ago
• uclaml / SPIN
  The official implementation of Self-Play Fine-Tuning (SPIN)
  ☆1,206Updated last year
• allenai / mmc4
  MultimodalC4 is a multimodal extension of c4 that interleaves millions of images with text.
  ☆941Updated 7 months ago