Alternatives and similar repositories for AdaZeta

Users that are interested in AdaZeta are comparing it to the libraries listed below

• yifanycc / loretta
  [NAACL 24 Oral] LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models
  ☆35Updated 5 months ago
• OPTML-Group / DeepZero
  [ICLR'24] "DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training" by Aochuan Chen*, Yimeng Zhang*, Jinghan Jia, James Di…
  ☆60Updated 8 months ago
• cjyaras / deep-lora-transformers
  Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation (ICML'24 Oral)
  ☆13Updated 11 months ago
• ZO-Bench / ZO-LLM
  [ICML 2024] Official code for the paper "Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark ".
  ☆105Updated last year
• alvin-zyl / CoLA
  Implementation of CoLA: Compute-Efficient Pre-Training of LLMs via Low-Rank Activation
  ☆22Updated 4 months ago
• andyjm3 / SLTrain
  SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining (NeurIPS 2024)
  ☆31Updated 7 months ago
• pvti / Awesome-Tensor-Decomposition
  😎 A curated list of tensor decomposition resources for model compression.
  ☆69Updated this week
• IlanPrice / DCTpS
  Code for testing DCT plus Sparse (DCTpS) networks
  ☆14Updated 4 years ago
• zqOuO / GWT
  ☆13Updated 5 months ago
• yxli2123 / LoSparse
  ☆57Updated last year
• Yanjun-Zhao / HiZOO
  Second-Order Fine-Tuning without Pain for LLMs: a Hessian Informed Zeroth-Order Optimizer
  ☆15Updated 4 months ago
• UNITES-Lab / moe-quantization
  Official code for the paper "Examining Post-Training Quantization for Mixture-of-Experts: A Benchmark"
  ☆20Updated last year
• YefanZhou / TempBalance
  [NeurIPS 2023 Spotlight] Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training
  ☆35Updated 2 months ago
• VITA-Group / Junk_DNA_Hypothesis
  [ICML 2024] Junk DNA Hypothesis: A Task-Centric Angle of LLM Pre-trained Weights through Sparsity; Lu Yin*, Ajay Jaiswal*, Shiwei Liu, So…
  ☆16Updated 2 months ago
• zyxxmu / DSnoT
  Official Pytorch Implementation of Our Paper Accepted at ICLR 2024-- Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLM…
  ☆47Updated last year
• SqueezeAILab / SqueezedAttention
  SQUEEZED ATTENTION: Accelerating Long Prompt LLM Inference
  ☆48Updated 7 months ago
• allenbai01 / transformers-as-statisticians
  ☆32Updated last year
• jiwonsong-dev / SLEB
  Official Implementation of SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blocks
  ☆37Updated 4 months ago
• QingruZhang / PLATON
  This pytorch package implements PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance (ICML 2022).
  ☆46Updated 2 years ago
• hahnyuan / ASVD4LLM
  Activation-aware Singular Value Decomposition for Compressing Large Language Models
  ☆72Updated 8 months ago
• Leiay / looped_transformer
  ☆28Updated last year
• UbiquitousLearning / Backpropagation_Free_Training_Survey
  ☆23Updated last year
• zyushun / hessian-spectrum
  Code for the paper: Why Transformers Need Adam: A Hessian Perspective
  ☆59Updated 3 months ago
• Jiacheng-Zhu-AIML / AsymmetryLoRA
  Preprint: Asymmetry in Low-Rank Adapters of Foundation Models
  ☆35Updated last year
• fmfi-compbio / admm-pruning
  ☆28Updated 11 months ago
• MarlonBecker / MSAM
  ☆19Updated last year
• epfml / optML-pku
  summer school materials
  ☆44Updated last year
• TNAS-DCS / TNAS-DCS
  ☆13Updated 2 years ago
• Raiden-Zhu / ICML-2023-DSGD-and-SAM
  [ICML 2023] Decentralized SGD and Average-direction SAM are Asymptotically Equivalent
  ☆19Updated last year
• machine-discovery / deer
  Parallelizing non-linear sequential models over the sequence length
  ☆52Updated this week