Alternatives and similar repositories for Backpropagation_Free_Training_Survey

Users that are interested in Backpropagation_Free_Training_Survey are comparing it to the libraries listed below

• ZO-Bench / ZO-LLM
  [ICML‘24] Official code for the paper "Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark ".
  ☆124Updated 6 months ago
• OPTML-Group / DeepZero
  [ICLR'24] "DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training" by Aochuan Chen*, Yimeng Zhang*, Jinghan Jia, James Di…
  ☆70Updated last year
• Yanjun-Zhao / HiZOO
  Second-Order Fine-Tuning without Pain for LLMs: a Hessian Informed Zeroth-Order Optimizer
  ☆22Updated 11 months ago
• falcon-xu / early-exit-papers
  A curated list of early exiting (LLM, CV, NLP, etc)
  ☆70Updated last year
• kwignb / NeuralTangentKernel-Papers
  Neural Tangent Kernel Papers
  ☆121Updated last year
• andyjm3 / SLTrain
  SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining (NeurIPS 2024)
  ☆39Updated last year
• sahagobinda / SGP
  Official [AAAI] Code Repository for "Continual Learning with Scaled Gradient Projection".
  ☆15Updated 2 years ago
• hanshen95 / penalized-bilevel-gradient-descent
  An implementation of the penalty-based bilevel gradient descent (PBGD) algorithm and the iterative differentiation (ITD/RHG) methods.
  ☆19Updated 2 years ago
• lliai / Awesome-Low-Rank-Adaptation
  Awesome-Low-Rank-Adaptation
  ☆127Updated last year
• cjyaras / deep-lora-transformers
  Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation (ICML'24 Oral)
  ☆13Updated last year
• zyushun / hessian-spectrum
  Code for the paper: Why Transformers Need Adam: A Hessian Perspective
  ☆63Updated 10 months ago
• theyoungkwon / TinyTrain
  The official implementation of TinyTrain [ICML '24]
  ☆23Updated last year
• epfml / optML-pku
  summer school materials
  ☆46Updated 2 years ago
• Jiacheng-Zhu-AIML / AsymmetryLoRA
  Preprint: Asymmetry in Low-Rank Adapters of Foundation Models
  ☆37Updated last year
• UbiquitousLearning / Efficient_Foundation_Model_Survey
  Survey Paper List - Efficient LLM and Foundation Models
  ☆260Updated last year
• osehmathias / lisa
  LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning
  ☆36Updated last year
• locuslab / edge-of-stability
  ☆73Updated last year
• ycjing / Awesome-Model-Merging
  A curated list of Model Merging methods.
  ☆96Updated last month
• baekrok / DASH-Direction-Aware-SHrinking
  ☆13Updated last year
• vectozavr / llm-hessian
  Using PyTorch autograd to compute Hessian of Perplexity for Large Language Models
  ☆27Updated 9 months ago
• drimpossible / BudgetCL
  Code for CVPR paper: Computationally Budgeted Continual Learning: What Does Matter?
  ☆18Updated last year
• AngusDujw / SAF
  ☆35Updated 3 years ago
• Raiden-Zhu / ICML-2023-DSGD-and-SAM
  [ICML 2023] Decentralized SGD and Average-direction SAM are Asymptotically Equivalent
  ☆20Updated 2 years ago
• EnnengYang / AdaMerging
  AdaMerging: Adaptive Model Merging for Multi-Task Learning. ICLR, 2024.
  ☆100Updated last year
• neu-spiral / SparCL
  SparCL: Sparse Continual Learning on the Edge @ NeurIPS 22
  ☆30Updated 2 years ago
• alvin-zyl / CoLA
  Implementation of CoLA: Compute-Efficient Pre-Training of LLMs via Low-Rank Activation
  ☆25Updated 11 months ago
• LGrCo / L-GreCo
  AN EFFICIENT AND GENERAL FRAMEWORK FOR LAYERWISE-ADAPTIVE GRADIENT COMPRESSION
  ☆14Updated 2 years ago
• AIoT-MLSys-Lab / FedAIoT
  [DMLR 2024] FedAIoT: A Federated Learning Benchmark for Artificial Intelligence of Things
  ☆59Updated last year
• UbiquitousLearning / FwdLLM
  ☆35Updated last year
• tanganke / weight-ensembling_MoE
  Code for paper "Merging Multi-Task Models via Weight-Ensembling Mixture of Experts"
  ☆30Updated last year