UbiquitousLearning / Backpropagation_Free_Training_SurveyLinks
☆23Updated last year
Alternatives and similar repositories for Backpropagation_Free_Training_Survey
Users that are interested in Backpropagation_Free_Training_Survey are comparing it to the libraries listed below
Sorting:
- [ICML 2024] Official code for the paper "Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark ".☆105Updated last year
- [ICLR'24] "DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training" by Aochuan Chen*, Yimeng Zhang*, Jinghan Jia, James Di…☆60Updated 8 months ago
- Second-Order Fine-Tuning without Pain for LLMs: a Hessian Informed Zeroth-Order Optimizer☆15Updated 4 months ago
- ☆36Updated 2 years ago
- A curated list of early exiting (LLM, CV, NLP, etc)☆55Updated 10 months ago
- Activation-aware Singular Value Decomposition for Compressing Large Language Models☆72Updated 8 months ago
- ☆32Updated last year
- Preprint: Asymmetry in Low-Rank Adapters of Foundation Models☆35Updated last year
- ☆57Updated last year
- Efficient LLM Inference Acceleration using Prompting☆48Updated 8 months ago
- The official implementation of TinyTrain [ICML '24]☆22Updated 11 months ago
- This is an official repository for "LAVA: Data Valuation without Pre-Specified Learning Algorithms" (ICLR2023).☆48Updated last year
- Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costs☆19Updated 6 months ago
- An implementation of the penalty-based bilevel gradient descent (PBGD) algorithm and the iterative differentiation (ITD/RHG) methods.☆18Updated 2 years ago
- Code for the paper: Why Transformers Need Adam: A Hessian Perspective☆59Updated 3 months ago
- ☆44Updated 10 months ago
- Libraries for efficient and scalable group-structured dataset pipelines.☆26Updated last week
- ☆46Updated last year
- SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining (NeurIPS 2024)☆31Updated 7 months ago
- ☆99Updated last year
- Official code implementation for 2025 ICLR accepted paper "Dobi-SVD : Differentiable SVD for LLM Compression and Some New Perspectives"☆34Updated 3 months ago
- [EMNLP 24] Source code for paper 'AdaZeta: Adaptive Zeroth-Order Tensor-Train Adaption for Memory-Efficient Large Language Models Fine-Tu…☆11Updated 6 months ago
- [ICML 2023] Decentralized SGD and Average-direction SAM are Asymptotically Equivalent☆19Updated last year
- Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation (ICML'24 Oral)☆13Updated 11 months ago
- Deep Learning & Information Bottleneck☆60Updated last year
- This is the official implementation of the ICML 2023 paper - Can Forward Gradient Match Backpropagation ?☆12Updated 2 years ago
- ☆11Updated 2 years ago
- LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters☆35Updated 3 months ago
- [ICML 2021] "Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training" by Shiwei Liu, Lu Yin, De…☆45Updated last year
- Sharpness-Aware Minimization Leads to Low-Rank Features [NeurIPS 2023]☆28Updated last year