UbiquitousLearning / Backpropagation_Free_Training_SurveyLinks
☆25Updated last year
Alternatives and similar repositories for Backpropagation_Free_Training_Survey
Users that are interested in Backpropagation_Free_Training_Survey are comparing it to the libraries listed below
Sorting:
- Second-Order Fine-Tuning without Pain for LLMs: a Hessian Informed Zeroth-Order Optimizer☆20Updated 9 months ago
- [ICML‘24] Official code for the paper "Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark ".☆117Updated 4 months ago
- [ICLR'24] "DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training" by Aochuan Chen*, Yimeng Zhang*, Jinghan Jia, James Di…☆65Updated last year
- A curated list of early exiting (LLM, CV, NLP, etc)☆68Updated last year
- Preprint: Asymmetry in Low-Rank Adapters of Foundation Models☆37Updated last year
- AN EFFICIENT AND GENERAL FRAMEWORK FOR LAYERWISE-ADAPTIVE GRADIENT COMPRESSION☆14Updated 2 years ago
- ☆35Updated last year
- LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning☆36Updated last year
- The official implementation of TinyTrain [ICML '24]☆23Updated last year
- SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining (NeurIPS 2024)☆36Updated last year
- ☆35Updated 2 years ago
- Survey Paper List - Efficient LLM and Foundation Models☆257Updated last year
- [ICML 2024] Model Tailor: Mitigating Catastrophic Forgetting in Multi-modal Large Language Models☆32Updated last year
- ☆221Updated 2 years ago
- [EMNLP 24] Source code for paper 'AdaZeta: Adaptive Zeroth-Order Tensor-Train Adaption for Memory-Efficient Large Language Models Fine-Tu…☆12Updated 11 months ago
- Code for the paper: Why Transformers Need Adam: A Hessian Perspective☆63Updated 8 months ago
- AdaMerging: Adaptive Model Merging for Multi-Task Learning. ICLR, 2024.☆96Updated last year
- Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costs☆21Updated 2 weeks ago
- Using PyTorch autograd to compute Hessian of Perplexity for Large Language Models☆26Updated 7 months ago
- [TPAMI 2023] Low Dimensional Landscape Hypothesis is True: DNNs can be Trained in Tiny Subspaces☆42Updated 3 years ago
- This is an official repository for "LAVA: Data Valuation without Pre-Specified Learning Algorithms" (ICLR2023).☆51Updated last year
- Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation (ICML'24 Oral)☆13Updated last year
- Efficient LLM Inference Acceleration using Prompting☆51Updated last year
- Selective Aggregation for Low-Rank Adaptation in Federated Learning [ICLR 2025]☆51Updated 7 months ago
- This repository contains the implementation of the paper "MeteoRA: Multiple-tasks Embedded LoRA for Large Language Models".☆24Updated 6 months ago
- Federated reconnaissance mini-ImageNet benchmark and baseline models☆13Updated 4 years ago
- An implementation of the penalty-based bilevel gradient descent (PBGD) algorithm and the iterative differentiation (ITD/RHG) methods.☆19Updated 2 years ago
- ☆99Updated 10 months ago
- The official implement of paper "Does Federated Learning Really Need Backpropagation?"☆23Updated 2 years ago
- ☆101Updated last year