[ICLR'24] "DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training" by Aochuan Chen*, Yimeng Zhang*, Jinghan Jia, James Diffenderfer, Jiancheng Liu, Konstantinos Parasyris, Yihua Zhang, Zheng Zhang, Bhavya Kailkhura, Sijia Liu
☆70Oct 9, 2024Updated last year
Alternatives and similar repositories for DeepZero
Users that are interested in DeepZero are comparing it to the libraries listed below
Sorting:
- [ICML‘24] Official code for the paper "Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark ".☆124Jul 6, 2025Updated 7 months ago
- Second-Order Fine-Tuning without Pain for LLMs: a Hessian Informed Zeroth-Order Optimizer☆23Feb 11, 2025Updated last year
- Code the ICML 2024 paper: "Variance-reduced Zeroth-Order Methods for Fine-Tuning Language Models"☆12Jun 25, 2024Updated last year
- Zeroth-Order Fine-Tuning of LLMs in Random Subspaces (ICCV 2025)☆16Nov 22, 2024Updated last year
- [NeurIPS 2023] MeZO: Fine-Tuning Language Models with Just Forward Passes. https://arxiv.org/abs/2305.17333☆1,149Jan 11, 2024Updated 2 years ago
- [EMNLP 24] Source code for paper 'AdaZeta: Adaptive Zeroth-Order Tensor-Train Adaption for Memory-Efficient Large Language Models Fine-Tu…☆12Dec 15, 2024Updated last year
- This is the official implementation of the ICML 2023 paper - Can Forward Gradient Match Backpropagation ?☆13May 31, 2023Updated 2 years ago
- [ICLR 2023] Eva: Practical Second-order Optimization with Kronecker-vectorized Approximation☆12Jul 31, 2023Updated 2 years ago
- Grams: Gradient Descent with Adaptive Momentum Scaling (ICLR 2025 Workshop)☆17Mar 6, 2025Updated 11 months ago
- Source code for the TMLR paper "Black-Box Prompt Learning for Pre-trained Language Models"☆57Sep 7, 2023Updated 2 years ago
- PyTorch-based auto-differentiable orbital-free density functional theory package☆13Mar 19, 2024Updated last year
- ZOSVRG-BlackBox-Adv☆13Oct 30, 2018Updated 7 years ago
- ☆20Aug 16, 2021Updated 4 years ago
- LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning☆36Apr 4, 2024Updated last year
- Implementation of "Gradients without backpropagation" paper (https://arxiv.org/abs/2202.08587) using functorch☆114Jun 14, 2023Updated 2 years ago
- This is a PyTorch implementation of Kaggle's Cassava Disease Visual Classification challenge (5th place in private leaderboard)☆12Jun 9, 2019Updated 6 years ago
- ☆19Dec 12, 2023Updated 2 years ago
- ☆19Apr 10, 2017Updated 8 years ago
- Official implementation of ICML 2024 paper "ExCP: Extreme LLM Checkpoint Compression via Weight-Momentum Joint Shrinking".☆47Jul 12, 2024Updated last year
- Revisiting Parameter Sharing for Automatic Neural Channel Number Search, NeurIPS 2020☆22Nov 15, 2020Updated 5 years ago
- [ICLR 2024 Spotlight] "Negative Label Guided OOD Detection with Pretrained Vision-Language Models"☆21Oct 23, 2024Updated last year
- Official code for the ICLR 2025 paper, "Scaling Offline Model-Based RL via Jointly-Optimized World-Action Model Pretraining"☆28Dec 1, 2024Updated last year
- "ZINB-based Graph Embedding Autoencoder for Single-cell RNA-seq Interpretations" in AAAI 2022☆25Feb 5, 2023Updated 3 years ago
- ☆57Jun 10, 2024Updated last year
- This repo contains code for the paper: "Can Foundation Models Help Us Achieve Perfect Secrecy?"☆24Feb 9, 2023Updated 3 years ago
- ☆27Mar 21, 2024Updated last year
- Official code for the paper "Examining Post-Training Quantization for Mixture-of-Experts: A Benchmark"☆29Jun 30, 2025Updated 8 months ago
- Robustify Black-Box Models (ICLR'22 - Spotlight)☆24Jan 29, 2023Updated 3 years ago
- ☆28Dec 2, 2024Updated last year
- Software Engineering Back End Microservices Project☆15Nov 20, 2024Updated last year
- [ICML2022] Training Your Sparse Neural Network Better with Any Mask. Ajay Jaiswal, Haoyu Ma, Tianlong Chen, ying Ding, and Zhangyang Wang☆30Jul 24, 2022Updated 3 years ago
- ☆34Aug 23, 2023Updated 2 years ago
- source code for paper "Riemannian Preconditioned LoRA for Fine-Tuning Foundation Models"☆34Jun 20, 2024Updated last year
- ☆31Oct 13, 2023Updated 2 years ago
- ☆27Nov 9, 2022Updated 3 years ago
- ☆29Nov 29, 2023Updated 2 years ago
- Implementation of the Budgeted Super Networks☆25Feb 25, 2019Updated 7 years ago
- ☆30Sep 5, 2021Updated 4 years ago
- ☆28Jun 27, 2022Updated 3 years ago