liangyuwang / zo2
ZO2 (Zeroth-Order Offloading): Full Parameter Fine-Tuning 175B LLMs with 18GB GPU Memory
☆93Updated this week
Alternatives and similar repositories for zo2:
Users that are interested in zo2 are comparing it to the libraries listed below
- CoT-Valve: Length-Compressible Chain-of-Thought Tuning☆67Updated 2 months ago
- Official code of *Virgo: A Preliminary Exploration on Reproducing o1-like MLLM*☆100Updated 2 months ago
- Rethinking RL Scaling for Vision Language Models: A Transparent, From-Scratch Framework and Comprehensive Evaluation Scheme☆120Updated 3 weeks ago
- Efficient Mixture of Experts for LLM Paper List☆64Updated 4 months ago
- This is a repo for showcasing using MCTS with LLMs to solve gsm8k problems☆75Updated last month
- ICML2025: Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning☆39Updated this week
- Official code for our paper, "LoRA-Pro: Are Low-Rank Adapters Properly Optimized? "☆116Updated last month
- Open-Pandora: On-the-fly Control Video Generation☆34Updated 5 months ago
- ☆73Updated last year
- SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuning. COLM 2024 Accepted Paper☆32Updated 11 months ago
- ☆37Updated 3 weeks ago
- ☆95Updated last month
- Official code of *Virgo: A Preliminary Exploration on Reproducing o1-like MLLM*☆20Updated 2 months ago
- A Framework for Decoupling and Assessing the Capabilities of VLMs☆42Updated 10 months ago
- [ArXiv] V2PE: Improving Multimodal Long-Context Capability of Vision-Language Models with Variable Visual Position Encoding☆46Updated 4 months ago
- [EMNLP 2024 Findings🔥] Official implementation of ": LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context In…☆93Updated 5 months ago
- The huggingface implementation of Fine-grained Late-interaction Multi-modal Retriever.☆86Updated 3 months ago
- ☆80Updated 3 weeks ago
- From Hours to Minutes: Lossless Acceleration of Ultra Long Sequence Generation☆89Updated last month
- ☆100Updated 10 months ago
- Exploring Efficient Fine-Grained Perception of Multimodal Large Language Models☆60Updated 6 months ago
- [ACL 2024] The official codebase for the paper "Self-Distillation Bridges Distribution Gap in Language Model Fine-tuning".☆119Updated 6 months ago
- MMR1: Advancing the Frontiers of Multimodal Reasoning☆159Updated last month
- An benchmark for evaluating the capabilities of large vision-language models (LVLMs)☆46Updated last year
- ☆40Updated 2 months ago
- Reformatted Alignment☆115Updated 7 months ago
- CPPO: Accelerating the Training of Group Relative Policy Optimization-Based Reasoning Models☆123Updated this week
- Exploring the Limit of Outcome Reward for Learning Mathematical Reasoning☆175Updated last month
- A generalized framework for subspace tuning methods in parameter efficient fine-tuning.☆139Updated 3 months ago
- [ICLR 2025] SuperCorrect: Advancing Small LLM Reasoning with Thought Template Distillation and Self-Correction☆69Updated last month