Ledzy / BAdamLinks
[NeurIPS 2024] BAdam: A Memory Efficient Full Parameter Optimization Method for Large Language Models
☆258Updated 2 months ago
Alternatives and similar repositories for BAdam
Users that are interested in BAdam are comparing it to the libraries listed below
Sorting:
- A recipe for online RLHF and online iterative DPO.☆514Updated 5 months ago
- APOLLO: SGD-like Memory, AdamW-level Performance☆232Updated last month
- Train your Agent model via our easy and efficient framework☆776Updated this week
- PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models(NeurIPS 2024 Spotlight)☆356Updated 4 months ago
- ☆195Updated 3 weeks ago
- adds Sequence Parallelism into LLaMA-Factory☆498Updated this week
- The official implementation of Self-Play Preference Optimization (SPPO)☆563Updated 4 months ago
- minimal-cost for training 0.5B R1-Zero☆730Updated 3 weeks ago
- 使用deepspeed从头开 始训练一个LLM,经过pretrain和sft阶段,验证llm学习知识、理解语言、回答问题的能力☆154Updated 10 months ago
- Codebase for Iterative DPO Using Rule-based Rewards☆245Updated last month
- ☆198Updated 7 months ago
- ☆188Updated last month
- A highly capable 2.4B lightweight LLM using only 1T pre-training data with all details.☆182Updated this week
- [ICLR 2025🔥] SVD-LLM & [NAACL 2025🔥] SVD-LLM V2☆208Updated 2 months ago
- ☆201Updated 3 months ago
- LoRAMoE: Revolutionizing Mixture of Experts for Maintaining World Knowledge in Language Model Alignment☆339Updated last year
- MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models☆432Updated last year
- Controllable Text Generation for Large Language Models: A Survey☆175Updated 9 months ago
- The framework to prune LLMs to any size and any config.☆92Updated last year
- verl-agent is an extension of veRL, designed for training LLM/VLM agents via RL. verl-agent is also the official code for paper "Group-in…☆232Updated this week
- Recipes to train reward model for RLHF.☆1,356Updated last month
- ☆138Updated 10 months ago
- [SIGIR'24] The official implementation code of MOELoRA.☆167Updated 10 months ago
- [ACL 2024] Long-Context Language Modeling with Parallel Encodings☆153Updated 11 months ago
- ☆174Updated 10 months ago
- Explore concepts like Self-Correct, Self-Refine, Self-Improve, Self-Contradict, Self-Play, and Self-Knowledge, alongside o1-like reasonin…☆168Updated 5 months ago
- Official codebase for "Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling".☆261Updated 3 months ago
- [ACL 2024] User-friendly evaluation framework: Eval Suite & Benchmarks: UHGEval, HaluEval, HalluQA, etc.☆167Updated 6 months ago
- TransMLA: Multi-Head Latent Attention Is All You Need☆284Updated this week
- ✨ A synthetic dataset generation framework that produces diverse coding questions and verifiable solutions - all in one framwork☆226Updated 2 months ago