XU-YIJIE / hobo-llm-from-scratchLinks
From Llama to Deepseek, grpo/mtp implemented. With pt/sft/lora/qlora included
☆27Updated 2 months ago
Alternatives and similar repositories for hobo-llm-from-scratch
Users that are interested in hobo-llm-from-scratch are comparing it to the libraries listed below
Sorting:
- Train your grpo with zero dataset and low resources, 8bit/4bit/lora/qlora supported, multi-gpu supported ...☆73Updated last month
- ☆63Updated 7 months ago
- RLHF experiments on a single A100 40G GPU. Support PPO, GRPO, REINFORCE, RAFT, RLOO, ReMax, DeepSeek R1-Zero reproducing.☆64Updated 4 months ago
- 使用单个24G显卡,从0开始训练LLM☆55Updated last month
- This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…☆57Updated last year
- Finetuning LLaMA with RLHF (Reinforcement Learning with Human Feedback) based on DeepSpeed Chat☆114Updated 2 years ago
- This is a repo for showcasing using MCTS with LLMs to solve gsm8k problems☆84Updated 3 months ago
- A curated list of awesome works in Routing LLMs paradigm (👉 Welcome to submit your contributions to this code repository)☆40Updated last month
- A visuailzation tool to make deep understaning and easier debugging for RLHF training.☆216Updated 4 months ago
- ☆33Updated 9 months ago
- A Comprehensive Survey on Long Context Language Modeling☆152Updated 3 weeks ago
- ☆33Updated 3 weeks ago
- Trinity-RFT is a general-purpose, flexible and scalable framework designed for reinforcement fine-tuning (RFT) of large language models (…☆127Updated this week
- 基于DPO算法微调语言大模型,简单好上手。☆39Updated 11 months ago
- ☆34Updated 6 months ago
- Pretrain、decay、SFT a CodeLLM from scratch 🧙♂️☆36Updated last year
- ☆84Updated last year
- Reinforcement Learning in LLM and NLP.☆39Updated last week
- ☆18Updated 2 months ago
- realize the reinforcement learning training for gpt2 llama bloom and so on llm model☆25Updated last year
- 怎么训练一个LLM分词器☆150Updated last year
- ☆150Updated last week
- Official completion of “Training on the Benchmark Is Not All You Need”.☆34Updated 5 months ago
- ☆220Updated last month
- A high-throughput and memory-efficient inference and serving engine for LLMs☆30Updated 5 months ago
- ☆19Updated last week
- CLongEval: A Chinese Benchmark for Evaluating Long-Context Large Language Models☆40Updated last year
- [NeurIPS 2024] Fast Best-of-N Decoding via Speculative Rejection☆45Updated 7 months ago
- ☆95Updated 6 months ago
- ☆103Updated 6 months ago