XU-YIJIE / hobo-llm-from-scratchLinks
From Llama to Deepseek, grpo/mtp implemented. With pt/sft/lora/qlora included
☆26Updated 3 months ago
Alternatives and similar repositories for hobo-llm-from-scratch
Users that are interested in hobo-llm-from-scratch are comparing it to the libraries listed below
Sorting:
- Train your grpo with zero dataset and low resources, 8bit/4bit/lora/qlora supported, multi-gpu supported ...☆75Updated 3 months ago
- A visuailzation tool to make deep understaning and easier debugging for RLHF training.☆241Updated 5 months ago
- RLHF experiments on a single A100 40G GPU. Support PPO, GRPO, REINFORCE, RAFT, RLOO, ReMax, DeepSeek R1-Zero reproducing.☆68Updated 5 months ago
- Trinity-RFT is a general-purpose, flexible and scalable framework designed for reinforcement fine-tuning (RFT) of large language models (…☆218Updated last week
- ☆20Updated 4 months ago
- ☆65Updated 8 months ago
- ☆113Updated 9 months ago
- 使用单个24G显卡,从0开始训练LLM☆55Updated last month
- 怎么训练一个LLM分词器☆152Updated 2 years ago
- OpenSeek aims to unite the global open source community to drive collaborative innovation in algorithms, data and systems to develop next…☆221Updated last week
- This is a repo for showcasing using MCTS with LLMs to solve gsm8k problems☆86Updated 4 months ago
- ☆125Updated last year
- ☆84Updated last year
- [ACL 2024 Demo] Official GitHub repo for UltraEval: An open source framework for evaluating foundation models.☆246Updated 9 months ago
- an implementation of transformer, bert, gpt, and diffusion models for learning purposes☆155Updated 10 months ago
- ☆108Updated last year
- ☆144Updated last year
- pytorch分布式训练☆67Updated 2 years ago
- ☆174Updated last year
- ☆36Updated 7 months ago
- ☆129Updated last year
- ☆145Updated last year
- Counting-Stars (★)☆83Updated 2 months ago
- Finetuning LLaMA with RLHF (Reinforcement Learning with Human Feedback) based on DeepSpeed Chat☆114Updated 2 years ago
- Code for a New Loss for Mitigating the Bias of Learning Difficulties in Generative Language Models☆65Updated 5 months ago
- Max的有趣数据集 / Max's awesome datasets☆34Updated 3 months ago
- llm & rl☆182Updated 2 weeks ago
- A flexible and efficient training framework for large-scale alignment tasks☆403Updated this week
- Llama-3-SynE: A Significantly Enhanced Version of Llama-3 with Advanced Scientific Reasoning and Chinese Language Capabilities | 继续预训练提升 …☆34Updated 2 months ago
- The related works and background techniques about Openai o1☆224Updated 7 months ago