Alternatives and similar repositories for awesome-llm-pretraining:

Users that are interested in awesome-llm-pretraining are comparing it to the libraries listed below

• LCLM-Horizon / A-Comprehensive-Survey-For-Long-Context-Language-Modeling
  A Comprehensive Survey on Long Context Language Modeling
  ☆131Updated last month
• KbsdJames / Omni-MATH
  The official repository of the Omni-MATH benchmark.
  ☆80Updated 4 months ago
• SuperGPQA / SuperGPQA
  ☆146Updated last month
• GAIR-NLP / LIMR
  ☆187Updated 2 months ago
• GAIR-NLP / ReAlign
  Reformatted Alignment
  ☆115Updated 7 months ago
• hkust-nlp / dart-math
  [NeurIPS'24] Official code for *🎯DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving*
  ☆104Updated 4 months ago
• princeton-nlp / ProLong
  Homepage for ProLong (Princeton long-context language models) and paper "How to Train Long-Context Language Models (Effectively)"
  ☆175Updated last month
• Wangmerlyn / MCTS-GSM8k-Demo
  This is a repo for showcasing using MCTS with LLMs to solve gsm8k problems
  ☆74Updated last month
• GAIR-NLP / ToRL
  ☆125Updated 3 weeks ago
• Zanette-Labs / SpeculativeRejection
  [NeurIPS 2024] Fast Best-of-N Decoding via Speculative Rejection
  ☆42Updated 5 months ago
• AlphaPav / mem-kk-logic
  On Memorization of Large Language Models in Logical Reasoning
  ☆65Updated 3 weeks ago
• SkyworkAI / skywork-o1-prm-inference
  ☆63Updated 5 months ago
• modelscope / Trinity-RFT
  Trinity-RFT is a general-purpose, flexible and scalable framework designed for reinforcement fine-tuning (RFT) of large language models (…
  ☆50Updated this week
• HKUNLP / critic-rl
  Code for Paper: Teaching Language Models to Critique via Reinforcement Learning
  ☆94Updated last week
• ganler / code-r1
  Reproducing R1 for Code with Reliable Rewards
  ☆179Updated this week
• RyanLiu112 / GenPRM
  Official codebase for "GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoning".
  ☆64Updated last week
• hemingkx / TokenSkip
  TokenSkip: Controllable Chain-of-Thought Compression in LLMs
  ☆133Updated last month
• step-law / steplaw
  ☆179Updated 2 weeks ago
• DAMO-NLP-SG / LongPO
  [ICLR 2025] LongPO: Long Context Self-Evolution of Large Language Models through Short-to-Long Preference Optimization
  ☆34Updated 2 months ago
• RUC-GSAI / YuLan-Mini
  A highly capable 2.4B lightweight LLM using only 1T pre-training data with all details.
  ☆174Updated 2 weeks ago
• multimodal-art-projection / COIG-P
  ☆37Updated 2 weeks ago
• InternLM / OREAL
  Exploring the Limit of Outcome Reward for Learning Mathematical Reasoning
  ☆173Updated last month
• agentica-project / verl-pipeline
  Async pipelined version of Verl
  ☆60Updated 2 weeks ago
• SkyworkAI / Skywork-MoE
  Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
  ☆132Updated 10 months ago
• OpenSparseLLMs / Linear-MoE
  ☆76Updated last week
• LLM360 / MegaMath
  An Open Math Pre-trainng Dataset with 370B Tokens.
  ☆72Updated 3 weeks ago
• CMU-AIRe / MRT
  Research Code for preprint "Optimizing Test-Time Compute via Meta Reinforcement Finetuning".
  ☆95Updated last month
• zhiyuanhubj / LongRecipe
  LongRecipe: Recipe for Efficient Long Context Generalization in Large Language Models
  ☆76Updated 6 months ago
• sail-sg / oat-zero
  A lightweight reproduction of DeepSeek-R1-Zero with indepth analysis of self-reflection behavior.
  ☆229Updated last week
• THUDM / T1
  Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling
  ☆101Updated 3 months ago