Alternatives and similar repositories for recurrent-pretraining:

Users that are interested in recurrent-pretraining are comparing it to the libraries listed below

• facebookresearch / coconut
  Training Large Language Model to Reason in a Continuous Latent Space
  ☆1,062Updated 3 months ago
• microsoft / rStar
  ☆519Updated last week
• HKUNLP / Dream
  Dream 7B, a large diffusion language model
  ☆572Updated 2 weeks ago
• sail-sg / understand-r1-zero
  Understanding R1-Zero-Like Training: A Critical Perspective
  ☆882Updated last week
• huggingface / search-and-learn
  Recipes to scale inference-time compute of open models
  ☆1,058Updated 2 months ago
• GAIR-NLP / LIMO
  LIMO: Less is More for Reasoning
  ☆920Updated 2 weeks ago
• sail-sg / oat
  🌾 OAT: A research-friendly framework for LLM online alignment, including preference learning, reinforcement learning, etc.
  ☆325Updated this week
• facebookresearch / memory
  Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, spars…
  ☆317Updated 4 months ago
• willccbb / verifiers
  Verifiers for LLM Reinforcement Learning
  ☆827Updated 3 weeks ago
• groundlight / r1_vlm
  Build your own visual reasoning model
  ☆341Updated this week
• SimpleBerry / LLaMA-O1
  Large Reasoning Models
  ☆802Updated 4 months ago
• open-thought / reasoning-gym
  procedural reasoning datasets
  ☆571Updated this week
• KellerJordan / Muon
  Muon optimizer: +>30% sample efficiency with <3% wallclock overhead
  ☆577Updated last month
• ML-GSAI / LLaDA
  Official PyTorch implementation for "Large Language Diffusion Models"
  ☆1,520Updated 2 weeks ago
• microsoft / Samba
  [ICLR 2025] Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling
  ☆862Updated 2 months ago
• McGill-NLP / nano-aha-moment
  Single File, Single GPU, From Scratch, Efficient, Full Parameter Tuning library for "RL for LLMs"
  ☆385Updated 2 weeks ago
• srush / awesome-o1
  A bibliography and survey of the papers surrounding o1
  ☆1,187Updated 5 months ago
• zhentingqi / rStar
  ☆922Updated 3 months ago
• kuleshov-group / bd3lms
  Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models
  ☆551Updated last week
• allenai / OLMoE
  OLMoE: Open Mixture-of-Experts Language Models
  ☆716Updated last month
• ekinakyurek / marc
  Public repository for "The Surprising Effectiveness of Test-Time Training for Abstract Reasoning"
  ☆304Updated 5 months ago
• BytedTsinghua-SIA / DAPO
  An Open-source RL System from ByteDance Seed and Tsinghua AIR
  ☆1,171Updated 2 weeks ago
• RAGEN-AI / RAGEN
  RAGEN leverages reinforcement learning to train LLM reasoning agents in interactive, stochastic environments.
  ☆1,462Updated this week
• MoonshotAI / Moonlight
  Muon is Scalable for LLM Training
  ☆1,029Updated 3 weeks ago
• huggingface / Math-Verify
  ☆647Updated 3 weeks ago
• SWE-Gym / SWE-Gym
  Code for Paper: Training Software Engineering Agents and Verifiers with SWE-Gym
  ☆438Updated 3 weeks ago
• mlfoundations / evalchemy
  Automatic evals for LLMs
  ☆373Updated this week
• AIDC-AI / Marco-o1
  An Open Large Reasoning Model for Real-World Solutions
  ☆1,483Updated last month
• facebookresearch / swe-rl
  Official codebase for "SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution"
  ☆509Updated last month
• RyanLiu112 / compute-optimal-tts
  Official codebase for "Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling".
  ☆252Updated 2 months ago