Alternatives and similar repositories for recurrent-pretraining:

Users that are interested in recurrent-pretraining are comparing it to the libraries listed below

• facebookresearch / coconut
  Training Large Language Model to Reason in a Continuous Latent Space
  ☆998Updated 2 months ago
• facebookresearch / memory
  Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, spars…
  ☆310Updated 3 months ago
• MoonshotAI / Moonlight
  Muon is Scalable for LLM Training
  ☆974Updated last month
• huggingface / search-and-learn
  Recipes to scale inference-time compute of open models
  ☆1,044Updated last month
• microsoft / rStar
  ☆485Updated last week
• groundlight / r1_vlm
  Build your own visual reasoning model
  ☆312Updated last week
• ML-GSAI / LLaDA
  Official PyTorch implementation for "Large Language Diffusion Models"
  ☆1,313Updated 2 weeks ago
• GAIR-NLP / LIMO
  LIMO: Less is More for Reasoning
  ☆864Updated last month
• sail-sg / understand-r1-zero
  Understanding R1-Zero-Like Training: A Critical Perspective
  ☆568Updated this week
• sail-sg / oat
  🌾 OAT: A research-friendly framework for LLM online alignment, including preference learning, reinforcement learning, etc.
  ☆283Updated this week
• KellerJordan / Muon
  Muon optimizer: +>30% sample efficiency with <3% wallclock overhead
  ☆529Updated this week
• open-thought / reasoning-gym
  procedural reasoning datasets
  ☆534Updated this week
• willccbb / verifiers
  Verifiers for LLM Reinforcement Learning
  ☆686Updated this week
• qixucen / atom
  ☆514Updated last week
• huggingface / Math-Verify
  ☆559Updated 2 weeks ago
• microsoft / Samba
  Official implementation of "Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling"
  ☆857Updated last month
• SimpleBerry / LLaMA-O1
  Large Reasoning Models
  ☆800Updated 3 months ago
• srush / awesome-o1
  A bibliography and survey of the papers surrounding o1
  ☆1,183Updated 4 months ago
• SWE-Gym / SWE-Gym
  Code for Paper: Training Software Engineering Agents and Verifiers with SWE-Gym
  ☆403Updated 2 weeks ago
• ekinakyurek / marc
  Public repository for "The Surprising Effectiveness of Test-Time Training for Abstract Reasoning"
  ☆300Updated 4 months ago
• zhentingqi / rStar
  ☆910Updated 2 months ago
• Haiyang-W / TokenFormer
  [ICLR2025 Spotlight🔥] Official Implementation of TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters
  ☆539Updated last month
• BytedTsinghua-SIA / DAPO
  An Open-source RL System from ByteDance Seed and Tsinghua AIR
  ☆767Updated last week
• trotsky1997 / MathBlackBox
  ☆1,011Updated 3 months ago
• kuleshov-group / bd3lms
  Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models
  ☆397Updated this week
• ezelikman / quiet-star
  Code for Quiet-STaR
  ☆721Updated 7 months ago
• Open-Reasoner-Zero / Open-Reasoner-Zero
  Official Repo for Open-Reasoner-Zero
  ☆1,667Updated 3 weeks ago
• PRIME-RL / PRIME
  Scalable RL solution for advanced reasoning of language models
  ☆1,419Updated last week
• facebookresearch / swe-rl
  Official codebase for "SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution"
  ☆474Updated last week
• mlfoundations / evalchemy
  Automatic evals for LLMs
  ☆340Updated last week