Alternatives and similar repositories for recurrent-pretraining

Users that are interested in recurrent-pretraining are comparing it to the libraries listed below

• facebookresearch / coconut

  View on GitHub
  Training Large Language Model to Reason in a Continuous Latent Space
  ☆1,503Aug 12, 2025Updated 6 months ago
• facebookresearch / large_concept_model

  View on GitHub
  Large Concept Models: Language modeling in a sentence representation space
  ☆2,333Jan 29, 2025Updated last year
• OSU-NLP-Group / GrokkedTransformer

  View on GitHub
  Code for NeurIPS'24 paper 'Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization'
  ☆235Jul 19, 2025Updated 6 months ago
• facebookresearch / blt

  View on GitHub
  Code for BLT research paper
  ☆2,028Nov 3, 2025Updated 3 months ago
• simplescaling / s1

  View on GitHub
  s1: Simple test-time scaling
  ☆6,631Jun 25, 2025Updated 7 months ago
• rllm-org / rllm

  View on GitHub
  Democratizing Reinforcement Learning for LLMs
  ☆5,106Updated this week
• SakanaAI / self-adaptive-llms

  View on GitHub
  A Self-adaptation Framework🐙 that adapts LLMs for unseen tasks in real-time!
  ☆1,185Jan 30, 2025Updated last year
• Jiayi-Pan / TinyZero

  View on GitHub
  Minimal reproduction of DeepSeek R1-Zero
  ☆12,748Apr 24, 2025Updated 9 months ago
• Open-Reasoner-Zero / Open-Reasoner-Zero

  View on GitHub
  Official Repo for Open-Reasoner-Zero
  ☆2,085Jun 2, 2025Updated 8 months ago
• hkust-nlp / simpleRL-reason

  View on GitHub
  Simple RL training for reasoning
  ☆3,827Dec 23, 2025Updated last month
• ezelikman / quiet-star

  View on GitHub
  Code for Quiet-STaR
  ☆740Aug 21, 2024Updated last year
• GAIR-NLP / LIMO

  View on GitHub
  [COLM 2025] LIMO: Less is More for Reasoning
  ☆1,063Jul 30, 2025Updated 6 months ago
• ML-GSAI / LLaDA

  View on GitHub
  Official PyTorch implementation for "Large Language Diffusion Models"
  ☆3,569Nov 12, 2025Updated 3 months ago
• facebookresearch / memory

  View on GitHub
  Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, spars…
  ☆371Dec 12, 2024Updated last year
• eddycmu / demystify-long-cot

  View on GitHub
  ☆331May 31, 2025Updated 8 months ago
• facebookresearch / RAM

  View on GitHub
  A framework to study AI models in Reasoning, Alignment, and use of Memory (RAM).
  ☆344Dec 16, 2025Updated last month
• mcleish7 / arithmetic

  View on GitHub
  Code to reproduce "Transformers Can Do Arithmetic with the Right Embeddings", McLeish et al (NeurIPS 2024)
  ☆198May 28, 2024Updated last year
• MoonshotAI / Moonlight

  View on GitHub
  Muon is Scalable for LLM Training
  ☆1,432Aug 3, 2025Updated 6 months ago
• Gen-Verse / ReasonFlux

  View on GitHub
  [NeurIPS 2025 Spotlight] ReasonFlux (long-CoT), ReasonFlux-PRM (process reward model) and ReasonFlux-Coder (code generation)
  ☆520Sep 27, 2025Updated 4 months ago
• kazuki-irie / kv-memory-brain

  View on GitHub
  Official Code Repository for the paper "Key-value memory in the brain"
  ☆31Feb 25, 2025Updated 11 months ago
• mcleish7 / gemstone-scaling-laws

  View on GitHub
  Gemstones: A Model Suite for Multi-Faceted Scaling Laws (NeurIPS 2025)
  ☆33Sep 28, 2025Updated 4 months ago
• kuleshov-group / bd3lms

  View on GitHub
  [ICLR 2025 Oral] Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models
  ☆956Jul 10, 2025Updated 7 months ago
• NovaSky-AI / SkyThought

  View on GitHub
  Sky-T1: Train your own O1 preview model within $450
  ☆3,370Jul 12, 2025Updated 7 months ago
• huggingface / nanotron

  View on GitHub
  Minimalistic large language model 3D-parallelism training
  ☆2,544Dec 11, 2025Updated 2 months ago
• sail-sg / understand-r1-zero

  View on GitHub
  Understanding R1-Zero-Like Training: A Critical Perspective
  ☆1,209Aug 27, 2025Updated 5 months ago
• fla-org / flash-linear-attention

  View on GitHub
  🚀 Efficient implementations of state-of-the-art linear attention models
  ☆4,379Updated this week
• NVlabs / GatedDeltaNet

  View on GitHub
  [ICLR 2025] Official PyTorch Implementation of Gated Delta Networks: Improving Mamba2 with Delta Rule
  ☆452Sep 15, 2025Updated 5 months ago
• allenai / open-instruct

  View on GitHub
  AllenAI's post-training codebase
  ☆3,573Updated this week
• StarsfieldAI / R1-V

  View on GitHub
  Witness the aha moment of VLM with less than $3.
  ☆4,032May 19, 2025Updated 8 months ago
• open-thoughts / open-thoughts

  View on GitHub
  Fully open data curation for reasoning models
  ☆2,207Dec 2, 2025Updated 2 months ago
• huggingface / open-r1

  View on GitHub
  Fully open reproduction of DeepSeek-R1
  ☆25,879Nov 24, 2025Updated 2 months ago
• hkust-nlp / CodeIO

  View on GitHub
  [ICML 2025 Oral] CodeI/O: Condensing Reasoning Patterns via Code Input-Output Prediction
  ☆567May 6, 2025Updated 9 months ago
• arcee-ai / mergekit

  View on GitHub
  Tools for merging pretrained large language models.
  ☆6,783Jan 26, 2026Updated 2 weeks ago
• qixucen / atom

  View on GitHub
  [NeurIPS 2025] Atom of Thoughts for Markov LLM Test-Time Scaling
  ☆640Feb 7, 2026Updated last week
• JonasGeiping / linear_cross_entropy_loss

  View on GitHub
  A fusion of a linear layer and a cross entropy loss, written for pytorch in triton.
  ☆75Aug 2, 2024Updated last year
• facebookresearch / sweet_rl

  View on GitHub
  Benchmark and research code for the paper SWEET-RL Training Multi-Turn LLM Agents onCollaborative Reasoning Tasks
  ☆262May 5, 2025Updated 9 months ago
• lucidrains / nGPT-pytorch

  View on GitHub
  Quick implementation of nGPT, learning entirely on the hypersphere, from NvidiaAI
  ☆293Jun 3, 2025Updated 8 months ago
• BlinkDL / LinearAttentionArena

  View on GitHub
  Here we will test various linear attention designs.
  ☆62Apr 25, 2024Updated last year
• lucidrains / titans-pytorch

  View on GitHub
  Unofficial implementation of Titans, SOTA memory for transformers, in Pytorch
  ☆1,935Updated this week