seal-rg / recurrent-pretraining
Pretraining code for a large-scale depth-recurrent language model
☆760Updated last month
Alternatives and similar repositories for recurrent-pretraining
Users that are interested in recurrent-pretraining are comparing it to the libraries listed below
Sorting:
- Training Large Language Model to Reason in a Continuous Latent Space☆1,109Updated 3 months ago
- Dream 7B, a large diffusion language model☆630Updated 2 weeks ago
- Recipes to scale inference-time compute of open models☆1,071Updated last week
- Build your own visual reasoning model☆362Updated this week
- Understanding R1-Zero-Like Training: A Critical Perspective☆925Updated last month
- Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, spars…☆327Updated 5 months ago
- 🌾 OAT: A research-friendly framework for LLM online alignment, including preference learning, reinforcement learning, etc.☆352Updated last week
- procedural reasoning datasets☆580Updated this week
- LIMO: Less is More for Reasoning☆940Updated last month
- OLMoE: Open Mixture-of-Experts Language Models☆746Updated 2 months ago
- A bibliography and survey of the papers surrounding o1☆1,192Updated 6 months ago
- Single File, Single GPU, From Scratch, Efficient, Full Parameter Tuning library for "RL for LLMs"☆450Updated this week
- Large Reasoning Models☆805Updated 5 months ago
- Public repository for "The Surprising Effectiveness of Test-Time Training for Abstract Reasoning"☆307Updated 5 months ago
- Continuous Thought Machines, because thought takes time and reasoning is a process.☆492Updated this week
- Muon optimizer: +>30% sample efficiency with <3% wallclock overhead☆623Updated last month
- TTRL: Test-Time Reinforcement Learning☆488Updated 2 weeks ago
- Verifiers for LLM Reinforcement Learning☆953Updated this week
- Official codebase for "SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution"☆517Updated 2 months ago
- Atom of Thoughts for Markov LLM Test-Time Scaling☆563Updated this week
- ☆527Updated last month
- Code for Paper: Training Software Engineering Agents and Verifiers with SWE-Gym [ICML 2025]☆455Updated last week
- Muon is Scalable for LLM Training☆1,044Updated last month
- [ICLR2025 Spotlight🔥] Official Implementation of TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters☆555Updated 3 months ago
- ☆1,019Updated 5 months ago
- Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models☆595Updated last month
- Code for BLT research paper☆1,587Updated this week
- Official PyTorch implementation for "Large Language Diffusion Models"☆1,592Updated last week
- Code for Quiet-STaR☆731Updated 8 months ago
- ☆691Updated 2 weeks ago