seal-rg / recurrent-pretrainingLinks
Pretraining code for a large-scale depth-recurrent language model
☆776Updated last week
Alternatives and similar repositories for recurrent-pretraining
Users that are interested in recurrent-pretraining are comparing it to the libraries listed below
Sorting:
- Dream 7B, a large diffusion language model☆737Updated this week
- Training Large Language Model to Reason in a Continuous Latent Space☆1,138Updated 4 months ago
- Recipes to scale inference-time compute of open models☆1,090Updated 2 weeks ago
- procedural reasoning datasets☆770Updated this week
- OLMoE: Open Mixture-of-Experts Language Models☆773Updated 2 months ago
- Build your own visual reasoning model☆379Updated last week
- Understanding R1-Zero-Like Training: A Critical Perspective☆973Updated 2 weeks ago
- ☆562Updated last month
- Muon: An optimizer for hidden layers in neural networks☆678Updated last week
- Verifiers for LLM Reinforcement Learning☆1,197Updated this week
- Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, spars…☆333Updated 5 months ago
- Code for BLT research paper☆1,675Updated 2 weeks ago
- LIMO: Less is More for Reasoning☆955Updated 2 months ago
- 🌾 OAT: A research-friendly framework for LLM online alignment, including reinforcement learning, preference learning, etc.☆367Updated last week
- Large Reasoning Models☆803Updated 6 months ago
- Single File, Single GPU, From Scratch, Efficient, Full Parameter Tuning library for "RL for LLMs"☆464Updated last week
- Automatic evals for LLMs☆407Updated this week
- ☆744Updated last month
- ☆1,024Updated 5 months ago
- Public repository for "The Surprising Effectiveness of Test-Time Training for Abstract Reasoning"☆311Updated 6 months ago
- Muon is Scalable for LLM Training☆1,059Updated 2 months ago
- ☆936Updated 4 months ago
- Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models☆674Updated last month
- A bibliography and survey of the papers surrounding o1☆1,194Updated 6 months ago
- System 2 Reasoning Link Collection☆835Updated 2 months ago
- SkyRL-v0: Train Real-World Long-Horizon Agents via Reinforcement Learning☆375Updated this week
- Code to train and evaluate Neural Attention Memory Models to obtain universally-applicable memory systems for transformers.☆310Updated 7 months ago
- [ICLR 2025] Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling☆879Updated last month
- MLGym A New Framework and Benchmark for Advancing AI Research Agents☆505Updated 3 weeks ago
- ReasonFlux Series - Open-Sourced Strong Reasoning LLMs☆401Updated this week