Code for "Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining"
☆29Oct 14, 2025Updated 9 months ago
Alternatives and similar repositories for openrlhf-pretrain
Users that are interested in openrlhf-pretrain are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- 6,080-param transformer achieving 100% accuracy on 10-digit addition. Trained from scratch in 10 minutes.☆22Feb 19, 2026Updated 5 months ago
- ☆15Sep 29, 2022Updated 3 years ago
- ☆75Jun 23, 2025Updated last year
- Official Repository of Paper "Watch the Weights: Unsupervised monitoring and control of fine-tuned LLMs"☆15Sep 25, 2025Updated 10 months ago
- Exploration of automated dataset selection approaches at large scales.☆55Mar 4, 2025Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆12Apr 25, 2022Updated 4 years ago
- Official Code Repository for [AutoScale📈: Scale-Aware Data Mixing for Pre-Training LLMs] Published as a conference paper at **COLM 2025*…☆14Aug 8, 2025Updated 11 months ago
- ☆26Feb 20, 2026Updated 5 months ago
- Code for Continual Learning of Control Primitives☆18Nov 11, 2020Updated 5 years ago
- A Sober Look at Language Model Reasoning☆92Nov 18, 2025Updated 8 months ago
- The Full Spectrum of Deepnet Hessians at Scale: Dynamics with SGD Training and Sample Size☆19May 19, 2019Updated 7 years ago
- A project designed to build and render a full Minecraft crafting tree.☆10Aug 10, 2021Updated 4 years ago
- ☆26Apr 10, 2025Updated last year
- Revisiting Mid-training in the Era of Reinforcement Learning Scaling☆189Jul 23, 2025Updated last year
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Code and data from the paper 'Human Feedback is not Gold Standard'☆20May 5, 2026Updated 2 months ago
- Fairer Preferences Elicit Improved Human-Aligned Large Language Model Judgments (Zhou et al., EMNLP 2024)☆14Oct 3, 2024Updated last year
- [ICML 2026 Spotlight] On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models☆164Jun 8, 2026Updated last month
- Code for "What really matters in matrix-whitening optimizers?"☆25Oct 31, 2025Updated 8 months ago
- Code to accompany the paper "The Information Geometry of Unsupervised Reinforcement Learning"☆20Oct 6, 2021Updated 4 years ago
- Distributional Successor Features Enable Zero-Shot Policy Optimization☆15Apr 11, 2025Updated last year
- Code for reproducing our paper "Low Rank Adapting Models for Sparse Autoencoder Features"☆17Mar 31, 2025Updated last year
- Computing the greatest common divisor with transformers, source code for the paper https//arxiv.org/abs/2308.15594☆14Aug 11, 2025Updated 11 months ago
- Benchmarking Optimizers for LLM Pretraining☆60May 3, 2026Updated 2 months ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- ThetaEvolve: Test-time Learning on Open Problems, enabling RL training on AlphaEvolve/OpenEvolve and emphasizing scaling test-time comput…☆172Feb 27, 2026Updated 5 months ago
- ☆28Aug 9, 2025Updated 11 months ago
- This is the official implementation for our ACL 2024 paper: "Causal Estimation of Memorisation Profiles".☆25Mar 25, 2025Updated last year
- Official Code for What Makes and Breaks Safety Fine-tuning? A Mechanistic Study (NeurIPS 2024)☆12Oct 31, 2024Updated last year
- Code for paper "Hierarchically Decoupled Imitation for Morphological Transfer"☆17Mar 24, 2023Updated 3 years ago
- Code for the paper "Learning Options via Compression" at NeurIPS 2022☆27Jan 11, 2023Updated 3 years ago
- This is the official implementation for paper "PENCIL: Long Thoughts with Short Memory".☆81May 9, 2025Updated last year
- ☆275Dec 2, 2024Updated last year
- [ICML 2026] Heima☆75May 20, 2026Updated 2 months ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Flax (JAX) implementation of Progressive Growing of GANs for Improved Quality, Stability, and Variation☆12May 24, 2021Updated 5 years ago
- Defeating the Training-Inference Mismatch via FP16☆197Nov 14, 2025Updated 8 months ago
- ☆52Feb 12, 2025Updated last year
- [TMLR] Process Reward Models That Think☆90Nov 29, 2025Updated 8 months ago
- Joint Selection for Large-Scale Pre-Training Data via Policy Gradient-based Mask Learning☆21Jan 4, 2026Updated 6 months ago
- Code for our EMNLP-2023 paper: "Active Instruction Tuning: Improving Cross-Task Generalization by Training on Prompt Sensitive Tasks"☆26Nov 16, 2023Updated 2 years ago
- LightRFT (Light Reinforcement Fine-Tuning) is an advanced reinforcement learning fine-tuning framework designed for Large Language Models…☆19Jan 12, 2026Updated 6 months ago