Minimal pretraining script for language modeling in PyTorch. Supporting torch compilation and DDP. It includes a model implementation and a data preprocessing script.
☆52Apr 27, 2026Updated 6 months ago
Alternatives and similar repositories for plainLM
Users that are interested in plainLM are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Discretax is a light weight collection of state space models implemented in JAX ⚡️☆38Mar 9, 2026Updated 8 months ago
- ☆14May 5, 2025Updated last year
- Code implementation of DuoGPT (NeurIPS 2025, https://arxiv.org/abs/2506.20194)☆10Sep 23, 2025Updated last year
- Real-Time RTUs☆13Mar 20, 2026Updated 8 months ago
- Merge of megatron-train, autoexperiment and oellm_pretrain.☆19Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆53Jun 10, 2024Updated 2 years ago
- Code for implementing central flows☆58Sep 5, 2025Updated last year
- Pretraining codebase for Apertus models, based on Megatron-LM☆24Nov 4, 2026Updated 2 weeks ago
- Agar.io for Continual Reinforcement Learning☆28Aug 27, 2026Updated 2 months ago
- ☆12Feb 6, 2025Updated last year
- ☆11Mar 29, 2026Updated 7 months ago
- Code for ICLR 2022 Paper (HyperDQN: A Randomized Exploration Method for Deep Reinforcement Learning)☆12Nov 28, 2023Updated 2 years ago
- ☆11Mar 18, 2025Updated last year
- ☆11Jan 15, 2026Updated 10 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆17Oct 25, 2022Updated 4 years ago
- Code for "The Surprising Agreement Between Convex Optimization Theory and Learning-Rate Scheduling for Large Model Training"☆11Feb 4, 2026Updated 9 months ago
- Code for Paper (Policy Optimization in RLHF: The Impact of Out-of-preference Data)☆30Dec 19, 2023Updated 2 years ago
- KFAC from scratch (KFS)---Paper & Code☆13Jan 13, 2026Updated 10 months ago
- ☆18Jun 16, 2026Updated 5 months ago
- Official implementation for the paper "Controlled Sparsity via Constrained Optimization"☆12Aug 10, 2022Updated 4 years ago
- ☆16Jan 16, 2023Updated 3 years ago
- [TMLR] "Can You Win Everything with Lottery Ticket?" by Tianlong Chen, Zhenyu Zhang, Jun Wu, Randy Huang, Sijia Liu, Shiyu Chang, Zhangy…☆10Sep 8, 2022Updated 4 years ago
- [CVPR 2022] "Quarantine: Sparsity Can Uncover the Trojan Attack Trigger for Free" by Tianlong Chen*, Zhenyu Zhang*, Yihua Zhang*, Shiyu C…☆28Oct 5, 2022Updated 4 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Implementation of the psquare algorithm for quantile value estimation☆11Apr 21, 2024Updated 2 years ago
- A simple, line-oriented tabular data format☆10Jan 20, 2018Updated 8 years ago
- PyTorch implementation of the paper The Lottery Ticket Hypothesis for Object Recognition☆22Apr 22, 2021Updated 5 years ago
- This repository demonstrates the application of our proposed task-free continual learning method on a synthetic experiment.☆13Jun 24, 2019Updated 7 years ago
- ☆37Oct 4, 2026Updated last month
- A programming project on automatic differentiation in OCaml☆11Dec 22, 2022Updated 3 years ago
- Code release for Normalizing Flow Maps☆21Oct 19, 2026Updated last month
- Transductive Active Learning with Application to Safe Bayesian Optimization☆11Mar 5, 2025Updated last year
- Code for NeurIPS 2024 Spotlight: "Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations"☆94Oct 30, 2024Updated 2 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- MoMo: Momentum Models for Adaptive Learning Rates☆19Jun 12, 2024Updated 2 years ago
- Pytorch2Jax is a small Python library that provides functions that wraps PyTorch models into Jax functions and Flax modules.☆21Feb 20, 2023Updated 3 years ago
- ☆19Jun 21, 2021Updated 5 years ago
- Code to create bugged python scripts for OpenAssistant Training, maintained by https://twitter.com/Cyndesama☆24Jul 23, 2023Updated 3 years ago
- Parameter-Free Optimizers for Pytorch☆132Apr 23, 2024Updated 2 years ago
- Source code for my PhD thesis: Backpropagation Beyond the Gradient☆21Feb 25, 2023Updated 3 years ago
- Implementation of the models and datasets used in "An Information-theoretic Approach to Distribution Shifts"☆26Nov 2, 2021Updated 5 years ago
Are these results useful?