Minimal pretraining script for language modeling in PyTorch. Supporting torch compilation and DDP. It includes a model implementation and a data preprocessing script.
☆50Apr 27, 2026Updated 3 months ago
Alternatives and similar repositories for plainLM
Users that are interested in plainLM are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A CLI to run reproducible evaluation across EuroHPC clusters☆19Updated this week
- Code for the paper: Why Transformers Need Adam: A Hessian Perspective☆65Mar 11, 2025Updated last year
- Pretraining codebase for Apertus models, based on Megatron-LM☆21Sep 25, 2025Updated 10 months ago
- This repository is for setting-up cuda-9/8, nvidia-396/387/384 driver, OpenCV-3.3, ROS Kinetic, Tensorflow-1.11/1.7/1.4/1.2.1, Pytorch-0.…☆30Jul 7, 2022Updated 4 years ago
- Gradient-based Hyperparameter Optimization Over Long Horizons☆14Sep 29, 2021Updated 4 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Official code for "Algorithmic Capabilities of Random Transformers" (NeurIPS 2024)☆15Sep 28, 2024Updated last year
- Agar.io for Continual Reinforcement Learning☆24Jul 24, 2025Updated last year
- ☆70Apr 8, 2026Updated 4 months ago
- [ICLR 2023] "Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together!" Shiwei Liu, Tianlong Chen, Zhenyu Zhang, Xuxi Chen…☆28Aug 29, 2023Updated 2 years ago
- Lattice Recurrent Unit: Improving Convergence and Statistical Efficiency for Sequence Modeling☆36Jan 30, 2018Updated 8 years ago
- Irene is a python package that aims to be a toolkit for global optimization problems that can be realized algebraically. It generalizes L…☆15Updated this week
- Julia package interfacing the Fortran code MPBNGC.☆13Jun 28, 2025Updated last year
- ☆17Oct 25, 2022Updated 3 years ago
- Efficiently fill landscape depressions in digital elevation models☆15Sep 15, 2016Updated 9 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆11May 8, 2024Updated 2 years ago
- Code for Paper (Policy Optimization in RLHF: The Impact of Out-of-preference Data)☆29Dec 19, 2023Updated 2 years ago
- Using deep learning to predict lake bathymetry based on the surrounding terrain☆11Sep 7, 2023Updated 2 years ago
- Hessian trace estimation using PyTorch and Hutch++☆20Oct 29, 2020Updated 5 years ago
- Official implementation for the paper "Controlled Sparsity via Constrained Optimization"☆12Aug 10, 2022Updated 4 years ago
- This repository regroups learning ressources about performance estimation problems☆15Mar 18, 2026Updated 4 months ago
- Breaking the 'doomscrolling' cycle with Contre Sozial.☆17Jan 20, 2021Updated 5 years ago
- working implimention of deepseek MLA☆44Jan 8, 2025Updated last year
- [CVPR 2022] "Quarantine: Sparsity Can Uncover the Trojan Attack Trigger for Free" by Tianlong Chen*, Zhenyu Zhang*, Yihua Zhang*, Shiyu C…☆27Oct 5, 2022Updated 3 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Jaxplorer is a Jax reinforcement learning (RL) framework for exploring new ideas.☆12Jul 19, 2024Updated 2 years ago
- Randomized Linear Algebra in Python☆13Mar 21, 2017Updated 9 years ago
- A Signal Propagation Perspective for Pruning Neural Networks at Initialization☆14Jun 23, 2020Updated 6 years ago
- Code for implementing central flows☆49Sep 5, 2025Updated 11 months ago
- flexible meta-learning in jax☆16Oct 19, 2023Updated 2 years ago
- ☆30Jun 22, 2020Updated 6 years ago
- [ICML2023] Instant Soup Cheap Pruning Ensembles in A Single Pass Can Draw Lottery Tickets from Large Models. Ajay Jaiswal, Shiwei Liu, Ti…☆11Nov 28, 2023Updated 2 years ago
- JAX implementation of the Mistral 7b v0.1 model☆13Mar 27, 2024Updated 2 years ago
- MoMo: Momentum Models for Adaptive Learning Rates☆19Jun 12, 2024Updated 2 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- A set of solutions to ETHZ ROS lectures☆13Jul 19, 2017Updated 9 years ago
- ☆34Aug 4, 2026Updated last week
- MuJoCo SO-101 robot arm simulator for learning manipulation policies (ACT, Diffusion, SmolVLA)☆15Mar 27, 2026Updated 4 months ago
- The ros_qml package☆12Sep 7, 2015Updated 10 years ago
- Reimplementation of ToMNet with some extensions for RL as well☆14Apr 28, 2018Updated 8 years ago
- This repository contains code for the MicroAdam paper.☆21Dec 14, 2024Updated last year
- It is a tiny C# app, which is able to read SL3 sonar log files made by Lowrance & Simard devices and export the frame headers into a CSV …☆20May 24, 2026Updated 2 months ago