Minimal pretraining script for language modeling in PyTorch. Supporting torch compilation and DDP. It includes a model implementation and a data preprocessing script.
☆48Apr 27, 2026Updated last month
Alternatives and similar repositories for plainLM
Users that are interested in plainLM are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Real-Time RTUs☆12Mar 20, 2026Updated 2 months ago
- Linear Attention for Efficient Bidirectional Sequence Modeling☆16May 13, 2025Updated last year
- Gradient-based Hyperparameter Optimization Over Long Horizons☆14Sep 29, 2021Updated 4 years ago
- Hessian trace estimation using PyTorch and Hutch++☆20Oct 29, 2020Updated 5 years ago
- Official repository for the paper "Exploring the Promise and Limits of Real-Time Recurrent Learning" (ICLR 2024)☆13Jun 11, 2025Updated 11 months ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Official implementation for the paper "Controlled Sparsity via Constrained Optimization"☆12Aug 10, 2022Updated 3 years ago
- Code for the paper: Why Transformers Need Adam: A Hessian Perspective☆65Mar 11, 2025Updated last year
- ☆194Updated this week
- Optimization methods course at MIPT☆11Apr 24, 2018Updated 8 years ago
- Optimize nonsmooth functions with gradient sampling, (ns) BFGS...☆10Feb 11, 2025Updated last year
- Breaking the 'doomscrolling' cycle with Contre Sozial.☆17Jan 20, 2021Updated 5 years ago
- Julia package interfacing the Fortran code MPBNGC.☆13Jun 28, 2025Updated 11 months ago
- A set of solutions to ETHZ ROS lectures☆13Jul 19, 2017Updated 8 years ago
- An implementation of LassoNet for arbitrary network architectures☆14Nov 4, 2022Updated 3 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Convert CVXPY expressions to PyTorch expressions☆18Jul 8, 2025Updated 11 months ago
- [ICML 2024] Sparse Model Inversion: Efficient Inversion of Vision Transformers with Less Hallucination☆14Apr 29, 2025Updated last year
- Free subscriptions for students.☆15Mar 4, 2020Updated 6 years ago
- Spatial reconstruction of dissociated single-cell data☆20Jan 12, 2024Updated 2 years ago
- ☆28Mar 29, 2025Updated last year
- [CVPR 2025 Highlight] FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximation☆28Jun 16, 2025Updated 11 months ago
- SMART introduces a novel test-time framework where Small Language Models (SLMs) reason step-by-step, and Large Language Models (LLMs) pro…☆12Jul 9, 2025Updated 10 months ago
- [CVPR 2025] QuartDepth☆18Mar 24, 2025Updated last year
- Distributed pretraining of large language models (LLMs) on cloud TPU slices, with Jax and Equinox.☆25Sep 29, 2024Updated last year
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆15Apr 6, 2026Updated 2 months ago
- Finetune Google's pre-trained ViT models from HuggingFace's model hub.☆19Apr 4, 2021Updated 5 years ago
- Source code of our TNNLS paper "Boosting Convolutional Neural Networks with Middle Spectrum Grouped Convolution"☆12Apr 14, 2023Updated 3 years ago
- ☆13May 11, 2026Updated 3 weeks ago
- A PyTorch Implementation of Neural Turing Machine☆14Jul 24, 2020Updated 5 years ago
- ☆14Jul 14, 2025Updated 10 months ago
- [ICML 2025] Official PyTorch implementation of "NegMerge: Sign-Consensual Weight Merging for Machine Unlearning"☆16Nov 25, 2025Updated 6 months ago
- ☆23Feb 24, 2022Updated 4 years ago
- Implementation of the Prioritized Option-Critic on the Four-Rooms Environment☆17Dec 24, 2017Updated 8 years ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- ☆15Apr 11, 2024Updated 2 years ago
- A large database of artificial neural network statistics during training☆15Dec 8, 2020Updated 5 years ago
- Benchmarking Optimizers for LLM Pretraining☆60May 3, 2026Updated last month
- Source code for my PhD thesis: Backpropagation Beyond the Gradient☆21Feb 25, 2023Updated 3 years ago
- Mastodoner is a command line tool (and Python library) for archiving Mastodon, a decentralized micro-blogging social network.☆14Oct 21, 2024Updated last year
- A manufacture dashboard demo app showing real-time process control.☆26May 15, 2019Updated 7 years ago
- [CVPR 2026] MergeVLA: Cross-Skill Model Merging Toward a Generalist Vision-Language-Action Agent☆32Apr 30, 2026Updated last month