[ICLR 2025] How Does Critical Batch Size Scale in Pre-training?
☆12Feb 20, 2025Updated last year
Alternatives and similar repositories for critical-batch-size
Users that are interested in critical-batch-size are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Combining SOAP and MUON☆25Feb 11, 2025Updated last year
- Code for NeurIPS 2024 Spotlight: "Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations"☆94Oct 30, 2024Updated last year
- Code to generate figures of paper "When do spectral gradient updates help in deep learning?"☆18Dec 3, 2025Updated 10 months ago
- Official code for "Algorithmic Capabilities of Random Transformers" (NeurIPS 2024)☆16Sep 28, 2024Updated 2 years ago
- Irene is a python package that aims to be a toolkit for global optimization problems that can be realized algebraically. It generalizes L…☆15Oct 13, 2026Updated last week
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- A toy eval suite for tracing generalization dynamics of LM pre-training☆27May 19, 2026Updated 5 months ago
- Spectral Sphere Optimizer☆136Mar 23, 2026Updated 6 months ago
- Code for ICLR 2022 Paper (HyperDQN: A Randomized Exploration Method for Deep Reinforcement Learning)☆12Nov 28, 2023Updated 2 years ago
- [Poster; ICLR 2026] [Oral; Neurips OPT2024] μLO: Compute-Efficient Meta-Generalization of Learned Optimizers☆16Apr 15, 2026Updated 6 months ago
- ☆14Mar 2, 2025Updated last year
- ☆70Aug 28, 2026Updated last month
- ☆10Aug 18, 2016Updated 10 years ago
- Code for the paper "Function-Space Learning Rates"☆23Jun 3, 2025Updated last year
- manipulating cointegrated pairs to achieve a market-neutral strategy that outperforms indices☆11Jan 12, 2021Updated 5 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆40Feb 26, 2024Updated 2 years ago
- ☆69Mar 21, 2025Updated last year
- Sharpness-Aware Minimization Leads to Low-Rank Features [NeurIPS 2023]☆29Sep 22, 2023Updated 3 years ago
- GoldFinch and other hybrid transformer components☆47Jul 20, 2024Updated 2 years ago
- Implementation of ZoreCoT☆10Feb 1, 2025Updated last year
- A library for unit scaling in PyTorch☆136Jul 11, 2025Updated last year
- Welcome to the 'In Context Learning Theory' Reading Group☆31Nov 8, 2024Updated last year
- Training vision models with full-batch gradient descent and regularization☆39Feb 14, 2023Updated 3 years ago
- ☆13Oct 8, 2021Updated 5 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Code for the paper: Why Transformers Need Adam: A Hessian Perspective☆66Mar 11, 2025Updated last year
- Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"☆27Apr 17, 2024Updated 2 years ago
- ☆12Dec 14, 2024Updated last year
- ☆14Sep 14, 2026Updated last month
- Mini Model Daemon☆13Nov 9, 2024Updated last year
- coloring terminal text with intensities (used for plotting probability, entropy with tokens)☆12Oct 11, 2024Updated 2 years ago
- RWKV-7 mini☆12Mar 29, 2025Updated last year
- Codebase for the paper "A Percolation Model of Emergence: Analyzing Transformers Trained on a Formal Language"☆11Oct 11, 2024Updated 2 years ago
- GoldFinch and other hybrid transformer components☆17Dec 9, 2025Updated 10 months ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- ☆34Apr 8, 2025Updated last year
- This repository contains the code to replicate the results in the paper: "GULP: a prediction-based metric between representations".☆11Sep 6, 2023Updated 3 years ago
- test images with not appropriate labels in MNIST dataset☆10Mar 3, 2018Updated 8 years ago
- Open Source Version of florr.io☆10Jan 16, 2023Updated 3 years ago
- The loss landscape of Large Language Models resemble basin!☆48Jul 8, 2025Updated last year
- Direct Preference Optimization for RWKV, aiming for RWKV-5 and 6.☆11Mar 1, 2024Updated 2 years ago
- ☆54May 20, 2024Updated 2 years ago