epfml / llm-optimizer-benchmarkLinks
Benchmarking Optimizers for LLM Pretraining
☆49Updated last month
Alternatives and similar repositories for llm-optimizer-benchmark
Users that are interested in llm-optimizer-benchmark are comparing it to the libraries listed below
Sorting:
- Code for NeurIPS 2024 Spotlight: "Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations"☆88Updated last year
- Using FlexAttention to compute attention with different masking patterns☆47Updated last year
- The official repository for SkyLadder: Better and Faster Pretraining via Context Window Scheduling☆42Updated last month
- Long Context Extension and Generalization in LLMs☆62Updated last year
- Replicating O1 inference-time scaling laws☆92Updated last year
- ☆33Updated last year
- Defeating the Training-Inference Mismatch via FP16☆181Updated 2 months ago
- Simple and efficient pytorch-native transformer training and inference (batched)☆79Updated last year
- FROM $f(x)$ AND $g(x)$ TO $f(g(x))$: LLMs Learn New Skills in RL by Composing Old Ones☆57Updated last week
- Code and Configs for Asynchronous RLHF: Faster and More Efficient RL for Language Models☆68Updated 9 months ago
- The code for creating the iGSM datasets in papers "Physics of Language Models Part 2.1, Grade-School Math and the Hidden Reasoning Proces…☆84Updated last year
- Gemstones: A Model Suite for Multi-Faceted Scaling Laws (NeurIPS 2025)☆32Updated 4 months ago
- The repository contains code for Adaptive Data Optimization☆32Updated last year
- Language models scale reliably with over-training and on downstream tasks☆99Updated last year
- The simplest implementation of recent Sparse Attention patterns for efficient LLM inference.☆92Updated 6 months ago
- ☆53Updated last year
- Kinetics: Rethinking Test-Time Scaling Laws☆86Updated 6 months ago
- Physics of Language Models: Part 4.2, Canon Layers at Scale where Synthetic Pretraining Resonates in Reality☆316Updated last month
- [ICML 2025] Predictive Data Selection: The Data That Predicts Is the Data That Teaches☆60Updated 11 months ago
- PostTrainBench measures how well CLI agents like Claude Code or Codex CLI can post-train base LLMs on a single H100 GPU in 10 hours☆131Updated this week
- Reinforcing General Reasoning without Verifiers☆96Updated 7 months ago
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models" [AISTATS …☆60Updated last year
- Stick-breaking attention☆62Updated 7 months ago
- ☆91Updated last year
- ☆112Updated last year
- ☆108Updated last year
- ☆20Updated 3 months ago
- Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Fl…☆78Updated last year
- Code for ICLR 2025 Paper "What is Wrong with Perplexity for Long-context Language Modeling?"☆109Updated 3 months ago
- Organize the Web: Constructing Domains Enhances Pre-Training Data Curation☆76Updated 9 months ago