Alternatives and similar repositories for schedules-and-scaling

Users that are interested in schedules-and-scaling are comparing it to the libraries listed below

• sail-sg / scaling-with-vocab

  View on GitHub
  [NeurIPS-2024] 📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623
  ☆89Sep 26, 2024Updated last year
• iwiwi / epochraft

  View on GitHub
  Checkpointable dataset utilities for foundation model training
  ☆32Jan 29, 2024Updated 2 years ago
• shikaiqiu / compute-better-spent

  View on GitHub
  ☆63Oct 3, 2024Updated last year
• tilde-research / nsa-impl

  View on GitHub
  An efficient implementation of the NSA (Native Sparse Attention) kernel
  ☆129Jun 24, 2025Updated 7 months ago
• feifeibear / ChituAttention

  View on GitHub
  Quantized Attention on GPU
  ☆44Nov 22, 2024Updated last year
• graphcore-research / jax-scalify

  View on GitHub
  JAX Scalify: end-to-end scaled arithmetics
  ☆18Oct 30, 2024Updated last year
• sail-sg / SkyLadder

  View on GitHub
  The official repository for SkyLadder: Better and Faster Pretraining via Context Window Scheduling
  ☆42Dec 29, 2025Updated last month
• formll / resolving-scaling-law-discrepancies

  View on GitHub
  ☆20Nov 4, 2025Updated 3 months ago
• epfml / llm-baselines

  View on GitHub
  nanoGPT-like codebase for LLM training
  ☆113Nov 7, 2025Updated 3 months ago
• cloneofsimo / repa-rf

  View on GitHub
  ☆33Nov 4, 2024Updated last year
• dayal-kalra / low-memory-adam

  View on GitHub
  ☆14Mar 2, 2025Updated 11 months ago
• shawntan / stickbreaking-attention

  View on GitHub
  Stick-breaking attention
  ☆62Jul 1, 2025Updated 7 months ago
• feifeibear / DPSKV3MFU

  View on GitHub
  Estimate MFU for DeepSeekV3
  ☆26Jan 5, 2025Updated last year
• srush / tangent

  View on GitHub
  Source-to-Source Debuggable Derivatives in Pure Python
  ☆15Jan 23, 2024Updated 2 years ago
• apple / ml-cross-entropy

  View on GitHub
  ☆579Sep 23, 2025Updated 4 months ago
• GindaChen / FlexFlashAttention3

  View on GitHub
  FlexAttention w/ FlashAttention3 Support
  ☆27Oct 5, 2024Updated last year
• hkust-nlp / llm-compression-intelligence

  View on GitHub
  Official github repo for the paper "Compression Represents Intelligence Linearly" [COLM 2024]
  ☆147Sep 20, 2024Updated last year
• upiterbarg / lintseq

  View on GitHub
  [ICLR 2025] "Training LMs on Synthetic Edit Sequences Improves Code Synthesis" (Piterbarg, Pinto, Fergus)
  ☆19Feb 11, 2025Updated last year
• layer6ai-labs / calo-forest

  View on GitHub
  A scalable implementation of diffusion and flow-matching with XGBoost models, applied to calorimeter data.
  ☆19Nov 3, 2024Updated last year
• cloneofsimo / zeroshampoo

  View on GitHub
  ☆34Sep 10, 2024Updated last year
• PiotrNawrot / nano-sparse-attention

  View on GitHub
  The simplest implementation of recent Sparse Attention patterns for efficient LLM inference.
  ☆92Jul 17, 2025Updated 6 months ago
• TRI-ML / linear_open_lm

  View on GitHub
  A repository for research on medium sized language models.
  ☆77May 23, 2024Updated last year
• sail-sg / ScaleLong

  View on GitHub
  The official repository of paper "ScaleLong: Towards More Stable Training of Diffusion Model via Scaling Network Long Skip Connection" (N…
  ☆50Oct 23, 2023Updated 2 years ago
• tml-epfl / long-is-more-for-alignment

  View on GitHub
  Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning [ICML 2024]
  ☆21May 2, 2024Updated last year
• yikangshen / megablocks

  View on GitHub
  ☆20May 30, 2024Updated last year
• tascj / kaggle-feedback-prize-2021

  View on GitHub
  Solution of Kaggle competition: Feedback Prize - Evaluating Student Writing
  ☆16Mar 30, 2022Updated 3 years ago
• nikhilvyas / SOAP

  View on GitHub
  ☆250Dec 2, 2024Updated last year
• philippe-eecs / vitok

  View on GitHub
  ☆34May 14, 2025Updated 9 months ago
• renll / SeqBoat

  View on GitHub
  [NeurIPS 2023] Sparse Modular Activation for Efficient Sequence Modeling
  ☆40Dec 2, 2023Updated 2 years ago
• thinking-machines-lab / manifolds

  View on GitHub
  Supporting code for the blog post on modular manifolds.
  ☆115Sep 26, 2025Updated 4 months ago
• berlino / seq_icl

  View on GitHub
  ☆53May 20, 2024Updated last year
• IST-DASLab / MicroAdam

  View on GitHub
  This repository contains code for the MicroAdam paper.
  ☆22Dec 14, 2024Updated last year
• ethansmith2000 / TransformerExperiments

  View on GitHub
  ☆19Dec 4, 2025Updated 2 months ago
• mlfoundations / scaling

  View on GitHub
  Language models scale reliably with over-training and on downstream tasks
  ☆99Apr 2, 2024Updated last year
• OpenSparseLLMs / Linear-MoE

  View on GitHub
  ☆129Jun 6, 2025Updated 8 months ago
• ggjy / vision_weak_to_strong

  View on GitHub
  ☆38Feb 8, 2024Updated 2 years ago
• kyleliang919 / Super_Muon

  View on GitHub
  ☆67Mar 21, 2025Updated 10 months ago
• zyushun / Adam-mini

  View on GitHub
  Code for Adam-mini: Use Fewer Learning Rates To Gain More https://arxiv.org/abs/2406.16793
  ☆452May 13, 2025Updated 9 months ago
• KempnerInstitute / llm_uncertainty

  View on GitHub
  Code for the paper "Distinguishing the Knowable from the Unknowable with Language Models"
  ☆11Apr 15, 2024Updated last year