HomebrewML/HeavyBall external links

---

GitHub - View repository

GitHub.dev - Open in online code editor

Star history - Growth chart over time

deps.dev - Dependencies and security vulnerabilities

Gitingest - Export source code for LLM usage

Repomix - Export source code for LLM usage

uithub - Export source code for LLM usage

Close

HomebrewML/HeavyBall

Readme badge preview -

If you own this repo, copy the snippet below and add it to your README.md

---

Copy

[![RelatedRepos](https://img.shields.io/badge/related-repos-yellow)](https://relatedrepos.com/gh/HomebrewML/HeavyBall)

Close

HomebrewML / HeavyBallView on GitHubMore

• External links
• Readme badge

Efficient optimizers

☆294Mar 16, 2026Updated this week

Alternatives and similar repositories for HeavyBall

Users that are interested in HeavyBall are comparing it to the libraries listed below

• ethansmith2000 / fsdp_optimizers

  View on GitHub
  supporting pytorch FSDP for optimizers
  ☆84Dec 8, 2024Updated last year
• nikhilvyas / SOAP

  View on GitHub
  ☆255Dec 2, 2024Updated last year
• evanatyourservice / kron_torch

  View on GitHub
  An implementation of PSGD Kron second-order optimizer for PyTorch
  ☆99Jul 24, 2025Updated 7 months ago
• fal-ai / diffusion-speedrun

  View on GitHub
  Focused on fast experimentation and simplicity
  ☆80Dec 24, 2024Updated last year
• evanatyourservice / llm-jax

  View on GitHub
  Train a SmolLM-style llm on fineweb-edu in JAX/Flax with an assortment of optimizers.
  ☆19Jul 24, 2025Updated 7 months ago
• lixilinx / psgd_torch

  View on GitHub
  Pytorch implementation of preconditioned stochastic gradient descent (Kron and affine preconditioner, low-rank approximation precondition…
  ☆192Jan 11, 2026Updated 2 months ago
• facebookresearch / schedule_free

  View on GitHub
  Schedule-Free Optimization in PyTorch
  ☆2,265May 21, 2025Updated 10 months ago
• KellerJordan / Muon

  View on GitHub
  Muon is an optimizer for hidden layers in neural networks
  ☆2,398Jan 19, 2026Updated 2 months ago
• facebookresearch / optimizers

  View on GitHub
  For optimization algorithm research and development.
  ☆561Mar 3, 2026Updated 2 weeks ago
• cloneofsimo / repa-rf

  View on GitHub
  ☆33Nov 4, 2024Updated last year
• BlinkDL / SmallInitEmb

  View on GitHub
  LayerNorm(SmallInit(Embedding)) in a Transformer to improve convergence
  ☆61Feb 21, 2022Updated 4 years ago
• ClashLuke / SOAP

  View on GitHub
  ☆22Nov 9, 2024Updated last year
• cloneofsimo / zeroshampoo

  View on GitHub
  ☆34Sep 10, 2024Updated last year
• riverstone496 / awesome-second-order-optimization

  View on GitHub
  ☆29Sep 30, 2025Updated 5 months ago
• nikhilvyas / SOAP_MUON

  View on GitHub
  Combining SOAP and MUON
  ☆19Feb 11, 2025Updated last year
• modula-systems / modula

  View on GitHub
  🧱 Modula software package
  ☆324Aug 18, 2025Updated 7 months ago
• cloneofsimo / scaling-guide

  View on GitHub
  WIP
  ☆94Aug 13, 2024Updated last year
• proger / accelerated-scan

  View on GitHub
  Accelerated First Order Parallel Associative Scan
  ☆196Jan 7, 2026Updated 2 months ago
• lindermanlab / elk

  View on GitHub
  Scalable and Stable Parallelization of Nonlinear RNNS
  ☆29Mar 6, 2026Updated 2 weeks ago
• LIONS-EPFL / scion

  View on GitHub
  ☆56Feb 24, 2026Updated 3 weeks ago
• cloneofsimo / ezmup

  View on GitHub
  Simple implementation of muP, based on Spectral Condition for Feature Learning. The implementation is SGD only, dont use it for Adam
  ☆86Jul 28, 2024Updated last year
• apple / ml-cross-entropy

  View on GitHub
  ☆591Sep 23, 2025Updated 5 months ago
• kyleliang919 / C-Optim

  View on GitHub
  [ICLR 2026] When it comes to optimizers, it's always better to be safe than sorry
  ☆408Sep 26, 2025Updated 5 months ago
• graphcore-research / unit-scaling

  View on GitHub
  A library for unit scaling in PyTorch
  ☆133Jul 11, 2025Updated 8 months ago
• microsoft / dion

  View on GitHub
  Dion optimizer algorithm
  ☆456Jan 16, 2026Updated 2 months ago
• ClashLuke / tpucare

  View on GitHub
  Automatically take good care of your preemptible TPUs
  ☆37May 15, 2023Updated 2 years ago
• xjdr-alt / mla_blog_translation

  View on GitHub
  ☆13Jun 18, 2024Updated last year
• proger / nanokitchen

  View on GitHub
  Parallel Associative Scan for Language Models
  ☆18Jan 8, 2024Updated 2 years ago
• Sike-Wang / low-bit-Shampoo

  View on GitHub
  4-bit Shampoo for Memory-Efficient Network Training (NeurIPS 2024)
  ☆13Feb 13, 2025Updated last year
• GindaChen / FlexFlashAttention3

  View on GitHub
  FlexAttention w/ FlashAttention3 Support
  ☆27Oct 5, 2024Updated last year
• NVIDIA-NeMo / Emerging-Optimizers

  View on GitHub
  ☆164Updated this week
• euclaise / supertrainer2000

  View on GitHub
  ☆50Mar 14, 2024Updated 2 years ago
• warner-benjamin / optimi

  View on GitHub
  Fast, Modern, and Low Precision PyTorch Optimizers
  ☆128Dec 29, 2025Updated 2 months ago
• Knowledgator / FlashDeBERTa

  View on GitHub
  Trully flash implementation of DeBERTa disentangled attention mechanism.
  ☆81Feb 10, 2026Updated last month
• opooladz / Preconditioned-Stochastic-Gradient-Descent

  View on GitHub
  A repo based on XiLin Li's PSGD repo that extends some of the experiments.
  ☆14Oct 7, 2024Updated last year
• theAdamColton / ijepa-enhanced

  View on GitHub
  recipe for training fully-featured self supervised image jepa models
  ☆12Jun 4, 2025Updated 9 months ago
• KellerJordan / modded-nanogpt

  View on GitHub
  NanoGPT (124M) in 2 minutes
  ☆4,848Updated this week
• KellerJordan / cifar10-airbench

  View on GitHub
  CIFAR-10 speedruns: 94% in 2.6 seconds and 96% in 27 seconds
  ☆365Nov 15, 2025Updated 4 months ago
• fal-ai-community / NativeSparseAttention

  View on GitHub
  research impl of Native Sparse Attention (2502.11089)
  ☆63Feb 19, 2025Updated last year