Alternatives and similar repositories for grokfast

Users that are interested in grokfast are comparing it to the libraries listed below

• facebookresearch / optimizers
  For optimization algorithm research and development.
  ☆513Updated this week
• srush / annotated-mamba
  Annotated version of the Mamba paper
  ☆483Updated last year
• KellerJordan / Muon
  Muon optimizer: +>30% sample efficiency with <3% wallclock overhead
  ☆623Updated last month
• lucidrains / nGPT-pytorch
  Quick implementation of nGPT, learning entirely on the hypersphere, from NvidiaAI
  ☆281Updated last month
• HomebrewML / HeavyBall
  Efficient optimizers
  ☆193Updated this week
• KindXiaoming / grow-crystals
  Getting crystal-like representations with harmonic loss
  ☆183Updated last month
• nikhilvyas / SOAP
  ☆180Updated 5 months ago
• kyegomez / Sophia
  Effortless plugin and play Optimizer to cut model training costs by 50%. New optimizer that is 2x faster than Adam on LLMs.
  ☆381Updated 11 months ago
• Haiyang-W / TokenFormer
  [ICLR2025 Spotlight🔥] Official Implementation of TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters
  ☆555Updated 3 months ago
• HazyResearch / m2
  Repo for "Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture"
  ☆550Updated 4 months ago
• pbelcak / fastfeedforward
  A repository for log-time feedforward networks
  ☆222Updated last year
• nanowell / AdEMAMix-Optimizer-Pytorch
  The AdEMAMix Optimizer: Better, Faster, Older.
  ☆183Updated 8 months ago
• LucasPrietoAl / grokking-at-the-edge-of-numerical-stability
  ☆94Updated 3 months ago
• CG80499 / KAN-GPT-2
  Training small GPT-2 style models using Kolmogorov-Arnold networks.
  ☆117Updated 11 months ago
• srush / annotated-s4
  Implementation of https://srush.github.io/annotated-s4
  ☆494Updated 2 years ago
• imbue-ai / carbs
  Cost aware hyperparameter tuning algorithm
  ☆152Updated 10 months ago
• google-deepmind / recurrentgemma
  Open weights language model from Google DeepMind, based on Griffin.
  ☆639Updated 2 months ago
• apple / ml-sigma-reparam
  ☆302Updated 10 months ago
• modula-systems / modula
  🧱 Modula software package
  ☆189Updated last month
• NVIDIA / ngpt
  Normalized Transformer (nGPT)
  ☆176Updated 5 months ago
• jerber / lang-jepa
  ☆111Updated 4 months ago
• Zyphra / BlackMamba
  Code repository for Black Mamba
  ☆246Updated last year
• google-deepmind / nanodo
  ☆217Updated 10 months ago
• PeaBrane / mamba-tiny
  Simple, minimal implementation of the Mamba SSM in one pytorch file. Using logcumsumexp (Heisen sequence).
  ☆116Updated 6 months ago
• KellerJordan / cifar10-airbench
  CIFAR-10 speedruns: 94% in 2.6 seconds and 96% in 27 seconds
  ☆233Updated 2 months ago
• facebookresearch / spdl
  Scalable and Performant Data Loading
  ☆258Updated this week
• open-thought / reasoning-gym
  procedural reasoning datasets
  ☆578Updated last week
• neoneye / ARC-Interactive-History-Dataset
  The history files when recording human interaction while solving ARC tasks
  ☆109Updated 2 weeks ago
• SakanaAI / evo-memory
  Code to train and evaluate Neural Attention Memory Models to obtain universally-applicable memory systems for transformers.
  ☆307Updated 6 months ago
• Sohl-Dickstein / fractal
  The boundary of neural network trainability is fractal
  ☆198Updated last year