Alternatives and similar repositories for grokfast

Users that are interested in grokfast are comparing it to the libraries listed below

• nanowell / AdEMAMix-Optimizer-Pytorch
  The AdEMAMix Optimizer: Better, Faster, Older.
  ☆183Updated 9 months ago
• srush / annotated-mamba
  Annotated version of the Mamba paper
  ☆485Updated last year
• lucidrains / nGPT-pytorch
  Quick implementation of nGPT, learning entirely on the hypersphere, from NvidiaAI
  ☆285Updated 3 weeks ago
• microsoft / Samba
  [ICLR 2025] Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling
  ☆883Updated last month
• google-deepmind / recurrentgemma
  Open weights language model from Google DeepMind, based on Griffin.
  ☆641Updated 3 weeks ago
• facebookresearch / optimizers
  For optimization algorithm research and development.
  ☆521Updated this week
• HomebrewML / HeavyBall
  Efficient optimizers
  ☆220Updated this week
• kvfrans / jax-diffusion-transformer
  Implementation of Diffusion Transformer (DiT) in JAX
  ☆278Updated last year
• pbelcak / fastfeedforward
  A repository for log-time feedforward networks
  ☆222Updated last year
• HazyResearch / m2
  Repo for "Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture"
  ☆554Updated 5 months ago
• nikhilvyas / SOAP
  ☆190Updated 6 months ago
• CG80499 / KAN-GPT-2
  Training small GPT-2 style models using Kolmogorov-Arnold networks.
  ☆118Updated last year
• iShohei220 / adopt
  Official Implementation of "ADOPT: Modified Adam Can Converge with Any β2 with the Optimal Rate"
  ☆426Updated 6 months ago
• facebookresearch / schedule_free
  Schedule-Free Optimization in PyTorch
  ☆2,180Updated last month
• KindXiaoming / grow-crystals
  Getting crystal-like representations with harmonic loss
  ☆190Updated 2 months ago
• KellerJordan / cifar10-airbench
  CIFAR-10 speedruns: 94% in 2.6 seconds and 96% in 27 seconds
  ☆251Updated 3 months ago
• Haiyang-W / TokenFormer
  [ICLR2025 Spotlight🔥] Official Implementation of TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters
  ☆562Updated 4 months ago
• EleutherAI / sparsify
  Sparsify transformers with SAEs and transcoders
  ☆568Updated last week
• google-deepmind / nanodo
  ☆270Updated 11 months ago
• LucasPrietoAl / grokking-at-the-edge-of-numerical-stability
  ☆98Updated 5 months ago
• da-fr / arc-prize-2024
  Our solution for the arc challenge 2024
  ☆149Updated last week
• modula-systems / modula
  🧱 Modula software package
  ☆200Updated 2 months ago
• NX-AI / xlstm-jax
  Official JAX implementation of xLSTM including fast and efficient training and inference code. 7B model available at https://huggingface.…
  ☆92Updated 5 months ago
• dingo-actual / infini-transformer
  PyTorch implementation of Infini-Transformer from "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention…
  ☆289Updated last year
• PeaBrane / mamba-tiny
  Simple, minimal implementation of the Mamba SSM in one pytorch file. Using logcumsumexp (Heisen sequence).
  ☆120Updated 8 months ago
• KellerJordan / Muon
  Muon: An optimizer for hidden layers in neural networks
  ☆897Updated 2 weeks ago
• bloc97 / DeMo
  DeMo: Decoupled Momentum Optimization
  ☆188Updated 6 months ago
• apple / ml-sigma-reparam
  ☆303Updated last year
• NVIDIA / ngpt
  Normalized Transformer (nGPT)
  ☆184Updated 7 months ago
• srush / annotated-s4
  Implementation of https://srush.github.io/annotated-s4
  ☆499Updated this week