Alternatives and similar repositories for grokfast:

Users that are interested in grokfast are comparing it to the libraries listed below

• KellerJordan / cifar10-airbench
  CIFAR-10 speedruns: 94% in 2.6 seconds and 96% in 27 seconds
  ☆210Updated this week
• nanowell / AdEMAMix-Optimizer-Pytorch
  The AdEMAMix Optimizer: Better, Faster, Older.
  ☆178Updated 5 months ago
• srush / annotated-mamba
  Annotated version of the Mamba paper
  ☆474Updated last year
• lucidrains / nGPT-pytorch
  Quick implementation of nGPT, learning entirely on the hypersphere, from NvidiaAI
  ☆273Updated 3 months ago
• HomebrewML / HeavyBall
  Efficient optimizers
  ☆177Updated last week
• pbelcak / fastfeedforward
  A repository for log-time feedforward networks
  ☆220Updated 10 months ago
• nikhilvyas / SOAP
  ☆161Updated 3 months ago
• microsoft / Samba
  Official implementation of "Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling"
  ☆849Updated last week
• facebookresearch / optimizers
  For optimization algorithm research and development.
  ☆497Updated this week
• KellerJordan / Muon
  Muon optimizer: +>30% sample efficiency with <3% wallclock overhead
  ☆434Updated this week
• NX-AI / xlstm-jax
  Official JAX implementation of xLSTM including fast and efficient training and inference code. 7B model available at https://huggingface.…
  ☆84Updated last month
• kvfrans / jax-diffusion-transformer
  Implementation of Diffusion Transformer (DiT) in JAX
  ☆265Updated 8 months ago
• google-deepmind / recurrentgemma
  Open weights language model from Google DeepMind, based on Griffin.
  ☆622Updated last week
• EleutherAI / sparsify
  Sparsify transformers with SAEs and transcoders
  ☆476Updated this week
• google-deepmind / nanodo
  ☆212Updated 7 months ago
• NVIDIA / ngpt
  Normalized Transformer (nGPT)
  ☆156Updated 3 months ago
• ethansmith2000 / fsdp_optimizers
  supporting pytorch FSDP for optimizers
  ☆77Updated 2 months ago
• neurallambda / neurallambda
  Reasoning Computers. Lambda Calculus, Fully Differentiable. Also Neural Stacks, Queues, Arrays, Lists, Trees, and Latches.
  ☆246Updated 4 months ago
• SakanaAI / evo-memory
  Code to train and evaluate Neural Attention Memory Models to obtain universally-applicable memory systems for transformers.
  ☆294Updated 4 months ago
• jerber / lang-jepa
  ☆100Updated 2 months ago
• facebookresearch / schedule_free
  Schedule-Free Optimization in PyTorch
  ☆2,105Updated this week
• rgreenblatt / arc_draw_more_samples_pub
  Draw more samples
  ☆186Updated 8 months ago
• bloc97 / DeMo
  DeMo: Decoupled Momentum Optimization
  ☆181Updated 3 months ago
• HazyResearch / m2
  Repo for "Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture"
  ☆548Updated 2 months ago
• CG80499 / KAN-GPT-2
  Training small GPT-2 style models using Kolmogorov-Arnold networks.
  ☆114Updated 9 months ago
• Zyphra / BlackMamba
  Code repository for Black Mamba
  ☆239Updated last year
• Sohl-Dickstein / fractal
  The boundary of neural network trainability is fractal
  ☆195Updated last year
• iShohei220 / adopt
  Official Implementation of "ADOPT: Modified Adam Can Converge with Any β2 with the Optimal Rate"
  ☆417Updated 2 months ago