Alternatives and similar repositories for AdaFisher

Users that are interested in AdaFisher are comparing it to the libraries listed below

• kozistr / pytorch_optimizer
  optimizer & lr scheduler & loss function collections in PyTorch
  ☆388Updated this week
• nikhilvyas / SOAP
  ☆246Updated last year
• MarlonBecker / MSAM
  ☆21Updated 2 years ago
• kyegomez / Griffin
  Implementation of Griffin from the paper: "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"
  ☆56Updated 3 months ago
• opooladz / Preconditioned-Stochastic-Gradient-Descent
  A repo based on XiLin Li's PSGD repo that extends some of the experiments.
  ☆14Updated last year
• nanowell / AdEMAMix-Optimizer-Pytorch
  The AdEMAMix Optimizer: Better, Faster, Older.
  ☆186Updated last year
• lucidrains / minGRU-pytorch
  Implementation of the proposed minGRU in Pytorch
  ☆319Updated last month
• apapiu / mamba_small_bench
  Trying out the Mamba architecture on small examples (cifar-10, shakespeare char level etc.)
  ☆47Updated 2 years ago
• lucidrains / adam-atan2-pytorch
  Implementation of the proposed Adam-atan2 from Google Deepmind in Pytorch
  ☆135Updated 3 months ago
• AmeenAli / HiddenMambaAttn
  Official PyTorch Implementation of "The Hidden Attention of Mamba Models"
  ☆231Updated 3 months ago
• Adamdad / rational_kat_cu
  ☆79Updated last year
• LIONS-EPFL / scion
  ☆52Updated last month
• AngusDujw / SAF
  ☆35Updated 3 years ago
• nblt / F-SAM
  [CVPR 2024] Friendly Sharpness-Aware Minimization
  ☆35Updated last year
• FarnoushRJ / MambaLRP
  [NeurIPS 2024] Official implementation of the paper "MambaLRP: Explaining Selective State Space Sequence Models" 🐍
  ☆45Updated last year
• lucidrains / hyper-connections
  Attempt to make multiple residual streams from Bytedance's Hyper-Connections paper accessible to the public
  ☆168Updated 2 weeks ago
• zqOuO / GWT
  ☆13Updated last year
• TianjinYellow / SPAM-Optimizer
  ☆36Updated 10 months ago
• lucidrains / soft-moe-pytorch
  Implementation of Soft MoE, proposed by Brain's Vision team, in Pytorch
  ☆344Updated 10 months ago
• kyleliang919 / C-Optim
  When it comes to optimizers, it's always better to be safe than sorry
  ☆402Updated 4 months ago
• lucidrains / gateloop-transformer
  Implementation of GateLoop Transformer in Pytorch and Jax
  ☆92Updated last year
• fkodom / soft-mixture-of-experts
  PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)
  ☆82Updated 2 years ago
• jacobfa / fft
  ☆129Updated 6 months ago
• KellerJordan / cifar10-airbench
  CIFAR-10 speedruns: 94% in 2.6 seconds and 96% in 27 seconds
  ☆352Updated 2 months ago
• myscience / x-lstm
  Pytorch implementation of the xLSTM model by Beck et al. (2024)
  ☆181Updated last year
• PeaBrane / mamba-tiny
  Simple, minimal implementation of the Mamba SSM in one pytorch file. Using logcumsumexp (Heisen sequence).
  ☆130Updated last year
• kyegomez / MambaTransformer
  Integrating Mamba/SSMs with Transformer for Enhanced Long Context and High-Quality Sequence Modeling
  ☆213Updated last week
• i404788 / s5-pytorch
  Pytorch implementation of Simplified Structured State-Spaces for Sequence Modeling (S5)
  ☆82Updated last year
• lucidrains / gradnorm-pytorch
  A practical implementation of GradNorm, Gradient Normalization for Adaptive Loss Balancing, in Pytorch
  ☆126Updated 5 months ago
• HomebrewML / HeavyBall
  Efficient optimizers
  ☆281Updated last month