Alternatives and similar repositories for AdaFisher

Users that are interested in AdaFisher are comparing it to the libraries listed below

• apapiu / mamba_small_bench
  Trying out the Mamba architecture on small examples (cifar-10, shakespeare char level etc.)
  ☆47Updated last year
• nikhilvyas / SOAP
  ☆223Updated 11 months ago
• lucidrains / adam-atan2-pytorch
  Implementation of the proposed Adam-atan2 from Google Deepmind in Pytorch
  ☆134Updated last month
• kozistr / pytorch_optimizer
  optimizer & lr scheduler & loss function collections in PyTorch
  ☆370Updated 3 weeks ago
• LIONS-EPFL / scion
  ☆47Updated last month
• opooladz / Preconditioned-Stochastic-Gradient-Descent
  A repo based on XiLin Li's PSGD repo that extends some of the experiments.
  ☆14Updated last year
• nanowell / AdEMAMix-Optimizer-Pytorch
  The AdEMAMix Optimizer: Better, Faster, Older.
  ☆186Updated last year
• kyegomez / Griffin
  Implementation of Griffin from the paper: "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"
  ☆56Updated 3 weeks ago
• Adamdad / rational_kat_cu
  ☆76Updated 9 months ago
• lucidrains / gateloop-transformer
  Implementation of GateLoop Transformer in Pytorch and Jax
  ☆90Updated last year
• MarlonBecker / MSAM
  ☆20Updated last year
• i404788 / s5-pytorch
  Pytorch implementation of Simplified Structured State-Spaces for Sequence Modeling (S5)
  ☆80Updated last year
• TorchJD / torchjd
  Library for Jacobian descent with PyTorch. It enables the optimization of neural networks with multiple losses (e.g. multi-task learning)…
  ☆276Updated last week
• TianjinYellow / SPAM-Optimizer
  ☆35Updated 8 months ago
• zqOuO / GWT
  ☆13Updated 10 months ago
• ruke1ire / RTF
  A State-Space Model with Rational Transfer Function Representation.
  ☆83Updated last year
• AngusDujw / SAF
  ☆35Updated 2 years ago
• lucidrains / gradnorm-pytorch
  A practical implementation of GradNorm, Gradient Normalization for Adaptive Loss Balancing, in Pytorch
  ☆117Updated 2 months ago
• lucidrains / infini-transformer-pytorch
  Implementation of Infini-Transformer in Pytorch
  ☆113Updated 10 months ago
• johnma2006 / candle
  Deep learning library implemented from scratch in numpy. Mixtral, Mamba, LLaMA, GPT, ResNet, and other experiments.
  ☆53Updated last year
• piotrpiekos / MoSA
  User-friendly implementation of the Mixture-of-Sparse-Attention (MoSA). MoSA selects distinct tokens for each head with expert choice rou…
  ☆28Updated 6 months ago
• KellerJordan / cifar10-airbench
  CIFAR-10 speedruns: 94% in 2.6 seconds and 96% in 27 seconds
  ☆324Updated 4 months ago
• proger / hippogriff
  Griffin MQA + Hawk Linear RNN Hybrid
  ☆89Updated last year
• FarnoushRJ / MambaLRP
  [NeurIPS 2024] Official implementation of the paper "MambaLRP: Explaining Selective State Space Sequence Models" 🐍
  ☆45Updated last year
• sustcsonglin / flash-linear-rnn
  Implementations of various linear RNN layers using pytorch and triton
  ☆54Updated 2 years ago
• fkodom / yet-another-retnet
  A simple but robust PyTorch implementation of RetNet from "Retentive Network: A Successor to Transformer for Large Language Models" (http…
  ☆106Updated last year
• AmeenAli / HiddenMambaAttn
  Official PyTorch Implementation of "The Hidden Attention of Mamba Models"
  ☆229Updated last month
• lucidrains / soft-moe-pytorch
  Implementation of Soft MoE, proposed by Brain's Vision team, in Pytorch
  ☆334Updated 7 months ago
• nblt / F-SAM
  [CVPR 2024] Friendly Sharpness-Aware Minimization
  ☆34Updated last year
• lucidrains / minGRU-pytorch
  Implementation of the proposed minGRU in Pytorch
  ☆308Updated 8 months ago