Alternatives and similar repositories for pytorch_optimizer

Users that are interested in pytorch_optimizer are comparing it to the libraries listed below

• pytorch / torcheval
  A library that contains a rich collection of performant PyTorch model metrics, a simple interface to create new metrics, a toolkit to fac…
  ☆233Updated 5 months ago
• facebookresearch / dadaptation
  D-Adaptation for SGD, Adam and AdaGrad
  ☆522Updated 5 months ago
• lessw2020 / Ranger21
  Ranger deep learning optimizer rewrite to use newest components
  ☆331Updated last year
• lucidrains / memory-efficient-attention-pytorch
  Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attention Does Not Need O(n²) Memory"
  ☆379Updated last year
• HazyResearch / flash-fft-conv
  FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores
  ☆319Updated 5 months ago
• TorchJD / torchjd
  Library for Jacobian descent with PyTorch. It enables the optimization of neural networks with multiple losses (e.g. multi-task learning)…
  ☆246Updated last week
• lucidrains / ema-pytorch
  A simple way to keep track of an Exponential Moving Average (EMA) version of your Pytorch model
  ☆592Updated 6 months ago
• kyleliang919 / C-Optim
  When it comes to optimizers, it's always better to be safe than sorry
  ☆241Updated 2 months ago
• HomebrewML / HeavyBall
  Efficient optimizers
  ☆220Updated this week
• nikhilvyas / SOAP
  ☆190Updated 6 months ago
• mlfoundations / model-soups
  Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
  ☆470Updated 11 months ago
• lucidrains / linear-attention-transformer
  Transformer based on a variant of attention that is linear complexity in respect to sequence length
  ☆777Updated last year
• kach / gradient-descent-the-ultimate-optimizer
  Code for our NeurIPS 2022 paper
  ☆368Updated 2 years ago
• lucidrains / rotary-embedding-torch
  Implementation of Rotary Embeddings, from the Roformer paper, in Pytorch
  ☆694Updated 6 months ago
• lucidrains / flash-cosine-sim-attention
  Implementation of fused cosine similarity attention in the same style as Flash Attention
  ☆214Updated 2 years ago
• OATML / RHO-Loss
  ☆208Updated 2 years ago
• facebookresearch / dropout
  Code release for "Dropout Reduces Underfitting"
  ☆313Updated 2 years ago
• lucidrains / Mega-pytorch
  Implementation of Mega, the Single-head Attention with Multi-headed EMA architecture that currently holds SOTA on Long Range Arena
  ☆204Updated last year
• lucidrains / linformer
  Implementation of Linformer for Pytorch
  ☆288Updated last year
• bobby-he / simplified_transformers
  ☆292Updated 6 months ago
• lucidrains / Adan-pytorch
  Implementation of the Adan (ADAptive Nesterov momentum algorithm) Optimizer in Pytorch
  ☆252Updated 2 years ago
• pseeth / autoclip
  Adaptive Gradient Clipping
  ☆133Updated 2 years ago
• fadel / pytorch_ema
  Tiny PyTorch library for maintaining a moving average of a collection of parameters.
  ☆430Updated 8 months ago
• lucidrains / adam-atan2-pytorch
  Implementation of the proposed Adam-atan2 from Google Deepmind in Pytorch
  ☆108Updated 6 months ago
• zyushun / Adam-mini
  Code for Adam-mini: Use Fewer Learning Rates To Gain More https://arxiv.org/abs/2406.16793
  ☆422Updated last month
• nanowell / AdEMAMix-Optimizer-Pytorch
  The AdEMAMix Optimizer: Better, Faster, Older.
  ☆183Updated 9 months ago
• sail-sg / Adan
  Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models
  ☆792Updated 2 weeks ago
• KellerJordan / cifar10-airbench
  CIFAR-10 speedruns: 94% in 2.6 seconds and 96% in 27 seconds
  ☆251Updated 3 months ago
• fkodom / fft-conv-pytorch
  Implementation of 1D, 2D, and 3D FFT convolutions in PyTorch. Much faster than direct convolutions for large kernel sizes.
  ☆499Updated last year
• proger / accelerated-scan
  Accelerated First Order Parallel Associative Scan
  ☆182Updated 10 months ago