JRC1995 / DemonRangerOptimizerLinks
Quasi Hyperbolic Rectified DEMON Adam/Amsgrad with AdaMod, Gradient Centralization, Lookahead, iterative averaging and decorrelated Weight Decay
☆25Updated 4 years ago
Alternatives and similar repositories for DemonRangerOptimizer
Users that are interested in DemonRangerOptimizer are comparing it to the libraries listed below
Sorting:
- Official code for the Stochastic Polyak step-size optimizer☆139Updated last year
- This repository contains the results for the paper: "Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers"☆182Updated 4 years ago
- Implements stochastic line search☆118Updated 2 years ago
- Python implementation of GLN in different frameworks☆97Updated 4 years ago
- Pytorch implementation of Variational Dropout Sparsifies Deep Neural Networks☆83Updated 3 years ago
- 🧀 Pytorch code for the Fromage optimiser.☆125Updated last year
- Cyclemoid implementation for PyTorch☆90Updated 3 years ago
- Torchélie is a set of utility functions, layers, losses, models, trainers and other things for PyTorch.☆110Updated 8 months ago
- 👩 Pytorch and Jax code for the Madam optimiser.☆51Updated 4 years ago
- Implements sharpness-aware minimization (https://arxiv.org/abs/2010.01412) in TensorFlow 2.☆60Updated 3 years ago
- Easy-to-use AdaHessian optimizer (PyTorch)☆79Updated 4 years ago
- Implementations of quasi-hyperbolic optimization algorithms.☆102Updated 5 years ago
- Structured matrices for compressing neural networks☆67Updated last year
- ☆36Updated 2 years ago
- ☆78Updated 5 years ago
- ☆36Updated last year
- Loss Patterns of Neural Networks☆85Updated 3 years ago
- A GPT, made only of MLPs, in Jax☆58Updated 4 years ago
- Code for: "Neural Rough Differential Equations for Long Time Series", (ICML 2021)☆118Updated 4 years ago
- Padé Activation Units: End-to-end Learning of Activation Functions in Deep Neural Network☆63Updated 4 years ago
- Keras implementation of Legendre Memory Units☆215Updated last month
- Layerwise Batch Entropy Regularization☆23Updated 3 years ago
- Gradient based Hyperparameter Tuning library in PyTorch☆290Updated 5 years ago
- A collection of optimizers, some arcane others well known, for Flax.☆29Updated 4 years ago
- Collection of the latest, greatest, deep learning optimizers (for Pytorch) - CNN, NLP suitable☆216Updated 4 years ago
- Sharpness-Aware Minimization for Efficiently Improving Generalization☆41Updated 3 years ago
- "Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts" (NeurIPS 2020), original PyTorch implemen…☆57Updated 4 years ago
- ☆74Updated 2 years ago
- Gradient Origin Networks - a new type of generative model that is able to quickly learn a latent representation without an encoder☆161Updated 4 years ago
- Code to accompany the paper Radial Bayesian Neural Networks: Beyond Discrete Support In Large-Scale Bayesian Deep Learning☆33Updated 5 years ago