JRC1995 / DemonRangerOptimizer
Quasi Hyperbolic Rectified DEMON Adam/Amsgrad with AdaMod, Gradient Centralization, Lookahead, iterative averaging and decorrelated Weight Decay
☆25Updated 4 years ago
Alternatives and similar repositories for DemonRangerOptimizer:
Users that are interested in DemonRangerOptimizer are comparing it to the libraries listed below
- This repository contains the results for the paper: "Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers"☆180Updated 3 years ago
- Implementations of quasi-hyperbolic optimization algorithms.☆102Updated 4 years ago
- Implements stochastic line search☆118Updated last year
- Cyclemoid implementation for PyTorch☆87Updated 2 years ago
- Ἀνατομή is a PyTorch library to analyze representation of neural networks☆62Updated last year
- ☆36Updated last year
- ☆36Updated 2 years ago
- Layerwise Batch Entropy Regularization☆22Updated 2 years ago
- Easy-to-use AdaHessian optimizer (PyTorch)☆77Updated 4 years ago
- 👩 Pytorch and Jax code for the Madam optimiser.☆51Updated 4 years ago
- 👑 Pytorch code for the Nero optimiser.☆20Updated 2 years ago
- Structured matrices for compressing neural networks☆66Updated last year
- Interactive Weak Supervision: Learning Useful Heuristics for Data Labeling☆31Updated 3 years ago
- Official code for the Stochastic Polyak step-size optimizer☆138Updated 8 months ago
- 🎲 Iterable dataset resampling in PyTorch☆91Updated 3 years ago
- CHOP: An optimization library based on PyTorch, with applications to adversarial examples and structured neural network training.☆77Updated 11 months ago
- ☆33Updated 4 years ago
- Bayesianize: A Bayesian neural network wrapper in pytorch☆88Updated 9 months ago
- Padé Activation Units: End-to-end Learning of Activation Functions in Deep Neural Network☆64Updated 3 years ago
- 🧀 Pytorch code for the Fromage optimiser.☆123Updated 7 months ago
- A GPT, made only of MLPs, in Jax☆57Updated 3 years ago
- Code to accompany the paper Radial Bayesian Neural Networks: Beyond Discrete Support In Large-Scale Bayesian Deep Learning☆33Updated 4 years ago
- Partial implementation of ODE-GAN technique from the paper Training Generative Adversarial Networks by Solving Ordinary Differential Equa…☆16Updated 4 years ago
- Implements sharpness-aware minimization (https://arxiv.org/abs/2010.01412) in TensorFlow 2.☆60Updated 3 years ago
- Code for: "Neural Rough Differential Equations for Long Time Series", (ICML 2021)☆115Updated 3 years ago
- Deep Probabilistic Programming Course @ DIKU☆57Updated 4 years ago
- Official code for UnICORNN (ICML 2021)☆27Updated 3 years ago
- ☆47Updated 2 years ago
- This library is a location of the LegacyLogger for PyTorch Lightning.☆26Updated 2 years ago
- Official code for Long Expressive Memory (ICLR 2022, Spotlight)☆69Updated 2 years ago