JRC1995 / DemonRangerOptimizer
Quasi Hyperbolic Rectified DEMON Adam/Amsgrad with AdaMod, Gradient Centralization, Lookahead, iterative averaging and decorrelated Weight Decay
β25Updated 4 years ago
Alternatives and similar repositories for DemonRangerOptimizer:
Users that are interested in DemonRangerOptimizer are comparing it to the libraries listed below
- Layerwise Batch Entropy Regularizationβ22Updated 2 years ago
- π© Pytorch and Jax code for the Madam optimiser.β51Updated 4 years ago
- Implementations of quasi-hyperbolic optimization algorithms.β102Updated 4 years ago
- β36Updated last year
- Easy-to-use AdaHessian optimizer (PyTorch)β78Updated 4 years ago
- π§ Pytorch code for the Fromage optimiser.β124Updated 9 months ago
- PadΓ© Activation Units: End-to-end Learning of Activation Functions in Deep Neural Networkβ64Updated 4 years ago
- A collection of optimizers, some arcane others well known, for Flax.β29Updated 3 years ago
- Code to accompany the paper Radial Bayesian Neural Networks: Beyond Discrete Support In Large-Scale Bayesian Deep Learningβ33Updated 5 years ago
- Cyclemoid implementation for PyTorchβ89Updated 3 years ago
- Pytorch implementation of Variational Dropout Sparsifies Deep Neural Networksβ83Updated 3 years ago
- Online Normalization for Training Neural Networks (Companion Repository)β81Updated 4 years ago
- Implements stochastic line searchβ118Updated 2 years ago
- This repository contains the results for the paper: "Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers"β180Updated 3 years ago
- diffGrad: An Optimization Method for Convolutional Neural Networksβ55Updated 2 years ago
- Study on the applicability of Direct Feedback Alignment to neural view synthesis, recommender systems, geometric learning, and natural laβ¦β87Updated 2 years ago
- β36Updated 2 years ago
- β47Updated 2 years ago
- Structured matrices for compressing neural networksβ66Updated last year
- β37Updated 3 years ago
- Codes accompanying the paper "LaProp: a Better Way to Combine Momentum with Adaptive Gradient"β28Updated 4 years ago
- General Invertible Transformations for Flow-based Generative Modelsβ17Updated 4 years ago
- Official repository for the paper "Going Beyond Linear Transformers with Recurrent Fast Weight Programmers" (NeurIPS 2021)β48Updated 2 years ago
- β73Updated 2 years ago
- A repository containing the code for the Bistable Recurrent Cellβ47Updated 4 years ago
- Image augmentation library for Jaxβ39Updated last year
- Official code for the Stochastic Polyak step-size optimizerβ139Updated 10 months ago
- Partial implementation of ODE-GAN technique from the paper Training Generative Adversarial Networks by Solving Ordinary Differential Equaβ¦β16Updated 4 years ago
- β33Updated 4 years ago
- Official code for Long Expressive Memory (ICLR 2022, Spotlight)β69Updated 3 years ago