JRC1995 / DemonRangerOptimizer
Quasi Hyperbolic Rectified DEMON Adam/Amsgrad with AdaMod, Gradient Centralization, Lookahead, iterative averaging and decorrelated Weight Decay
☆25Updated 4 years ago
Related projects ⓘ
Alternatives and complementary repositories for DemonRangerOptimizer
- Easy-to-use AdaHessian optimizer (PyTorch)☆77Updated 4 years ago
- Padé Activation Units: End-to-end Learning of Activation Functions in Deep Neural Network☆64Updated 3 years ago
- This repository contains the results for the paper: "Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers"☆180Updated 3 years ago
- ☆35Updated 9 months ago
- 🧀 Pytorch code for the Fromage optimiser.☆122Updated 4 months ago
- Implementations of quasi-hyperbolic optimization algorithms.☆102Updated 4 years ago
- Official code for the Stochastic Polyak step-size optimizer☆137Updated 5 months ago
- Implementations and checkpoints for ResNet, Wide ResNet, ResNeXt, ResNet-D, and ResNeSt in JAX (Flax).☆104Updated 2 years ago
- ☆36Updated last year
- Code for MSID, a Multi-Scale Intrinsic Distance for comparing generative models, studying neural networks, and more!☆50Updated 5 years ago
- Structured matrices for compressing neural networks☆67Updated last year
- Python implementation of GLN in different frameworks☆95Updated 4 years ago
- ☆29Updated 3 years ago
- 👑 Pytorch code for the Nero optimiser.☆20Updated 2 years ago
- Pytorch implementation of Variational Dropout Sparsifies Deep Neural Networks☆84Updated 2 years ago
- Layerwise Batch Entropy Regularization☆22Updated 2 years ago
- Partial implementation of ODE-GAN technique from the paper Training Generative Adversarial Networks by Solving Ordinary Differential Equa…☆16Updated 4 years ago
- Implements sharpness-aware minimization (https://arxiv.org/abs/2010.01412) in TensorFlow 2.☆61Updated 3 years ago
- 👩 Pytorch and Jax code for the Madam optimiser.☆51Updated 3 years ago
- Large dataset storage format for Pytorch☆45Updated 3 years ago
- Codebase for Learning Invariances in Neural Networks☆94Updated 2 years ago
- Pytorch implementation of preconditioned stochastic gradient descent (affine group preconditioner, low-rank approximation preconditioner …☆128Updated last month
- Cyclemoid implementation for PyTorch☆87Updated 2 years ago
- Deep neural architecture research framework☆12Updated last year
- Image augmentation library for Jax☆37Updated 7 months ago
- Torchélie is a set of utility functions, layers, losses, models, trainers and other things for PyTorch.☆111Updated 2 weeks ago
- Gradient Origin Networks - a new type of generative model that is able to quickly learn a latent representation without an encoder☆161Updated 3 years ago
- PyTorch-SSO: Scalable Second-Order methods in PyTorch☆142Updated last year
- Learning to Initialize Neural Networks for Stable and Efficient Training☆136Updated 2 years ago
- PyTorch optimizers with sparse momentum and weight decay☆9Updated 4 years ago