zhuchen03 / gradinit
Learning to Initialize Neural Networks for Stable and Efficient Training
☆138Updated 2 years ago
Alternatives and similar repositories for gradinit:
Users that are interested in gradinit are comparing it to the libraries listed below
- Unofficial implementation of Perceiver IO☆120Updated 2 years ago
- Compression schema for gradients of activations in backward pass☆44Updated last year
- Code for MSID, a Multi-Scale Intrinsic Distance for comparing generative models, studying neural networks, and more!☆51Updated 5 years ago
- Implementation of the Adan (ADAptive Nesterov momentum algorithm) Optimizer in Pytorch☆250Updated 2 years ago
- Implementation of Feedback Transformer in Pytorch☆105Updated 3 years ago
- A simple library that implements CLIP guided loss in PyTorch.☆77Updated 3 years ago
- ☆47Updated 4 years ago
- Easy-to-use AdaHessian optimizer (PyTorch)☆77Updated 4 years ago
- PyTorch implementation of L2L execution algorithm☆107Updated 2 years ago
- TF/Keras code for DiffStride, a pooling layer with learnable strides.☆124Updated 2 years ago
- A collection of code snippets for my PyTorch Lightning projects☆107Updated 4 years ago
- Simple and efficient RevNet-Library for PyTorch with XLA and DeepSpeed support and parameter offload☆125Updated 2 years ago
- Cyclemoid implementation for PyTorch☆87Updated 2 years ago
- Implementation of a Transformer that Ponders, using the scheme from the PonderNet paper☆80Updated 3 years ago
- A library helping to gather stats and run checks during training deep learning models with Pytorch☆36Updated 2 years ago
- Pytorch implementation of Variational Dropout Sparsifies Deep Neural Networks☆84Updated 3 years ago
- This repository contains the results for the paper: "Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers"☆180Updated 3 years ago
- GAN models implemented with Pytorch Lightning and Hydra configuration☆34Updated 2 years ago
- 🧀 Pytorch code for the Fromage optimiser.☆122Updated 6 months ago
- Data Reading Blocks for Python☆103Updated 4 years ago
- A GPT, made only of MLPs, in Jax☆57Updated 3 years ago
- A small library with distillation, quantization and pruning pipelines☆26Updated 3 years ago
- Implementation of Estimating Training Data Influence by Tracing Gradient Descent (NeurIPS 2020)☆224Updated 2 years ago
- ☆197Updated 2 years ago
- ☆11Updated 4 years ago
- Another attempt at a long-context / efficient transformer by me☆37Updated 2 years ago
- Collection of the latest, greatest, deep learning optimizers (for Pytorch) - CNN, NLP suitable☆210Updated 3 years ago
- Greedy Bayesian Posterior Approximation with Deep Ensembles. A. Tiulpin and M. B. Blaschko. (2021)☆11Updated 2 years ago
- EfficientNet, MobileNetV3, MobileNetV2, MixNet, etc in JAX w/ Flax Linen and Objax☆127Updated last year
- graftr: an interactive shell to view and edit PyTorch checkpoints.☆111Updated 4 years ago