zhuchen03 / gradinitLinks
Learning to Initialize Neural Networks for Stable and Efficient Training
☆139Updated 3 years ago
Alternatives and similar repositories for gradinit
Users that are interested in gradinit are comparing it to the libraries listed below
Sorting:
- Implementation of Feedback Transformer in Pytorch☆107Updated 4 years ago
- Unofficial implementation of Perceiver IO☆124Updated 3 years ago
- PyTorch implementation of L2L execution algorithm☆107Updated 2 years ago
- Compression schema for gradients of activations in backward pass☆44Updated 2 years ago
- Data Reading Blocks for Python☆105Updated 4 years ago
- GPT, but made only out of MLPs☆89Updated 4 years ago
- graftr: an interactive shell to view and edit PyTorch checkpoints.☆113Updated 4 years ago
- A collection of code snippets for my PyTorch Lightning projects☆108Updated 4 years ago
- [ICML 2020] code for "PowerNorm: Rethinking Batch Normalization in Transformers" https://arxiv.org/abs/2003.07845☆120Updated 4 years ago
- Pytorch implementation of Variational Dropout Sparsifies Deep Neural Networks☆83Updated 3 years ago
- GAN models implemented with Pytorch Lightning and Hydra configuration☆34Updated 3 years ago
- Simple and efficient RevNet-Library for PyTorch with XLA and DeepSpeed support and parameter offload☆128Updated 2 years ago
- EfficientNet, MobileNetV3, MobileNetV2, MixNet, etc in JAX w/ Flax Linen and Objax☆128Updated last year
- ☆47Updated 4 years ago
- Collection of the latest, greatest, deep learning optimizers (for Pytorch) - CNN, NLP suitable☆216Updated 4 years ago
- Implementation of Fast Transformer in Pytorch☆175Updated 3 years ago
- This repository contains the results for the paper: "Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers"☆181Updated 4 years ago
- An open source implementation of CLIP.☆32Updated 2 years ago
- A GPT, made only of MLPs, in Jax☆58Updated 4 years ago
- Unofficial PyTorch Implementation of EvoNorm☆122Updated 3 years ago
- Another attempt at a long-context / efficient transformer by me☆38Updated 3 years ago
- Implementation of Online Label Smoothing in PyTorch☆94Updated 2 years ago
- Minimal implementation of adaptive gradient clipping (https://arxiv.org/abs/2102.06171) in TensorFlow 2.☆85Updated 4 years ago
- ☆208Updated 2 years ago
- Implementation of the Adan (ADAptive Nesterov momentum algorithm) Optimizer in Pytorch☆252Updated 2 years ago
- Pre-trained NFNets with 99% of the accuracy of the official paper "High-Performance Large-Scale Image Recognition Without Normalization".☆159Updated 4 years ago
- A simple library that implements CLIP guided loss in PyTorch.☆77Updated 3 years ago
- A small demonstration of using WebDataset with ImageNet and PyTorch Lightning☆74Updated last year
- RUDOLPH: One Hyper-Tasking Transformer can be creative as DALL-E and GPT-3 and smart as CLIP☆254Updated 2 years ago
- Official code for "Distributed Deep Learning in Open Collaborations" (NeurIPS 2021)☆117Updated 3 years ago