zhuchen03 / gradinit
Learning to Initialize Neural Networks for Stable and Efficient Training
☆136Updated 2 years ago
Related projects ⓘ
Alternatives and complementary repositories for gradinit
- Compression schema for gradients of activations in backward pass☆44Updated last year
- A collection of code snippets for my PyTorch Lightning projects☆107Updated 3 years ago
- Unofficial implementation of Perceiver IO☆118Updated 2 years ago
- A GPT, made only of MLPs, in Jax☆55Updated 3 years ago
- Implementation of the Adan (ADAptive Nesterov momentum algorithm) Optimizer in Pytorch☆247Updated 2 years ago
- PyTorch implementation of L2L execution algorithm☆106Updated last year
- GAN models implemented with Pytorch Lightning and Hydra configuration☆34Updated 2 years ago
- EfficientNet, MobileNetV3, MobileNetV2, MixNet, etc in JAX w/ Flax Linen and Objax☆126Updated 10 months ago
- Implementation of Feedback Transformer in Pytorch☆104Updated 3 years ago
- GPT, but made only out of MLPs☆86Updated 3 years ago
- ☆187Updated 2 years ago
- Implementation of Fast Transformer in Pytorch☆171Updated 3 years ago
- Very simple and short implementation of gradient boosting in 18 lines of code☆9Updated 4 years ago
- A simple library that implements CLIP guided loss in PyTorch.☆77Updated 2 years ago
- ☆26Updated 3 years ago
- This repository contains the results for the paper: "Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers"☆180Updated 3 years ago
- http://nlp.seas.harvard.edu/2018/04/03/attention.html☆63Updated 3 years ago
- RUDOLPH: One Hyper-Tasking Transformer can be creative as DALL-E and GPT-3 and smart as CLIP☆253Updated last year
- ☆15Updated 10 months ago
- Implementation of Long-Short Transformer, combining local and global inductive biases for attention over long sequences, in Pytorch☆116Updated 3 years ago
- ☆35Updated 9 months ago
- Pytorch based library to rank predicted bounding boxes using text/image user's prompts.☆51Updated 2 years ago
- Simple and efficient RevNet-Library for PyTorch with XLA and DeepSpeed support and parameter offload☆124Updated 2 years ago
- Official code for "Distributed Deep Learning in Open Collaborations" (NeurIPS 2021)☆116Updated 2 years ago
- An open source implementation of CLIP.☆32Updated 2 years ago
- My repo for training neural nets using pytorch-lightning and hydra☆214Updated 3 months ago
- Code for MSID, a Multi-Scale Intrinsic Distance for comparing generative models, studying neural networks, and more!☆50Updated 5 years ago
- a lightweight transformer library for PyTorch☆72Updated 3 years ago
- Another attempt at a long-context / efficient transformer by me☆37Updated 2 years ago
- Data Reading Blocks for Python☆103Updated 3 years ago