davisyoshida / tf2-gradient-checkpointingLinks
Simple gradient checkpointing for eager mode execution
☆46Updated 4 years ago
Alternatives and similar repositories for tf2-gradient-checkpointing
Users that are interested in tf2-gradient-checkpointing are comparing it to the libraries listed below
Sorting:
- LAMB Optimizer for Large Batch Training (TensorFlow version)☆120Updated 5 years ago
- Implementation of the LAMB optimizer for Keras from the paper "Reducing BERT Pre-Training Time from 3 Days to 76 Minutes"☆75Updated 6 years ago
- Training Transformer-XL on 128 GPUs☆140Updated 5 years ago
- PyTorch implementation of L2L execution algorithm☆107Updated 2 years ago
- Efficient Transformers for research, PyTorch and Tensorflow using Locality Sensitive Hashing☆95Updated 5 years ago
- RNN benchmarks of pytorch, tensorflow and theano☆89Updated 6 years ago
- LM Pretraining with PyTorch/TPU☆134Updated 5 years ago
- Source code for "Efficient Training of BERT by Progressively Stacking"☆112Updated 5 years ago
- Keras/TF implementation of AdamW, SGDW, NadamW, Warm Restarts, and Learning Rate multipliers☆167Updated 3 years ago
- PyTorch Language Model for 1-Billion Word (LM1B / GBW) Dataset☆123Updated 5 years ago
- pytorch implement of Lookahead Optimizer☆190Updated 3 years ago
- ☆14Updated 6 years ago
- ☆218Updated 5 years ago
- A minimal PyTorch Lightning OpenAI GPT w DeepSpeed Training!☆111Updated 2 years ago
- Transformer-XL with checkpoint loader☆68Updated 3 years ago
- This repository contains the code for running the character-level Sandwich Transformers from our ACL 2020 paper on Improving Transformer …