davisyoshida / tf2-gradient-checkpointingLinks
Simple gradient checkpointing for eager mode execution
☆46Updated 4 years ago
Alternatives and similar repositories for tf2-gradient-checkpointing
Users that are interested in tf2-gradient-checkpointing are comparing it to the libraries listed below
Sorting:
- LAMB Optimizer for Large Batch Training (TensorFlow version)☆120Updated 5 years ago
- Implementation of the LAMB optimizer for Keras from the paper "Reducing BERT Pre-Training Time from 3 Days to 76 Minutes"☆75Updated 6 years ago
- Training Transformer-XL on 128 GPUs☆140Updated 4 years ago
- Source code for "Efficient Training of BERT by Progressively Stacking"☆112Updated 5 years ago
- This repository contains example code to build models on TPUs☆30Updated 2 years ago
- Efficient, check-pointed data loading for deep learning with massive data sets.☆208Updated last year
- pytorch implement of Lookahead Optimizer☆190Updated 2 years ago
- Adaptive Softmax implementation for PyTorch☆80Updated 6 years ago
- Concise deep learning for JAX☆183Updated 4 years ago
- Lookahead mechanism for optimizers in Keras.☆50Updated 3 years ago
- Torch Distributed Experimental☆117Updated 10 months ago
- ☆165Updated 6 years ago
- Scripts to train a bidirectional LSTM with knowledge distillation from BERT☆158Updated 5 years ago
- Official Pytorch Implementation of Length-Adaptive Transformer (ACL 2021)☆101Updated 4 years ago
- Make Torchtext work with Keras.☆18Updated 6 years ago
- LM Pretraining with PyTorch/TPU☆134Updated 5 years ago
- wrapping a keras optimizer to implement gradient accumulation☆119Updated 4 years ago
- PyTorch Language Model for 1-Billion Word (LM1B / GBW) Dataset☆123Updated 5 years ago
- Transformers without Tears: Improving the Normalization of Self-Attention☆131Updated last year
- AdamW optimizer for Keras☆115Updated 5 years ago
- a Pytorch implementation of the Reformer Network (https://openreview.net/pdf?id=rkgNKkHtvB)☆53Updated 2 years ago
- RNN benchmarks of pytorch, tensorflow and theano☆89Updated 6 years ago
- Keras/TF implementation of AdamW, SGDW, NadamW, Warm Restarts, and Learning Rate multipliers☆167Updated 3 years ago
- ☆47Updated 4 years ago
- Configure Python functions explicitly and safely☆126Updated 6 months ago
- Efficient Transformers for research, PyTorch and Tensorflow using Locality Sensitive Hashing☆95Updated 5 years ago
- Fork of huggingface/pytorch-pretrained-BERT for BERT on STILTs☆107Updated 2 years ago
- PyTorch Examples repo for "ReZero is All You Need: Fast Convergence at Large Depth"☆62Updated 10 months ago
- This repository contains the code for running the character-level Sandwich Transformers from our ACL 2020 paper on Improving Transformer …☆55Updated 4 years ago
- Personal collection of references for high performance mixed precision training.☆41Updated 5 years ago