titu1994 / keras-LAMB-OptimizerLinks
Implementation of the LAMB optimizer for Keras from the paper "Reducing BERT Pre-Training Time from 3 Days to 76 Minutes"
☆75Updated 6 years ago
Alternatives and similar repositories for keras-LAMB-Optimizer
Users that are interested in keras-LAMB-Optimizer are comparing it to the libraries listed below
Sorting:
- Transformer-XL with checkpoint loader☆68Updated 3 years ago
- Implementation of ULMFit algorithm for text classification via transfer learning☆94Updated 6 years ago
- Layer normalization implemented in Keras☆60Updated 3 years ago
- LAMB Optimizer for Large Batch Training (TensorFlow version)☆120Updated 5 years ago
- AdaBound optimizer in Keras☆56Updated 4 years ago
- RAdam optimizer for keras☆71Updated 5 years ago
- Some frequently used NLP blocks I implemented☆226Updated 6 years ago
- ☆58Updated 5 years ago
- Load GPT-2 checkpoint and generate texts☆127Updated 3 years ago
- Quasi-RNN for language modeling☆57Updated 8 years ago
- Multithreading inference in Tensorflow Estimators. This is a ServiceNow Research project that was started at Element AI.☆57Updated 2 years ago
- A set of simple examples ported from PyTorch for Tensorflow Eager Execution☆73Updated 6 years ago
- Simple Tensorflow implementation of "On The Variance Of The Adaptive Learning Rate And Beyond"☆97Updated 5 years ago
- Code for my blog post☆49Updated 6 years ago
- Adaptive embedding and softmax☆17Updated 3 years ago
- Corrupted labels and label smoothing☆129Updated 7 years ago
- QANet in keras (with Cove)☆66Updated 6 years ago
- AdamW optimizer for Keras☆115Updated 5 years ago
- Scripts to train a bidirectional LSTM with knowledge distillation from BERT☆158Updated 5 years ago
- The experiment result of LSTM language models on PTB (Penn Treebank) and GBW (Google Billion Word) using AdaptiveSoftmax on TensorFlow.☆100Updated 6 years ago
- PyTorch Language Model for 1-Billion Word (LM1B / GBW) Dataset☆123Updated 5 years ago
- Simple Tensorflow Implementation of "A Structured Self-attentive Sentence Embedding" (ICLR 2017)☆91Updated 6 years ago
- A Tensorflow implementation of Yin Wenpeng's recent paper on TACL "Attentive Convolution"☆33Updated 6 years ago
- Code for the Eager Translation Model from the paper You May Not Need Attention☆294Updated 6 years ago
- Lookahead mechanism for optimizers in Keras.☆50Updated 3 years ago
- Re-implementation of ELMo on Keras☆133Updated 2 years ago
- Reproducing Densely Interactive Inference Network in Keras☆75Updated 7 years ago
- LM, ULMFit et al.☆46Updated 5 years ago
- Deep-Atrous-CNN-Text-Network: End-to-end word level model for sentiment analysis and other text classifications☆64Updated 7 years ago
- An implementation of the Globally Normalized Reader☆58Updated 2 years ago