titu1994 / keras-LAMB-Optimizer
Implementation of the LAMB optimizer for Keras from the paper "Reducing BERT Pre-Training Time from 3 Days to 76 Minutes"
☆76Updated 5 years ago
Related projects ⓘ
Alternatives and complementary repositories for keras-LAMB-Optimizer
- Scripts to train a bidirectional LSTM with knowledge distillation from BERT☆157Updated 5 years ago
- LAMB Optimizer for Large Batch Training (TensorFlow version)☆120Updated 4 years ago
- Some frequently used NLP blocks I implemented☆227Updated 5 years ago
- AdaBound optimizer in Keras☆57Updated 4 years ago
- Transformer-XL with checkpoint loader☆68Updated 2 years ago
- Corrupted labels and label smoothing☆128Updated 7 years ago
- Implementation of ULMFit algorithm for text classification via transfer learning☆95Updated 5 years ago
- ☆58Updated 5 years ago
- Adaptive embedding and softmax☆17Updated 2 years ago
- Efficient Transformers for research, PyTorch and Tensorflow using Locality Sensitive Hashing☆93Updated 4 years ago
- Simple Tensorflow Implementation of "A Structured Self-attentive Sentence Embedding" (ICLR 2017)☆92Updated 6 years ago
- Lookahead mechanism for optimizers in Keras.☆49Updated 3 years ago
- Code for my blog post☆49Updated 6 years ago
- The experiment result of LSTM language models on PTB (Penn Treebank) and GBW (Google Billion Word) using AdaptiveSoftmax on TensorFlow.☆101Updated 6 years ago
- PyTorch implementation of StarSpace as described in "StarSpace: Embed All The Things!" by Ledell Wu, Adam Fisch, Sumit Chopra, Keith Adam…☆50Updated 6 years ago
- RAdam optimizer for keras☆71Updated 5 years ago
- Layer normalization implemented in Keras☆60Updated 2 years ago
- Load GPT-2 checkpoint and generate texts☆128Updated 2 years ago
- Language Model Fine-tuning for Moby Dick☆42Updated 5 years ago
- Beam search for neural network sequence to sequence (encoder-decoder) models.☆34Updated 5 years ago
- Simple Tensorflow implementation of "On The Variance Of The Adaptive Learning Rate And Beyond"☆97Updated 4 years ago
- Inference with state-of-the-art models (pre-trained by LD-Net / AutoNER / VanillaNER / ...)☆116Updated 5 years ago
- Pytorch and Torchtext implementation of Sequence to sequence☆60Updated 6 years ago
- Training Transformer-XL on 128 GPUs☆140Updated 4 years ago
- Quasi-RNN for language modeling☆58Updated 7 years ago
- ☆39Updated 7 years ago
- ☆47Updated 5 years ago
- ULMFiT + Siamese Network for Sentence Vectors☆35Updated 6 years ago
- A set of simple examples ported from PyTorch for Tensorflow Eager Execution☆73Updated 6 years ago