lyeoni / gpt-pytorch
PyTorch Implementation of OpenAI GPT
☆118Updated last year
Related projects ⓘ
Alternatives and complementary repositories for gpt-pytorch
- A simple and working implementation of Electra, the fastest way to pretrain language models from scratch, in Pytorch☆222Updated last year
- Pretrain and finetune ELECTRA with fastai and huggingface. (Results of the paper replicated !)☆324Updated 10 months ago
- Trains Transformer model variants. Data isn't shuffled between batches.☆141Updated 2 years ago
- Understanding the Difficulty of Training Transformers☆326Updated 2 years ago
- An implementation of masked language modeling for Pytorch, made as concise and simple as possible☆177Updated last year
- Official PyTorch Implementation of Long-Short Transformer (NeurIPS 2021).☆222Updated 2 years ago
- An original implementation of "MetaICL Learning to Learn In Context" by Sewon Min, Mike Lewis, Luke Zettlemoyer and Hannaneh Hajishirzi☆253Updated last year
- ☆452Updated 3 years ago
- Language Modeling Example with Transformers and PyTorch Lighting☆65Updated 3 years ago
- ☆204Updated 7 months ago
- Pytorch Implementation of ALBERT(A Lite BERT for Self-supervised Learning of Language Representations)☆225Updated 3 years ago
- Using Pytorch's nn.Transformer module to create an english to french neural machine translation model.☆77Updated 4 years ago
- Extremely simple and fast word2vec implementation with Negative Sampling + Sub-sampling☆178Updated 3 years ago
- Sequence modeling with Mega.☆297Updated last year
- Repo has PyTorch implementation "Attention is All you Need - Transformers" paper for Machine Translation from French queries to English.☆65Updated 4 years ago
- Code for Multi-Head Attention: Collaborate Instead of Concatenate☆150Updated last year
- Pytorch implementation of Compressive Transformers, from Deepmind☆157Updated 3 years ago
- Optimus: the first large-scale pre-trained VAE language model☆369Updated last year
- PyTorch Implementation of OpenAI GPT-2☆288Updated 4 months ago
- Implements Reformer: The Efficient Transformer in pytorch.☆84Updated 4 years ago
- A minimal PyTorch Lightning OpenAI GPT w DeepSpeed Training!☆111Updated last year
- Code for the paper "True Few-Shot Learning in Language Models" (https://arxiv.org/abs/2105.11447)☆142Updated 3 years ago
- This is a repository with the code for the ACL 2019 paper "Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, t…☆300Updated 3 years ago
- The corresponding code from our paper "DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations". Do not hesitate to o…☆379Updated last year
- Fine-tuning GPT-2 Small for Question Answering☆130Updated last year
- Repository containing code for "How to Train BERT with an Academic Budget" paper☆309Updated last year
- Statistics and accepted paper list of NLP conferences with arXiv link☆430Updated 3 years ago
- Code for the ALiBi method for transformer language models (ICLR 2022)☆506Updated last year
- Implementation of the first paper on word2vec☆202Updated 2 years ago
- [NAACL 2021] This is the code for our paper `Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self…☆200Updated 2 years ago