lena-voita / the-story-of-heads

Related projects ⓘ

Alternatives and complementary repositories for the-story-of-heads

• pmichel31415 / are-16-heads-really-better-than-1
  Code for the paper "Are Sixteen Heads Really Better than One?"
  ☆169Updated 4 years ago
• LiyuanLucasLiu / Transformer-Clinic
  Understanding the Difficulty of Training Transformers
  ☆328Updated 2 years ago
• IntelLabs / academic-budget-bert
  Repository containing code for "How to Train BERT with an Academic Budget" paper
  ☆309Updated last year
• richarddwang / electra_pytorch
  Pretrain and finetune ELECTRA with fastai and huggingface. (Results of the paper replicated !)
  ☆325Updated 10 months ago
• princeton-nlp / CoFiPruning
  [ACL 2022] Structured Pruning Learns Compact and Accurate Models https://arxiv.org/abs/2204.00408
  ☆192Updated last year
• urvashik / knnlm
  ☆315Updated 3 years ago
• lucidrains / routing-transformer
  Fully featured implementation of Routing Transformer
  ☆284Updated 3 years ago
• facebookresearch / Mask-Predict
  A masked language modeling objective to train a model to predict any subset of the target words, conditioned on both the input text and a…
  ☆241Updated 3 years ago
• asappresearch / revisit-bert-finetuning
  For the code release of our arXiv paper "Revisiting Few-sample BERT Fine-tuning" (https://arxiv.org/abs/2006.05987).
  ☆184Updated last year
• google-research / adapter-bert
  ☆484Updated last year
• kahne / NonAutoregGenProgress
  Tracking the progress in non-autoregressive generation (translation, transcription, etc.)
  ☆305Updated last year
• dreamgonfly / transformer-pytorch
  A PyTorch implementation of Transformer in "Attention is All You Need"
  ☆103Updated 3 years ago
• allenai / acl2022-zerofewshot-tutorial
  ☆294Updated last year
• castorini / DeeBERT
  DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference
  ☆153Updated 2 years ago
• successar / AttentionExplanation
  ☆310Updated 2 years ago
• facebookresearch / unlikelihood_training
  Neural Text Generation with Unlikelihood Training
  ☆310Updated 3 years ago
• clarkkev / attention-analysis
  ☆454Updated 3 years ago
• tnq177 / transformers_without_tears
  Transformers without Tears: Improving the Normalization of Self-Attention
  ☆130Updated 5 months ago
• huggingface / nn_pruning
  Prune a model while finetuning or training.
  ☆394Updated 2 years ago
• dreamgonfly / BERT-pytorch
  PyTorch implementation of BERT in "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding"
  ☆97Updated 6 years ago
• guolinke / TUPE
  Transformer with Untied Positional Encoding (TUPE). Code of paper "Rethinking Positional Encoding in Language Pre-training". Improve exis…
  ☆250Updated 3 years ago
• lucidrains / electra-pytorch
  A simple and working implementation of Electra, the fastest way to pretrain language models from scratch, in Pytorch
  ☆222Updated last year
• laiguokun / Funnel-Transformer
  ☆212Updated 4 years ago
• bert-nmt / bert-nmt
  ☆360Updated last year
• roomylee / nlp-papers-with-arxiv
  Statistics and accepted paper list of NLP conferences with arXiv link
  ☆430Updated 3 years ago
• JetRunner / BERT-of-Theseus
  ⛵️The official PyTorch implementation for "BERT-of-Theseus: Compressing BERT by Progressive Module Replacing" (EMNLP 2020).
  ☆312Updated last year
• princeton-nlp / LM-BFF
  [ACL 2021] LM-BFF: Better Few-shot Fine-tuning of Language Models https://arxiv.org/abs/2012.15723
  ☆721Updated 2 years ago
• bloodwass / mixout
  Implementation of Mixout with PyTorch
  ☆74Updated last year
• facebookresearch / MetaICL
  An original implementation of "MetaICL Learning to Learn In Context" by Sewon Min, Mike Lewis, Luke Zettlemoyer and Hannaneh Hajishirzi
  ☆254Updated last year
• ictnlp / awesome-transformer
  A collection of transformer's guides, implementations and variants.
  ☆103Updated 4 years ago