Alternatives and similar repositories for bigbird:

Users that are interested in bigbird are comparing it to the libraries listed below

• allenai / longformer
  Longformer: The Long-Document Transformer
  ☆2,072Updated last year
• google-research / long-range-arena
  Long Range Arena for Benchmarking Efficient Transformers
  ☆739Updated last year
• Lightning-Universe / lightning-transformers
  Flexible components pairing 🤗 Transformers with Pytorch Lightning
  ☆613Updated 2 years ago
• IntelLabs / academic-budget-bert
  Repository containing code for "How to Train BERT with an Academic Budget" paper
  ☆310Updated last year
• JohnGiorgi / DeCLUTR
  The corresponding code from our paper "DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations". Do not hesitate to o…
  ☆380Updated last year
• facebookresearch / SpanBERT
  Code for using and evaluating SpanBERT.
  ☆895Updated last year
• allenai / dont-stop-pretraining
  Code associated with the Don't Stop Pretraining ACL 2020 paper
  ☆529Updated 3 years ago
• lucidrains / performer-pytorch
  An implementation of Performer, a linear attention-based transformer, in Pytorch
  ☆1,110Updated 2 years ago
• microsoft / fastformers
  FastFormers - highly efficient transformer models for NLU
  ☆703Updated last year
• richarddwang / electra_pytorch
  Pretrain and finetune ELECTRA with fastai and huggingface. (Results of the paper replicated !)
  ☆325Updated last year
• ofirpress / attention_with_linear_biases
  Code for the ALiBi method for transformer language models (ICLR 2022)
  ☆510Updated last year
• google-research / byt5
  ☆489Updated 11 months ago
• lucidrains / reformer-pytorch
  Reformer, the efficient Transformer, in Pytorch
  ☆2,140Updated last year
• google-research / xtreme
  XTREME is a benchmark for the evaluation of the cross-lingual generalization ability of pre-trained multilingual models that covers 40 ty…
  ☆636Updated 2 years ago
• alexa / bort
  Repository for the paper "Optimal Subarchitecture Extraction for BERT"
  ☆471Updated 2 years ago
• facebookresearch / GENRE
  Autoregressive Entity Retrieval
  ☆775Updated last year
• LiyuanLucasLiu / Transformer-Clinic
  Understanding the Difficulty of Training Transformers
  ☆327Updated 2 years ago
• allenai / allennlp-models
  Officially supported AllenNLP models
  ☆534Updated 2 years ago
• allenai / naacl2021-longdoc-tutorial
  ☆344Updated 3 years ago
• tatp22 / linformer-pytorch
  My take on a practical implementation of Linformer for Pytorch.
  ☆409Updated 2 years ago
• lena-voita / the-story-of-heads
  This is a repository with the code for the ACL 2019 paper "Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, t…
  ☆306Updated 3 years ago
• microsoft / fastseq
  An efficient implementation of the popular sequence models for text generation, summarization, and translation tasks. https://arxiv.org/p…
  ☆429Updated 2 years ago
• lucidrains / mlm-pytorch
  An implementation of masked language modeling for Pytorch, made as concise and simple as possible
  ☆179Updated last year
• clarkkev / attention-analysis
  ☆460Updated 3 years ago
• bhoov / exbert
  A Visual Analysis Tool to Explore Learned Representations in Transformers Models
  ☆588Updated 11 months ago
• google-research / adapter-bert
  ☆484Updated last year
• lucidrains / routing-transformer
  Fully featured implementation of Routing Transformer
  ☆288Updated 3 years ago
• GEM-benchmark / NL-Augmenter
  NL-Augmenter 🦎 → 🐍 A Collaborative Repository of Natural Language Transformations
  ☆779Updated 7 months ago
• tunib-ai / parallelformers
  Parallelformers: An Efficient Model Parallelization Toolkit for Deployment
  ☆781Updated last year
• glample / fastBPE
  Fast BPE
  ☆659Updated 6 months ago