Alternatives and similar repositories for Fastformer-PyTorch

Users that are interested in Fastformer-PyTorch are comparing it to the libraries listed below

• lucidrains / fast-transformer-pytorch
  Implementation of Fast Transformer in Pytorch
  ☆175Updated 3 years ago
• 10-zin / Synthesizer
  A PyTorch implementation of the paper - "Synthesizer: Rethinking Self-Attention in Transformer Models"
  ☆73Updated 2 years ago
• lucidrains / long-short-transformer
  Implementation of Long-Short Transformer, combining local and global inductive biases for attention over long sequences, in Pytorch
  ☆119Updated 3 years ago
• epfml / collaborative-attention
  Code for Multi-Head Attention: Collaborate Instead of Concatenate
  ☆152Updated 2 years ago
• lucidrains / axial-positional-embedding
  Axial Positional Embedding for Pytorch
  ☆81Updated 4 months ago
• pkuzengqi / Skyformer
  Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr\"om Method (NeurIPS 2021)
  ☆61Updated 3 years ago
• ankandrew / online-label-smoothing-pt
  Implementation of Online Label Smoothing in PyTorch
  ☆94Updated 2 years ago
• sIncerass / powernorm
  [ICML 2020] code for "PowerNorm: Rethinking Batch Normalization in Transformers" https://arxiv.org/abs/2003.07845
  ☆120Updated 4 years ago
• lucidrains / coco-lm-pytorch
  Implementation of COCO-LM, Correcting and Contrasting Text Sequences for Language Model Pretraining, in Pytorch
  ☆46Updated 4 years ago
• lucidrains / memory-compressed-attention
  Implementation of Memory-Compressed Attention, from the paper "Generating Wikipedia By Summarizing Long Sequences"
  ☆71Updated 2 years ago
• wuch15 / Fastformer
  A pytorch &keras implementation and demo of Fastformer.
  ☆189Updated 2 years ago
• lucidrains / distilled-retriever-pytorch
  Implementation of the retriever distillation procedure as outlined in the paper "Distilling Knowledge from Reader to Retriever"
  ☆32Updated 4 years ago
• lucidrains / n-grammer-pytorch
  Implementation of N-Grammer, augmenting Transformers with latent n-grams, in Pytorch
  ☆74Updated 2 years ago
• tbachlechner / ReZero-examples
  PyTorch Examples repo for "ReZero is All You Need: Fast Convergence at Large Depth"
  ☆62Updated 11 months ago
• rish-16 / aft-pytorch
  Unofficial PyTorch implementation of Attention Free Transformer (AFT) layers by Apple Inc.
  ☆237Updated 3 years ago
• NVIDIA / transformer-ls
  Official PyTorch Implementation of Long-Short Transformer (NeurIPS 2021).
  ☆225Updated 3 years ago
• lucidrains / electra-pytorch
  A simple and working implementation of Electra, the fastest way to pretrain language models from scratch, in Pytorch
  ☆227Updated 2 years ago
• bloodwass / mixout
  Implementation of Mixout with PyTorch
  ☆75Updated 2 years ago
• lucidrains / charformer-pytorch
  Implementation of the GBST block from the Charformer paper, in Pytorch
  ☆117Updated 3 years ago
• ofirpress / shortformer
  Code for the Shortformer model, from the ACL 2021 paper by Ofir Press, Noah A. Smith and Mike Lewis.
  ☆147Updated 3 years ago
• lucidrains / rela-transformer
  Implementation of a Transformer using ReLA (Rectified Linear Attention) from https://arxiv.org/abs/2104.07012
  ☆49Updated 3 years ago
• lucidrains / feedback-transformer-pytorch
  Implementation of Feedback Transformer in Pytorch
  ☆107Updated 4 years ago
• lucidrains / multistream-transformers
  Implementation of Multistream Transformers in Pytorch
  ☆54Updated 3 years ago
• gsarti / lambda-bert
  A 🤗-style implementation of BERT using lambda layers instead of self-attention
  ☆69Updated 4 years ago
• lucidrains / h-transformer-1d
  Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning
  ☆161Updated last year
• lucidrains / tableformer-pytorch
  Implementation of TableFormer, Robust Transformer Modeling for Table-Text Encoding, in Pytorch
  ☆39Updated 3 years ago
• cloneofsimo / realformer-pytorch
  Implementation of RealFormer using pytorch
  ☆100Updated 4 years ago
• lucidrains / learning-to-expire-pytorch
  An implementation of Transformer with Expire-Span, a circuit for learning which memories to retain
  ☆34Updated 4 years ago
• laiguokun / Funnel-Transformer
  ☆218Updated 5 years ago
• lucidrains / g-mlp-gpt
  GPT, but made only out of MLPs
  ☆89Updated 4 years ago