Alternatives and similar repositories for Fastformer-PyTorch:

Users that are interested in Fastformer-PyTorch are comparing it to the libraries listed below

• lucidrains / fast-transformer-pytorch
  Implementation of Fast Transformer in Pytorch
  ☆172Updated 3 years ago
• lucidrains / axial-positional-embedding
  Axial Positional Embedding for Pytorch
  ☆69Updated last week
• lucidrains / long-short-transformer
  Implementation of Long-Short Transformer, combining local and global inductive biases for attention over long sequences, in Pytorch
  ☆118Updated 3 years ago
• wuch15 / Fastformer
  A pytorch &keras implementation and demo of Fastformer.
  ☆187Updated 2 years ago
• lucidrains / memory-compressed-attention
  Implementation of Memory-Compressed Attention, from the paper "Generating Wikipedia By Summarizing Long Sequences"
  ☆70Updated last year
• 10-zin / Synthesizer
  A PyTorch implementation of the paper - "Synthesizer: Rethinking Self-Attention in Transformer Models"
  ☆72Updated 2 years ago
• epfml / collaborative-attention
  Code for Multi-Head Attention: Collaborate Instead of Concatenate
  ☆152Updated last year
• NVIDIA / transformer-ls
  Official PyTorch Implementation of Long-Short Transformer (NeurIPS 2021).
  ☆226Updated 2 years ago
• lucidrains / charformer-pytorch
  Implementation of the GBST block from the Charformer paper, in Pytorch
  ☆117Updated 3 years ago
• gsarti / lambda-bert
  A 🤗-style implementation of BERT using lambda layers instead of self-attention
  ☆69Updated 4 years ago
• lucidrains / h-transformer-1d
  Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning
  ☆160Updated last year
• lucidrains / routing-transformer
  Fully featured implementation of Routing Transformer
  ☆288Updated 3 years ago
• pkuzengqi / Skyformer
  Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr\"om Method (NeurIPS 2021)
  ☆60Updated 2 years ago
• ankandrew / online-label-smoothing-pt
  Implementation of Online Label Smoothing in PyTorch
  ☆94Updated 2 years ago
• lucidrains / electra-pytorch
  A simple and working implementation of Electra, the fastest way to pretrain language models from scratch, in Pytorch
  ☆221Updated last year
• rish-16 / aft-pytorch
  Unofficial PyTorch implementation of Attention Free Transformer (AFT) layers by Apple Inc.
  ☆233Updated 2 years ago
• ofirpress / shortformer
  Code for the Shortformer model, from the ACL 2021 paper by Ofir Press, Noah A. Smith and Mike Lewis.
  ☆145Updated 3 years ago
• lucidrains / n-grammer-pytorch
  Implementation of N-Grammer, augmenting Transformers with latent n-grams, in Pytorch
  ☆72Updated 2 years ago
• lucidrains / feedback-transformer-pytorch
  Implementation of Feedback Transformer in Pytorch
  ☆105Updated 3 years ago
• sIncerass / powernorm
  [ICML 2020] code for "PowerNorm: Rethinking Batch Normalization in Transformers" https://arxiv.org/abs/2003.07845
  ☆119Updated 3 years ago
• rishikksh20 / FNet-pytorch
  Unofficial implementation of Google's FNet: Mixing Tokens with Fourier Transforms
  ☆258Updated 3 years ago
• lucidrains / sinkhorn-transformer
  Sinkhorn Transformer - Practical implementation of Sparse Sinkhorn Attention
  ☆257Updated 3 years ago
• lucidrains / nystrom-attention
  Implementation of Nyström Self-attention, from the paper Nyströmformer
  ☆127Updated last year
• tnq177 / transformers_without_tears
  Transformers without Tears: Improving the Normalization of Self-Attention
  ☆130Updated 8 months ago
• laiguokun / Funnel-Transformer
  ☆213Updated 4 years ago
• bloodwass / mixout
  Implementation of Mixout with PyTorch
  ☆74Updated 2 years ago
• ofirpress / sandwich_transformer
  This repository contains the code for running the character-level Sandwich Transformers from our ACL 2020 paper on Improving Transformer …
  ☆55Updated 4 years ago
• cloneofsimo / realformer-pytorch
  Implementation of RealFormer using pytorch
  ☆101Updated 4 years ago
• teddykoker / performer
  Simply Numpy implementation of the FAVOR+ attention mechanism, https://teddykoker.com/2020/11/performers/
  ☆37Updated 4 years ago
• tbachlechner / ReZero-examples
  PyTorch Examples repo for "ReZero is All You Need: Fast Convergence at Large Depth"
  ☆62Updated 6 months ago