NVIDIA / transformer-lsLinks
Official PyTorch Implementation of Long-Short Transformer (NeurIPS 2021).
☆227Updated 3 years ago
Alternatives and similar repositories for transformer-ls
Users that are interested in transformer-ls are comparing it to the libraries listed below
Sorting:
- Fully featured implementation of Routing Transformer☆298Updated 3 years ago
- Implementation of Long-Short Transformer, combining local and global inductive biases for attention over long sequences, in Pytorch☆120Updated 4 years ago
- Implementation of Linformer for Pytorch☆300Updated last year
- Sinkhorn Transformer - Practical implementation of Sparse Sinkhorn Attention☆269Updated 4 years ago
- Unofficial implementation of Google's FNet: Mixing Tokens with Fourier Transforms☆259Updated 4 years ago
- [ICLR 2022] Official implementation of cosformer-attention in cosFormer: Rethinking Softmax in Attention☆195Updated 2 years ago
- [ICML 2020] code for "PowerNorm: Rethinking Batch Normalization in Transformers" https://arxiv.org/abs/2003.07845☆120Updated 4 years ago
- My take on a practical implementation of Linformer for Pytorch.☆419Updated 3 years ago
- Implementation of Fast Transformer in Pytorch☆177Updated 4 years ago
- Sequence modeling with Mega.☆300Updated 2 years ago
- [ICML 2021 Oral] We show pure attention suffers rank collapse, and how different mechanisms combat it.