xyltt / Linear-TransformerLinks
Transformer are RNNs: Fast Autoregressive Transformer with Linear Attention
☆23Updated 4 years ago
Alternatives and similar repositories for Linear-Transformer
Users that are interested in Linear-Transformer are comparing it to the libraries listed below
Sorting:
- code for Explicit Sparse Transformer☆61Updated 2 years ago
- Implementation of the Transformer variant proposed in "Transformer Quality in Linear Time"☆369Updated last year
- [ICLR 2022] Official implementation of cosformer-attention in cosFormer: Rethinking Softmax in Attention☆196Updated 2 years ago
- ☆33Updated 4 years ago
- Implementation of AAAI 2022 Paper: Go wider instead of deeper☆32Updated 2 years ago
- Recent Advances in MLP-based Models (MLP is all you need!)☆116Updated 2 years ago
- An implementation of the efficient attention module.☆320Updated 4 years ago
- PyTorch implementation of Pay Attention to MLPs☆40Updated 4 years ago
- Code for the AAAI 2022 publication "Well-classified Examples are Underestimated in Classification with Deep Neural Networks"☆53Updated 2 years ago
- ☆64Updated 4 years ago
- The accompanying code for "Memory-efficient Transformers via Top-k Attention" (Ankit Gupta, Guy Dar, Shaya Goodman, David Ciprut, Jonatha…☆69Updated 3 years ago
- Learning to Encode Position for Transformer with Continuous Dynamical Model☆60Updated 5 years ago
- Unofficial Implementation of MLP-Mixer, gMLP, resMLP, Vision Permutator, S2MLP, S2MLPv2, RaftMLP, HireMLP, ConvMLP, AS-MLP, SparseMLP, Co…☆170Updated 3 years ago
- Sparse Attention with Linear Units☆19Updated 4 years ago
- Implementing SYNTHESIZER: Rethinking Self-Attention in Transformer Models using Pytorch☆70Updated 5 years ago
- 基于Gated Attention Unit的Transformer模型(尝鲜版)☆98Updated 2 years ago
- [ICLR 2022] "Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practice" by Peihao Wang, Wen…☆80Updated last year
- [AAAI 2022] This is the official PyTorch implementation of "Less is More: Pay Less Attention in Vision Transformers"☆97Updated 3 years ago
- Unofficial implementation of Google's FNet: Mixing Tokens with Fourier Transforms☆259Updated 4 years ago
- Reproducing the Linear Multihead Attention introduced in Linformer paper (Linformer: Self-Attention with Linear Complexity)☆75Updated 5 years ago
- Mixture of Attention Heads☆49Updated 2 years ago
- Implementation of RealFormer using pytorch☆101Updated 4 years ago
- ☆150Updated 11 months ago
- iFormer: Inception Transformer☆247Updated 2 years ago
- Implement the paper "Self-Attention with Relative Position Representations"☆138Updated 4 years ago
- ☆197Updated last year
- CVPR2022, BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learning, https://arxiv.org/abs/2203.01522☆251Updated 2 years ago
- A pytorch &keras implementation and demo of Fastformer.☆189Updated 2 years ago
- Implementation of Linformer for Pytorch☆296Updated last year
- BM-NAS: Bilevel Multimodal Neural Architecture Search (AAAI 2022 Oral)☆19Updated 2 years ago