Alternatives and similar repositories for Linear-Transformer

Users that are interested in Linear-Transformer are comparing it to the libraries listed below

• Noahs-ARK / RFA
  ☆33Updated 4 years ago
• lancopku / Explicit-Sparse-Transformer
  code for Explicit Sparse Transformer
  ☆62Updated last year
• yikangshen / MoA
  Mixture of Attention Heads
  ☆44Updated 2 years ago
• VITA-Group / ViT-Anti-Oversmoothing
  [ICLR 2022] "Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practice" by Peihao Wang, Wen…
  ☆79Updated last year
• XueFuzhao / WideNet_Code
  Implementation of AAAI 2022 Paper: Go wider instead of deeper
  ☆32Updated 2 years ago
• jaketae / g-mlp
  PyTorch implementation of Pay Attention to MLPs
  ☆40Updated 3 years ago
• OpenNLPLab / cosFormer
  [ICLR 2022] Official implementation of cosformer-attention in cosFormer: Rethinking Softmax in Attention
  ☆192Updated 2 years ago
• Hao840 / manifold-distillation
  Learning Efficient Vision Transformers via Fine-Grained Manifold Distillation. NeurIPS 2022.
  ☆32Updated 2 years ago
• ag1988 / top_k_attention
  The accompanying code for "Memory-efficient Transformers via Top-k Attention" (Ankit Gupta, Guy Dar, Shaya Goodman, David Ciprut, Jonatha…
  ☆68Updated 3 years ago
• lancopku / well-classified-examples-are-underestimated
  Code for the AAAI 2022 publication "Well-classified Examples are Underestimated in Classification with Deep Neural Networks"
  ☆51Updated 2 years ago
• kuixu / Linear-Multihead-Attention
  Reproducing the Linear Multihead Attention introduced in Linformer paper (Linformer: Self-Attention with Linear Complexity)
  ☆76Updated 4 years ago
• ziplab / LIT
  [AAAI 2022] This is the official PyTorch implementation of "Less is More: Pay Less Attention in Vision Transformers"
  ☆96Updated 2 years ago
• HKUNLP / efficient-attention
  [EVA ICLR'23; LARA ICML'22] Efficient attention mechanisms via control variates, random features, and importance sampling
  ☆85Updated 2 years ago
• xuanqing94 / FLOATER
  Learning to Encode Position for Transformer with Continuous Dynamical Model
  ☆59Updated 4 years ago
• ziyuwwang / DynaMixer
  ☆27Updated 2 years ago
• LibertFan / MAN
  Mask Attention Networks: Rethinking and Strengthen Transformer in NAACL2021
  ☆14Updated 3 years ago
• VITA-Group / SViTE
  [NeurIPS'21] "Chasing Sparsity in Vision Transformers: An End-to-End Exploration" by Tianlong Chen, Yu Cheng, Zhe Gan, Lu Yuan, Lei Zhang…
  ☆89Updated last year
• ggjy / Hire-Wave-MLP.pytorch
  Implementation of Hire-MLP: Vision MLP via Hierarchical Rearrangement and An Image Patch is a Wave: Phase-Aware Vision MLP.
  ☆37Updated 2 years ago
• szq0214 / S2-BNN
  S2-BNN: Bridging the Gap Between Self-Supervised Real and 1-bit Neural Networks via Guided Distribution Calibration (CVPR 2021)
  ☆64Updated 3 years ago
• linhezheng19 / CAT
  Official implement of "CAT: Cross Attention in Vision Transformer".
  ☆159Updated 2 years ago
• WailordHe / DenseSSM
  A repository for DenseSSMs
  ☆87Updated last year
• DRSY / EMO
  [ICLR 2024]EMO: Earth Mover Distance Optimization for Auto-Regressive Language Modeling(https://arxiv.org/abs/2310.04691)
  ☆122Updated last year
• sanagno / adaptively_sparse_attention
  ☆21Updated last year
• VITA-Group / UVC
  [ICLR 2022] "Unified Vision Transformer Compression" by Shixing Yu*, Tianlong Chen*, Jiayi Shen, Huan Yuan, Jianchao Tan, Sen Yang, Ji Li…
  ☆52Updated last year
• BaohaoLiao / mefts
  [NeurIPS 2023] Make Your Pre-trained Model Reversible: From Parameter to Memory Efficient Fine-Tuning
  ☆31Updated last year
• haofanwang / awesome-mlp-papers
  Recent Advances in MLP-based Models (MLP is all you need!)
  ☆115Updated 2 years ago
• ambisinister / lossfreebalance
  toy reproduction of Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts
  ☆13Updated 8 months ago
• RunxinXu / ContrastivePruning
  Source code for our AAAI'22 paper 《From Dense to Sparse: Contrastive Pruning for Better Pre-trained Language Model Compression》
  ☆24Updated 3 years ago
• szhangtju / The-compression-of-Transformer
  ☆64Updated 4 years ago
• rishikksh20 / rectified-linear-attention
  Sparse Attention with Linear Units
  ☆17Updated 4 years ago