Alternatives and similar repositories for rectified-linear-attention

Users that are interested in rectified-linear-attention are comparing it to the libraries listed below

• Chenglin-Yang / LESA
  Locally Enhanced Self-Attention: Rethinking Self-Attention as Local and Context Terms
  ☆20Updated 3 years ago
• jaketae / g-mlp
  PyTorch implementation of Pay Attention to MLPs
  ☆40Updated 3 years ago
• davidsvy / cosformer-pytorch
  Unofficial PyTorch implementation of the paper "cosFormer: Rethinking Softmax In Attention".
  ☆44Updated 3 years ago
• fawazsammani / mogrifier-lstm-pytorch
  Implementation of Mogrifier LSTM in PyTorch
  ☆35Updated 5 years ago
• hunto / ReLoss
  Official implementation for paper "Relational Surrogate Loss Learning", ICLR 2022
  ☆36Updated 2 years ago
• DequanWang / weightnet.pytorch
  WeightNet: Revisiting the Design Space of Weight Networks
  ☆19Updated 4 years ago
• rishikksh20 / CoaT-pytorch
  CoaT: Co-Scale Conv-Attentional Image Transformers
  ☆16Updated 4 years ago
• yuweihao / LV-BERT
  LV-BERT: Exploiting Layer Variety for BERT (Findings of ACL 2021)
  ☆18Updated 2 years ago
• lancopku / Explicit-Sparse-Transformer
  code for Explicit Sparse Transformer
  ☆62Updated last year
• Noahs-ARK / RFA
  ☆33Updated 4 years ago
• lzy1732008 / GaussionTransformer
  For paper《Gaussian Transformer: A Lightweight Approach for Natural Language Inference》
  ☆28Updated 5 years ago
• kuixu / Linear-Multihead-Attention
  Reproducing the Linear Multihead Attention introduced in Linformer paper (Linformer: Self-Attention with Linear Complexity)
  ☆76Updated 5 years ago
• TencentARC / DTN
  Official code for "Dynamic Token Normalization Improves Vision Transformer", ICLR 2022.
  ☆28Updated 3 years ago
• yikangshen / MoA
  Mixture of Attention Heads
  ☆47Updated 2 years ago
• cheneydon / efficient-bert
  This repository contains the code for the paper in Findings of EMNLP 2021: "EfficientBERT: Progressively Searching Multilayer Perceptron …
  ☆33Updated 2 years ago
• VITA-Group / layerGraftedPretraining_ICLR23
  [ICLR 2023] “ Layer Grafted Pre-training: Bridging Contrastive Learning And Masked Image Modeling For Better Representations”, Ziyu Jian…
  ☆24Updated 2 years ago
• DeadAt0m / adafactor-pytorch
  A pytorch realization of adafactor (https://arxiv.org/pdf/1804.04235.pdf )
  ☆24Updated 5 years ago
• btma48 / AutoLA
  Code of our Neurips2020 paper "Auto Learning Attention", coming soon
  ☆22Updated 4 years ago
• LibertFan / MAN
  Mask Attention Networks: Rethinking and Strengthen Transformer in NAACL2021
  ☆14Updated 4 years ago
• calclavia / Performer-Pytorch
  Pytorch implementation of Performer from the paper "Rethinking Attention with Performers".
  ☆25Updated 4 years ago
• luogen1996 / LWTransformer
  Lightweight Transformer for Multi-modal Tasks
  ☆16Updated 2 years ago
• elis2496 / maxup_implementation
  ☆12Updated 4 years ago
• ziplab / EcoFormer
  [NeurIPS 2022 Spotlight] This is the official PyTorch implementation of "EcoFormer: Energy-Saving Attention with Linear Complexity"
  ☆72Updated 2 years ago
• d-li14 / mlp-mixer.pytorch
  PyTorch implementation of MLP-Mixer
  ☆37Updated 4 years ago
• xuanqing94 / FLOATER
  Learning to Encode Position for Transformer with Continuous Dynamical Model
  ☆60Updated 4 years ago
• leaderj1001 / Synthesizer-Rethinking-Self-Attention-Transformer-Models
  Implementing SYNTHESIZER: Rethinking Self-Attention in Transformer Models using Pytorch
  ☆70Updated 5 years ago
• rentainhe / pytorch-pooling
  Test different pooling method used in CNN for Computer Vision Task
  ☆35Updated 4 years ago
• deep-spin / mcan-vqa-continuous-attention
  ☆23Updated 4 years ago
• megvii-research / revisitAIRL
  [ECCV2022] Revisiting the Critical Factors of Augmentation-Invariant Representation Learning
  ☆12Updated 2 years ago
• santient / sparse-transformer
  Sparse Transformer with limited attention span in PyTorch
  ☆13Updated 4 years ago