Alternatives and similar repositories for rectified-linear-attention

Users that are interested in rectified-linear-attention are comparing it to the libraries listed below

• jaketae / g-mlp
  PyTorch implementation of Pay Attention to MLPs
  ☆40Updated 3 years ago
• Chenglin-Yang / LESA
  Locally Enhanced Self-Attention: Rethinking Self-Attention as Local and Context Terms
  ☆20Updated 3 years ago
• davidsvy / cosformer-pytorch
  Unofficial PyTorch implementation of the paper "cosFormer: Rethinking Softmax In Attention".
  ☆44Updated 3 years ago
• rishikksh20 / CoaT-pytorch
  CoaT: Co-Scale Conv-Attentional Image Transformers
  ☆16Updated 4 years ago
• DequanWang / weightnet.pytorch
  WeightNet: Revisiting the Design Space of Weight Networks
  ☆19Updated 4 years ago
• calclavia / Performer-Pytorch
  Pytorch implementation of Performer from the paper "Rethinking Attention with Performers".
  ☆25Updated 4 years ago
• yuweihao / LV-BERT
  LV-BERT: Exploiting Layer Variety for BERT (Findings of ACL 2021)
  ☆18Updated 2 years ago
• Chenglin-Yang / LESA_classification
  Locally Enhanced Self-Attention: Rethinking Self-Attention as Local and Context Terms
  ☆11Updated 3 years ago
• DeadAt0m / adafactor-pytorch
  A pytorch realization of adafactor (https://arxiv.org/pdf/1804.04235.pdf )
  ☆24Updated 5 years ago
• fawazsammani / mogrifier-lstm-pytorch
  Implementation of Mogrifier LSTM in PyTorch
  ☆35Updated 5 years ago
• yikangshen / MoA
  Mixture of Attention Heads
  ☆44Updated 2 years ago
• hunto / ReLoss
  Official implementation for paper "Relational Surrogate Loss Learning", ICLR 2022
  ☆37Updated 2 years ago
• leaderj1001 / Bag-of-MLP
  Bag of MLP
  ☆20Updated 4 years ago
• raoyongming / AMixer
  [ECCV 2022] AMixer: Adaptive Weight Mixing for Self-attention Free Vision Transformers
  ☆28Updated 2 years ago
• luogen1996 / LWTransformer
  Lightweight Transformer for Multi-modal Tasks
  ☆16Updated 2 years ago
• HubHop / vit-attention-benchmark
  Benchmarking Attention Mechanism in Vision Transformers.
  ☆18Updated 2 years ago
• VITA-Group / layerGraftedPretraining_ICLR23
  [ICLR 2023] “ Layer Grafted Pre-training: Bridging Contrastive Learning And Masked Image Modeling For Better Representations”, Ziyu Jian…
  ☆24Updated 2 years ago
• lancopku / AdaNorm
  Code for "Understanding and Improving Layer Normalization"
  ☆46Updated 5 years ago
• Xianchao-Wu / perceiver-pytorch
  ☆42Updated 4 years ago
• kuixu / Linear-Multihead-Attention
  Reproducing the Linear Multihead Attention introduced in Linformer paper (Linformer: Self-Attention with Linear Complexity)
  ☆76Updated 4 years ago
• VITA-Group / Sandwich-Batch-Normalization
  [WACV 2022] "Sandwich Batch Normalization: A Drop-In Replacement for Feature Distribution Heterogeneity" by Xinyu Gong, Wuyang Chen, Tian…
  ☆50Updated 3 years ago
• CupidJay / MoCov3-pytorch
  custom pytorch implementation of MoCo v3
  ☆45Updated 4 years ago
• deep-spin / mcan-vqa-continuous-attention
  ☆23Updated 4 years ago
• lioutasb / TaLKConvolutions
  Official PyTorch implementation of Time-aware Large Kernel (TaLK) Convolutions (ICML 2020)
  ☆29Updated 4 years ago
• islamamirul / PermuteNet
  ☆32Updated 2 years ago
• google-research / noisy-fewshot-learning
  ☆23Updated 4 years ago
• Devin-Taylor / MultiAug
  Multi-modal data augmentation for machine learning
  ☆16Updated 6 years ago
• MetaLearners / Solution-to-CVPR2021-NAS-competition-Track-1
  ☆13Updated 3 years ago
• elis2496 / maxup_implementation
  ☆12Updated 4 years ago
• cheneydon / efficient-bert
  This repository contains the code for the paper in Findings of EMNLP 2021: "EfficientBERT: Progressively Searching Multilayer Perceptron …
  ☆32Updated last year