Alternatives and similar repositories for rectified-linear-attention:

Users that are interested in rectified-linear-attention are comparing it to the libraries listed below

• jaketae / g-mlp
  PyTorch implementation of Pay Attention to MLPs
  ☆40Updated 3 years ago
• Chenglin-Yang / LESA
  Locally Enhanced Self-Attention: Rethinking Self-Attention as Local and Context Terms
  ☆20Updated 3 years ago
• hunto / ReLoss
  Official implementation for paper "Relational Surrogate Loss Learning", ICLR 2022
  ☆36Updated 2 years ago
• DequanWang / weightnet.pytorch
  WeightNet: Revisiting the Design Space of Weight Networks
  ☆19Updated 4 years ago
• cheneydon / efficient-bert
  This repository contains the code for the paper in Findings of EMNLP 2021: "EfficientBERT: Progressively Searching Multilayer Perceptron …
  ☆32Updated last year
• elis2496 / maxup_implementation
  ☆12Updated 4 years ago
• rishikksh20 / CoaT-pytorch
  CoaT: Co-Scale Conv-Attentional Image Transformers
  ☆16Updated 3 years ago
• yuweihao / LV-BERT
  LV-BERT: Exploiting Layer Variety for BERT (Findings of ACL 2021)
  ☆18Updated last year
• lzy1732008 / GaussionTransformer
  For paper《Gaussian Transformer: A Lightweight Approach for Natural Language Inference》
  ☆28Updated 5 years ago
• davidsvy / cosformer-pytorch
  Unofficial PyTorch implementation of the paper "cosFormer: Rethinking Softmax In Attention".
  ☆44Updated 3 years ago
• ziplab / EcoFormer
  [NeurIPS 2022 Spotlight] This is the official PyTorch implementation of "EcoFormer: Energy-Saving Attention with Linear Complexity"
  ☆71Updated 2 years ago
• lancopku / AdaNorm
  Code for "Understanding and Improving Layer Normalization"
  ☆46Updated 5 years ago
• CupidJay / MoCov3-pytorch
  custom pytorch implementation of MoCo v3
  ☆45Updated 3 years ago
• btma48 / AutoLA
  Code of our Neurips2020 paper "Auto Learning Attention", coming soon
  ☆22Updated 3 years ago
• xuanqing94 / FLOATER
  Learning to Encode Position for Transformer with Continuous Dynamical Model
  ☆59Updated 4 years ago
• leaderj1001 / Synthesizer-Rethinking-Self-Attention-Transformer-Models
  Implementing SYNTHESIZER: Rethinking Self-Attention in Transformer Models using Pytorch
  ☆70Updated 4 years ago
• fawazsammani / mogrifier-lstm-pytorch
  Implementation of Mogrifier LSTM in PyTorch
  ☆35Updated 4 years ago
• LibertFan / MAN
  Mask Attention Networks: Rethinking and Strengthen Transformer in NAACL2021
  ☆14Updated 3 years ago
• calclavia / Performer-Pytorch
  Pytorch implementation of Performer from the paper "Rethinking Attention with Performers".
  ☆25Updated 4 years ago
• okojoalg / raft-mlp
  ☆25Updated 3 years ago
• guoyongcs / NATv2
  Implementation for NATv2.
  ☆23Updated 4 years ago
• luogen1996 / LWTransformer
  Lightweight Transformer for Multi-modal Tasks
  ☆15Updated 2 years ago
• researchmm / CyDAS
  Cyclic Differentiable Architecture Search
  ☆36Updated 3 years ago
• VITA-Group / layerGraftedPretraining_ICLR23
  [ICLR 2023] “ Layer Grafted Pre-training: Bridging Contrastive Learning And Masked Image Modeling For Better Representations”, Ziyu Jian…
  ☆24Updated 2 years ago
• ljpadam / LG-Transformer
  ☆16Updated 3 years ago
• Torment123 / DFS
  ☆15Updated 5 years ago
• mlpc-ucsd / BERT_Convolutions
  (ACL-IJCNLP 2021) Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language Models.
  ☆21Updated 2 years ago
• yikangshen / MoA
  Mixture of Attention Heads
  ☆43Updated 2 years ago
• leaderj1001 / Bag-of-MLP
  Bag of MLP
  ☆20Updated 3 years ago
• DeadAt0m / adafactor-pytorch
  A pytorch realization of adafactor (https://arxiv.org/pdf/1804.04235.pdf )
  ☆23Updated 5 years ago