Alternatives and similar repositories for rectified-linear-attention:

Users that are interested in rectified-linear-attention are comparing it to the libraries listed below

• jaketae / g-mlp
  PyTorch implementation of Pay Attention to MLPs
  ☆40Updated 3 years ago
• DequanWang / weightnet.pytorch
  WeightNet: Revisiting the Design Space of Weight Networks
  ☆19Updated 4 years ago
• Chenglin-Yang / LESA
  Locally Enhanced Self-Attention: Rethinking Self-Attention as Local and Context Terms
  ☆20Updated 3 years ago
• davidsvy / cosformer-pytorch
  Unofficial PyTorch implementation of the paper "cosFormer: Rethinking Softmax In Attention".
  ☆44Updated 3 years ago
• calclavia / Performer-Pytorch
  Pytorch implementation of Performer from the paper "Rethinking Attention with Performers".
  ☆25Updated 4 years ago
• xyltt / Linear-Transformer
  Transformer are RNNs: Fast Autoregressive Transformer with Linear Attention
  ☆22Updated 4 years ago
• lioutasb / TaLKConvolutions
  Official PyTorch implementation of Time-aware Large Kernel (TaLK) Convolutions (ICML 2020)
  ☆29Updated 4 years ago
• xuanqing94 / FLOATER
  Learning to Encode Position for Transformer with Continuous Dynamical Model
  ☆59Updated 4 years ago
• fawazsammani / mogrifier-lstm-pytorch
  Implementation of Mogrifier LSTM in PyTorch
  ☆35Updated 5 years ago
• Noahs-ARK / RFA
  ☆33Updated 4 years ago
• lzy1732008 / GaussionTransformer
  For paper《Gaussian Transformer: A Lightweight Approach for Natural Language Inference》
  ☆28Updated 5 years ago
• hunto / ReLoss
  Official implementation for paper "Relational Surrogate Loss Learning", ICLR 2022
  ☆37Updated 2 years ago
• lancopku / Explicit-Sparse-Transformer
  code for Explicit Sparse Transformer
  ☆60Updated last year
• DeadAt0m / adafactor-pytorch
  A pytorch realization of adafactor (https://arxiv.org/pdf/1804.04235.pdf )
  ☆23Updated 5 years ago
• elis2496 / maxup_implementation
  ☆12Updated 4 years ago
• rishikksh20 / CoaT-pytorch
  CoaT: Co-Scale Conv-Attentional Image Transformers
  ☆16Updated 4 years ago
• ZichenMiao / L3Net
  ICLR 2021 (spotlight): Graph Convolution with Low-rank Learnable Local Filters
  ☆15Updated 4 years ago
• kuixu / Linear-Multihead-Attention
  Reproducing the Linear Multihead Attention introduced in Linformer paper (Linformer: Self-Attention with Linear Complexity)
  ☆76Updated 4 years ago
• GeeeekExplorer / cupytorch
  A small framework mimics PyTorch using CuPy or NumPy
  ☆27Updated 3 years ago
• MetaLearners / Solution-to-CVPR2021-NAS-competition-Track-1
  ☆13Updated 3 years ago
• Devin-Taylor / MultiAug
  Multi-modal data augmentation for machine learning
  ☆16Updated 5 years ago
• okojoalg / raft-mlp
  ☆25Updated 3 years ago
• biswajitsc / sparse-embed
  Code for paper 'Minimizing FLOPs to Learn Efficient Sparse Representations' published at ICLR 2020
  ☆20Updated 5 years ago
• google-research / noisy-fewshot-learning
  ☆23Updated 4 years ago
• yikangshen / MoA
  Mixture of Attention Heads
  ☆44Updated 2 years ago
• CupidJay / MoCov3-pytorch
  custom pytorch implementation of MoCo v3
  ☆45Updated 4 years ago
• yuweihao / LV-BERT
  LV-BERT: Exploiting Layer Variety for BERT (Findings of ACL 2021)
  ☆18Updated last year
• cheneydon / efficient-bert
  This repository contains the code for the paper in Findings of EMNLP 2021: "EfficientBERT: Progressively Searching Multilayer Perceptron …
  ☆32Updated last year
• leaderj1001 / Synthesizer-Rethinking-Self-Attention-Transformer-Models
  Implementing SYNTHESIZER: Rethinking Self-Attention in Transformer Models using Pytorch
  ☆70Updated 4 years ago
• jinqixiao / ComCAT
  ☆16Updated last year