Alternatives and similar repositories for cosFormer:

Users that are interested in cosFormer are comparing it to the libraries listed below

• lucidrains / FLASH-pytorch
  Implementation of the Transformer variant proposed in "Transformer Quality in Linear Time"
  ☆358Updated last year
• NVIDIA / transformer-ls
  Official PyTorch Implementation of Long-Short Transformer (NeurIPS 2021).
  ☆226Updated 2 years ago
• lucidrains / linformer
  Implementation of Linformer for Pytorch
  ☆266Updated last year
• haofanwang / awesome-mlp-papers
  Recent Advances in MLP-based Models (MLP is all you need!)
  ☆113Updated 2 years ago
• lucidrains / sinkhorn-transformer
  Sinkhorn Transformer - Practical implementation of Sparse Sinkhorn Attention
  ☆257Updated 3 years ago
• OpenNLPLab / Transnormer
  [EMNLP 2022] Official implementation of Transnormer in our EMNLP 2022 paper - The Devil in Linear Transformer
  ☆59Updated last year
• lucidrains / local-attention
  An implementation of local windowed attention for language modeling
  ☆420Updated last month
• lucidrains / linear-attention-transformer
  Transformer based on a variant of attention that is linear complexity in respect to sequence length
  ☆738Updated 9 months ago
• kuixu / Linear-Multihead-Attention
  Reproducing the Linear Multihead Attention introduced in Linformer paper (Linformer: Self-Attention with Linear Complexity)
  ☆73Updated 4 years ago
• DRSY / EMO
  [ICLR 2024]EMO: Earth Mover Distance Optimization for Auto-Regressive Language Modeling(https://arxiv.org/abs/2310.04691)
  ☆119Updated 11 months ago
• lucidrains / long-short-transformer
  Implementation of Long-Short Transformer, combining local and global inductive biases for attention over long sequences, in Pytorch
  ☆118Updated 3 years ago
• lucidrains / routing-transformer
  Fully featured implementation of Routing Transformer
  ☆288Updated 3 years ago
• HKUNLP / efficient-attention
  [EVA ICLR'23; LARA ICML'22] Efficient attention mechanisms via control variates, random features, and importance sampling
  ☆80Updated last year
• cmsflash / efficient-attention
  An implementation of the efficient attention module.
  ☆301Updated 4 years ago
• thuml / LogME
  Code release for "LogME: Practical Assessment of Pre-trained Models for Transfer Learning" (ICML 2021) and Ranking and Tuning Pre-trained…
  ☆205Updated last year
• twistedcubic / attention-rank-collapse
  [ICML 2021 Oral] We show pure attention suffers rank collapse, and how different mechanisms combat it.
  ☆164Updated 3 years ago
• lancopku / Explicit-Sparse-Transformer
  code for Explicit Sparse Transformer
  ☆60Updated last year
• Arnav0400 / ViT-Slim
  Official code for our CVPR'22 paper “Vision Transformer Slimming: Multi-Dimension Searching in Continuous Optimization Space”
  ☆248Updated last year
• lucidrains / flash-cosine-sim-attention
  Implementation of fused cosine similarity attention in the same style as Flash Attention
  ☆210Updated 2 years ago
• wuch15 / Fastformer
  A pytorch &keras implementation and demo of Fastformer.
  ☆187Updated 2 years ago
• bzhangGo / rmsnorm
  Root Mean Square Layer Normalization
  ☆230Updated last year
• OpenNLPLab / Tnn
  [ICLR 2023] Official implementation of Transnormer in our ICLR 2023 paper - Toeplitz Neural Network for Sequence Modeling
  ☆78Updated 9 months ago
• david-knigge / ccnn
  Code repository of the paper "Modelling Long Range Dependencies in ND: From Task-Specific to a General Purpose CNN" https://arxiv.org/abs…
  ☆183Updated last year
• ziplab / EcoFormer
  [NeurIPS 2022 Spotlight] This is the official PyTorch implementation of "EcoFormer: Energy-Saving Attention with Linear Complexity"
  ☆70Updated 2 years ago
• zhihou7 / BatchFormer
  CVPR2022, BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learning, https://arxiv.org/abs/2203.01522
  ☆248Updated last year
• ylsung / Ladder-Side-Tuning
  PyTorch codes for "LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learning"
  ☆234Updated 2 years ago
• transformer-vq / transformer_vq
  ☆187Updated last year
• lucidrains / memformer
  Implementation of Memformer, a Memory-augmented Transformer, in Pytorch
  ☆111Updated 4 years ago
• microsoft / vision-longformer
  ☆245Updated 2 years ago
• lucidrains / fast-transformer-pytorch
  Implementation of Fast Transformer in Pytorch
  ☆172Updated 3 years ago