Alternatives and similar repositories for Nystromformer:

Users that are interested in Nystromformer are comparing it to the libraries listed below

• huggingface / pytorch_block_sparse
  Fast Block Sparse Matrices for Pytorch
  ☆545Updated 4 years ago
• lucidrains / sinkhorn-transformer
  Sinkhorn Transformer - Practical implementation of Sparse Sinkhorn Attention
  ☆260Updated 3 years ago
• google-research / long-range-arena
  Long Range Arena for Benchmarking Efficient Transformers
  ☆750Updated last year
• michaelsdr / momentumnet
  Drop-in replacement for any ResNet with a significantly reduced memory footprint and better representation capabilities
  ☆209Updated 11 months ago
• lucidrains / nystrom-attention
  Implementation of Nyström Self-attention, from the paper Nyströmformer
  ☆131Updated 3 weeks ago
• lucidrains / routing-transformer
  Fully featured implementation of Routing Transformer
  ☆293Updated 3 years ago
• NVIDIA / transformer-ls
  Official PyTorch Implementation of Long-Short Transformer (NeurIPS 2021).
  ☆225Updated 2 years ago
• facebookresearch / mega
  Sequence modeling with Mega.
  ☆295Updated 2 years ago
• lukemelas / do-you-even-need-attention
  Is the attention layer even necessary? (https://arxiv.org/abs/2105.02723)
  ☆484Updated 3 years ago
• ctlllll / SGConv
  ☆164Updated 2 years ago
• ischlag / fast-weight-transformers
  Official code repository of the paper Linear Transformers Are Secretly Fast Weight Programmers.
  ☆104Updated 3 years ago
• karpathy / deep-vector-quantization
  VQVAEs, GumbelSoftmaxes and friends
  ☆558Updated 3 years ago
• facebookresearch / bitsandbytes
  Library for 8-bit optimizers and quantization routines.
  ☆716Updated 2 years ago
• lucidrains / linear-attention-transformer
  Transformer based on a variant of attention that is linear complexity in respect to sequence length
  ☆755Updated 11 months ago
• facebookresearch / transformer-sequential
  Trains Transformer model variants. Data isn't shuffled between batches.
  ☆141Updated 2 years ago
• facebookresearch / torchdim
  Named tensors with first-class dimensions for PyTorch
  ☆320Updated last year
• lucidrains / linformer
  Implementation of Linformer for Pytorch
  ☆278Updated last year
• google-research / rigl
  End-to-end training of sparse deep neural networks with little-to-no performance loss.
  ☆320Updated 2 years ago
• deep-spin / entmax
  The entmax mapping and its loss, a family of sparse softmax alternatives.
  ☆430Updated 9 months ago
• HomebrewML / revlib
  Simple and efficient RevNet-Library for PyTorch with XLA and DeepSpeed support and parameter offload
  ☆127Updated 2 years ago
• LiyuanLucasLiu / Transformer-Clinic
  Understanding the Difficulty of Training Transformers
  ☆329Updated 2 years ago
• DeMoriarty / TorchPQ
  Approximate nearest neighbor search with product quantization on GPU in pytorch and cuda
  ☆223Updated last year
• lucidrains / memory-efficient-attention-pytorch
  Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attention Does Not Need O(n²) Memory"
  ☆375Updated last year
• facebookresearch / ppuda
  Code for Parameter Prediction for Unseen Deep Architectures (NeurIPS 2021)
  ☆487Updated last year
• krasserm / perceiver-io
  A PyTorch implementation of Perceiver, Perceiver IO and Perceiver AR with PyTorch Lightning scripts for distributed training
  ☆461Updated last year
• HazyResearch / hippo-code
  ☆175Updated 10 months ago
• kakaobrain / torchlars
  A LARS implementation in PyTorch
  ☆343Updated 5 years ago
• SirRob1997 / Crowded-Valley---Results
  This repository contains the results for the paper: "Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers"
  ☆180Updated 3 years ago
• rish-16 / aft-pytorch
  Unofficial PyTorch implementation of Attention Free Transformer (AFT) layers by Apple Inc.
  ☆235Updated 3 years ago
• epfml / collaborative-attention
  Code for Multi-Head Attention: Collaborate Instead of Concatenate
  ☆152Updated last year