Alternatives and similar repositories for FlashWindowAttention

Users that are interested in FlashWindowAttention are comparing it to the libraries listed below

• fla-org / flash-bidirectional-linear-attention
  Triton implement of bi-directional (non-causal) linear attention
  ☆65Updated last week
• zugexiaodui / torch_flops
  A library for calculating the FLOPs in the forward() process based on torch.fx
  ☆137Updated last month
• HazyResearch / flash-fft-conv
  FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores
  ☆341Updated last year
• kyleliang919 / C-Optim
  When it comes to optimizers, it's always better to be safe than sorry
  ☆402Updated 4 months ago
• nil0x9 / flash-muon
  Flash-Muon: An Efficient Implementation of Muon Optimizer
  ☆233Updated 7 months ago
• MzeroMiko / mamba-mini
  An efficient pytorch implementation of selective scan in one file, works with both cpu and gpu, with corresponding mathematical derivatio…
  ☆101Updated 3 months ago
• lucidrains / flash-cosine-sim-attention
  Implementation of fused cosine similarity attention in the same style as Flash Attention
  ☆220Updated 2 years ago
• OscarXZQ / weight-selection
  ☆191Updated last year
• karttikeya / minREV
  A simple minimal implementation of Reversible Vision Transformers
  ☆127Updated last year
• ivan-chai / torch-linear-assignment
  Batch computation of the linear assignment problem on GPU.
  ☆105Updated 4 months ago
• snu-mllab / LayerMerge
  Official PyTorch implementation of "LayerMerge: Neural Network Depth Compression through Layer Pruning and Merging" (ICML 2024)
  ☆31Updated last year
• mit-han-lab / patch_conv
  Patch convolution to avoid large GPU memory usage of Conv2D
  ☆95Updated last year
• lucidrains / adam-atan2-pytorch
  Implementation of the proposed Adam-atan2 from Google Deepmind in Pytorch
  ☆135Updated 3 months ago
• OpenNLPLab / lightning-attention
  Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models
  ☆340Updated 11 months ago
• Tencent-Hunyuan / flex-block-attn
  flex-block-attn: an efficient block sparse attention computation library
  ☆108Updated last month
• lucidrains / hyper-connections
  Attempt to make multiple residual streams from Bytedance's Hyper-Connections paper accessible to the public
  ☆168Updated 3 weeks ago
• ThisisBillhe / EfficientDM
  [ICLR 2024 Spotlight] This is the official PyTorch implementation of "EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Di…
  ☆68Updated last year
• kyegomez / FlashAttention20
  Get down and dirty with FlashAttention2.0 in pytorch, plug in and play no complex CUDA kernels
  ☆112Updated 2 years ago
• meta-pytorch / superblock
  A block oriented training approach for inference time optimization.
  ☆34Updated last year
• mdy666 / Scalable-Flash-Native-Sparse-Attention
  ☆48Updated last month
• kyegomez / SparseAttention
  Pytorch Implementation of the sparse attention from the paper: "Generating Long Sequences with Sparse Transformers"
  ☆94Updated last week
• NVlabs / GatedDeltaNet
  [ICLR 2025] Official PyTorch Implementation of Gated Delta Networks: Improving Mamba2 with Delta Rule
  ☆452Updated 4 months ago
• JeongsooP / RGB-no-more
  An official code release of the paper RGB no more: Minimally Decoded JPEG Vision Transformers
  ☆57Updated 2 years ago
• lucidrains / linformer
  Implementation of Linformer for Pytorch
  ☆305Updated 2 years ago
• jundaf2 / INT8-Flash-Attention-FMHA-Quantization
  ☆160Updated 2 years ago
• meta-pytorch / float8_experimental
  This repository contains the experimental PyTorch native float8 training UX
  ☆227Updated last year
• lucidrains / flash-attention
  Fast and memory-efficient exact attention
  ☆20Updated last year
• bobby-he / simplified_transformers
  ☆292Updated last year
• transformer-vq / transformer_vq
  ☆201Updated 2 years ago
• apple / ml-sigmoid-attention
  ☆307Updated 9 months ago