Alternatives and similar repositories for FlashWindowAttention

Users that are interested in FlashWindowAttention are comparing it to the libraries listed below

• fla-org / flash-bidirectional-linear-attention
  Triton implement of bi-directional (non-causal) linear attention
  ☆56Updated 9 months ago
• lucidrains / adam-atan2-pytorch
  Implementation of the proposed Adam-atan2 from Google Deepmind in Pytorch
  ☆134Updated last month
• lucidrains / flash-cosine-sim-attention
  Implementation of fused cosine similarity attention in the same style as Flash Attention
  ☆216Updated 2 years ago
• OscarXZQ / weight-selection
  ☆186Updated last year
• NVlabs / GatedDeltaNet
  [ICLR 2025] Official PyTorch Implementation of Gated Delta Networks: Improving Mamba2 with Delta Rule
  ☆367Updated 2 months ago
• kyleliang919 / C-Optim
  When it comes to optimizers, it's always better to be safe than sorry
  ☆378Updated last month
• zugexiaodui / torch_flops
  A library for calculating the FLOPs in the forward() process based on torch.fx
  ☆130Updated 7 months ago
• mit-han-lab / patch_conv
  Patch convolution to avoid large GPU memory usage of Conv2D
  ☆93Updated 9 months ago
• MzeroMiko / mamba-mini
  An efficient pytorch implementation of selective scan in one file, works with both cpu and gpu, with corresponding mathematical derivatio…
  ☆97Updated last month
• OpenNLPLab / lightning-attention
  Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models
  ☆333Updated 8 months ago
• HazyResearch / flash-fft-conv
  FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores
  ☆331Updated 10 months ago
• meta-pytorch / superblock
  A block oriented training approach for inference time optimization.
  ☆33Updated last year
• lucidrains / gradnorm-pytorch
  A practical implementation of GradNorm, Gradient Normalization for Adaptive Loss Balancing, in Pytorch
  ☆114Updated 2 months ago
• apple / ml-sigmoid-attention
  ☆302Updated 6 months ago
• jundaf2 / INT8-Flash-Attention-FMHA-Quantization
  ☆158Updated 2 years ago
• NX-AI / mlstm_kernels
  Tiled Flash Linear Attention library for fast and efficient mLSTM Kernels.
  ☆74Updated last week
• fkodom / yet-another-retnet
  A simple but robust PyTorch implementation of RetNet from "Retentive Network: A Successor to Transformer for Large Language Models" (http…
  ☆106Updated last year
• nil0x9 / flash-muon
  Flash-Muon: An Efficient Implementation of Muon Optimizer
  ☆206Updated 5 months ago
• lucidrains / deep-cross-attention
  Implementation of the proposed DeepCrossAttention by Heddes et al at Google research, in Pytorch
  ☆94Updated 8 months ago
• kyegomez / FlashAttention20
  Get down and dirty with FlashAttention2.0 in pytorch, plug in and play no complex CUDA kernels
  ☆109Updated 2 years ago
• ThisisBillhe / EfficientDM
  [ICLR 2024 Spotlight] This is the official PyTorch implementation of "EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Di…
  ☆66Updated last year
• bzhangGo / rmsnorm
  Root Mean Square Layer Normalization
  ☆256Updated 2 years ago
• ivan-chai / torch-linear-assignment
  Batch computation of the linear assignment problem on GPU.
  ☆96Updated 2 months ago
• fla-org / flame
  🔥 A minimal training framework for scaling FLA models
  ☆291Updated 2 months ago
• transformer-vq / transformer_vq
  ☆199Updated last year
• meta-pytorch / float8_experimental
  This repository contains the experimental PyTorch native float8 training UX
  ☆223Updated last year
• microsoft / only_train_once
  OTOv1-v3, NeurIPS, ICLR, TMLR, DNN Training, Compression, Structured Pruning, Erasing Operators, CNN, Diffusion, LLM
  ☆48Updated last year
• goombalab / hydra
  Official implementation of "Hydra: Bidirectional State Space Models Through Generalized Matrix Mixers"
  ☆165Updated 9 months ago
• SHI-Labs / NATTEN
  Fast Multi-dimensional Sparse Attention
  ☆654Updated 3 weeks ago
• lucidrains / maskbit-pytorch
  Implementation of the proposed MaskBit from Bytedance AI
  ☆82Updated last year