zzd1992 / FlashWindowAttention
Speedup the attention computation of Swin Transformer
☆13Updated 3 months ago
Alternatives and similar repositories for FlashWindowAttention:
Users that are interested in FlashWindowAttention are comparing it to the libraries listed below
- [CVPR-22] This is the official implementation of the paper "Adavit: Adaptive vision transformers for efficient image recognition".☆51Updated 2 years ago
- ☆50Updated last year
- Implementation of Soft MoE, proposed by Brain's Vision team, in Pytorch☆283Updated 3 weeks ago
- 1.5−3.0× lossless training or pre-training speedup. An off-the-shelf, easy-to-implement algorithm for the efficient training of foundatio…☆221Updated 8 months ago
- PyTorch implementation of "From Sparse to Soft Mixtures of Experts"☆53Updated last year
- Implementation of fused cosine similarity attention in the same style as Flash Attention☆213Updated 2 years ago
- Official code for ICCV 2023 paper "Convolutional Networks with Oriented 1D Kernels"☆47Updated last year
- [CVPR'23] SparseViT: Revisiting Activation Sparsity for Efficient High-Resolution Vision Transformer☆68Updated last year
- (Unofficial) PyTorch implementation of grouped-query attention (GQA) from "GQA: Training Generalized Multi-Query Transformer Models from …☆165Updated 11 months ago
- Recent Advances on Efficient Vision Transformers☆50Updated 2 years ago
- [ICLR 2023] "More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity"; [ICML 2023] "Are Large Kernels Better Teachers…☆267Updated last year
- [NeurIPS 2022 Spotlight] This is the official PyTorch implementation of "EcoFormer: Energy-Saving Attention with Linear Complexity"☆71Updated 2 years ago
- Official implementation of AAAI 2023 paper "Parameter-efficient Model Adaptation for Vision Transformers"☆104Updated last year
- ☆11Updated last year
- [ICCV 23]An approach to enhance the efficiency of Vision Transformer (ViT) by concurrently employing token pruning and token merging tech…☆94Updated last year
- Simple CIFAR-10 classification with ConvMixer☆43Updated 3 years ago
- [ECCV 2024] Isomorphic Pruning for Vision Models☆68Updated 9 months ago
- VIT inference in triton because, why not?☆27Updated 10 months ago
- Supercharge Your PyTorch Image Models: Bag of Tricks to 8x Faster Inference with ONNX Runtime & Optimizations☆23Updated 6 months ago
- [ICME 2022] code for the paper, SimVit: Exploring a simple vision transformer with sliding windows.☆68Updated 2 years ago
- [ECCV2024] Official implementation of paper, "DenseNets Reloaded: Paradigm Shift Beyond ResNets and ViTs".☆146Updated 8 months ago
- [ECCV 2024] Official PyTorch implementation of RoPE-ViT "Rotary Position Embedding for Vision Transformer"☆316Updated 4 months ago
- [CVPR 2023] Castling-ViT: Compressing Self-Attention via Switching Towards Linear-Angular Attention During Vision Transformer Inference☆30Updated last year
- ☆102Updated last year
- Implementation of Infini-Transformer in Pytorch☆110Updated 3 months ago
- PyTorch implementation of R-MAE https//arxiv.org/abs/2306.05411☆113Updated last year
- PyTorch code and checkpoints release for VanillaKD: https://arxiv.org/abs/2305.15781☆74Updated last year
- code release of research paper "Exploring Long-Sequence Masked Autoencoders"☆100Updated 2 years ago
- An efficient pytorch implementation of selective scan in one file, works with both cpu and gpu, with corresponding mathematical derivatio…☆85Updated last year
- A simple minimal implementation of Reversible Vision Transformers☆124Updated last year