zzd1992 / FlashWindowAttention
Speedup the attention computation of Swin Transformer
☆10Updated 2 months ago
Alternatives and similar repositories for FlashWindowAttention:
Users that are interested in FlashWindowAttention are comparing it to the libraries listed below
- PyTorch code and checkpoints release for VanillaKD: https://arxiv.org/abs/2305.15781☆75Updated last year
- [ICLR 2022] "As-ViT: Auto-scaling Vision Transformers without Training" by Wuyang Chen, Wei Huang, Xianzhi Du, Xiaodan Song, Zhangyang Wa…☆76Updated 3 years ago
- ☆50Updated last year
- [NeurIPS 2022 Spotlight] This is the official PyTorch implementation of "EcoFormer: Energy-Saving Attention with Linear Complexity"☆71Updated 2 years ago
- PyTorch implementation of SSQL (Accepted to ECCV2022 oral presentation)☆75Updated 2 years ago
- [ECCV 2024] Isomorphic Pruning for Vision Models☆66Updated 8 months ago
- [EMNLP 2022] Official implementation of Transnormer in our EMNLP 2022 paper - The Devil in Linear Transformer☆60Updated last year
- [CVPR-22] This is the official implementation of the paper "Adavit: Adaptive vision transformers for efficient image recognition".☆52Updated 2 years ago
- Official implementation for "SimA: Simple Softmax-free Attention for Vision Transformers"☆43Updated 11 months ago
- ☆179Updated 6 months ago
- ☆102Updated last year
- Implementation of fused cosine similarity attention in the same style as Flash Attention☆212Updated 2 years ago
- FastMIM, official pytorch implementation of our paper "FastMIM: Expediting Masked Image Modeling Pre-training for Vision"(https://arxiv.o…☆39Updated 2 years ago
- [NeurIPS'21] "Chasing Sparsity in Vision Transformers: An End-to-End Exploration" by Tianlong Chen, Yu Cheng, Zhe Gan, Lu Yuan, Lei Zhang…☆89Updated last year
- (CVPR 2022) Automated Progressive Learning for Efficient Training of Vision Transformers☆25Updated last month
- code for NASViT☆68Updated 2 years ago
- ☆71Updated 3 weeks ago
- Official implementation of AAAI 2023 paper "Parameter-efficient Model Adaptation for Vision Transformers"☆104Updated last year
- PyTorch implementation of "From Sparse to Soft Mixtures of Experts"☆53Updated last year
- [ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Mod…☆35Updated last year
- [ICML 2022] "DepthShrinker: A New Compression Paradigm Towards Boosting Real-Hardware Efficiency of Compact Neural Networks", by Yonggan …☆71Updated 2 years ago
- A library for calculating the FLOPs in the forward() process based on torch.fx☆100Updated 6 months ago
- Official codes for ConMIM (ICLR 2023)☆58Updated 2 years ago
- [ICLR2024] Exploring Target Representations for Masked Autoencoders☆53Updated last year
- A simple minimal implementation of Reversible Vision Transformers☆123Updated last year
- (Unofficial) PyTorch implementation of the paper Early Convolutions Help Transformers See Better☆43Updated 3 years ago
- Adaptive Token Sampling for Efficient Vision Transformers (ECCV 2022 Oral Presentation)☆99Updated 10 months ago
- An efficient pytorch implementation of selective scan in one file, works with both cpu and gpu, with corresponding mathematical derivatio…☆81Updated last year
- [CVPR'23] SparseViT: Revisiting Activation Sparsity for Efficient High-Resolution Vision Transformer☆67Updated 11 months ago
- [ICLR 2022] "Unified Vision Transformer Compression" by Shixing Yu*, Tianlong Chen*, Jiayi Shen, Huan Yuan, Jianchao Tan, Sen Yang, Ji Li…☆52Updated last year