robflynnyh / hydra-linear-attention
Implementation of: Hydra Attention: Efficient Attention with Many Heads (https://arxiv.org/abs/2209.07484)
☆13Updated 2 years ago
Alternatives and similar repositories for hydra-linear-attention:
Users that are interested in hydra-linear-attention are comparing it to the libraries listed below
- [CVPR 2023] Castling-ViT: Compressing Self-Attention via Switching Towards Linear-Angular Attention During Vision Transformer Inference☆29Updated last year
- GroupMamba: Parameter-Efficient and Accurate Group Visual State Space Model [CVPR -2025]☆76Updated this week
- Official implementation of CVPR 2024 paper "Multi-criteria Token Fusion with One-step-ahead Attention for Efficient Vision Transformers".☆34Updated 11 months ago
- Ofiicial Implementation for Mamba-ND: Selective State Space Modeling for Multi-Dimensional Data☆59Updated 8 months ago
- [CVPR'24] Multimodal Pathway: Improve Transformers with Irrelevant Data from Other Modalities☆99Updated last year
- [CVPR'23] Hard Patches Mining for Masked Image Modeling☆90Updated last year
- ☆43Updated 2 years ago
- [NeurIPS2024 Spotlight] The official implementation of MambaTree: Tree Topology is All You Need in State Space Model☆92Updated 9 months ago
- (AAAI 2023 Oral) Pytorch implementation of "CF-ViT: A General Coarse-to-Fine Method for Vision Transformer"☆103Updated last year
- ☆22Updated 3 years ago
- This repository is the official implementation of our Autoregressive Pretraining with Mamba in Vision☆71Updated 9 months ago
- Official repository of Slide-Transformer (CVPR2023)☆167Updated 7 months ago
- ☆62Updated last month
- open source the research work for published on arxiv. https://arxiv.org/abs/2106.02689☆51Updated 3 years ago
- ☆59Updated last year
- An efficient pytorch implementation of selective scan in one file, works with both cpu and gpu, with corresponding mathematical derivatio…☆80Updated last year
- ☆83Updated last year
- A Close Look at Spatial Modeling: From Attention to Convolution☆91Updated 2 years ago
- [ICLR 2022] "Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practice" by Peihao Wang, Wen…☆80Updated last year
- ☆136Updated 9 months ago
- [BMVC 2024] PlainMamba: Improving Non-hierarchical Mamba in Visual Recognition☆74Updated 7 months ago
- More dimensions = More fun☆21Updated 8 months ago
- ☆47Updated 11 months ago
- [ICLR 2023] Masked Frequency Modeling for Self-Supervised Visual Pre-Training☆74Updated last year
- This is code of paper "ScalableViT: Rethinking the Context-oriented Generalization of Vision Transformer"☆26Updated last year
- ☆54Updated last year
- [AAAI 2024] AVSegFormer: Audio-Visual Segmentation with Transformer☆63Updated 3 weeks ago
- Pytorch Implementation for CVPR 2024 paper: Learn to Rectify the Bias of CLIP for Unsupervised Semantic Segmentation☆36Updated last month
- PyTorch Implementation of "Your ViT is Secretly a Hybrid Discriminative-Generative Diffusion Model"☆48Updated 2 years ago
- An official code release of the paper RGB no more: Minimally Decoded JPEG Vision Transformers☆56Updated last year