LukasHedegaard / continual-transformers
Official Pytorch Implementation for "Continual Transformers: Redundancy-Free Attention for Online Inference" [ICLR 2023]
☆28Updated last year
Related projects ⓘ
Alternatives and complementary repositories for continual-transformers
- A Python library for Continual Inference Networks in PyTorch☆49Updated last year
- Unofficial PyTorch implementation of the paper "cosFormer: Rethinking Softmax In Attention".☆43Updated 3 years ago
- Code for the paper PermuteFormer☆42Updated 3 years ago
- STABILIZING GRADIENTS FOR DEEP NEURAL NETWORKS VIA EFFICIENT SVD PARAMETERIZATION☆16Updated 6 years ago
- Implementation of "compositional attention" from MILA, a multi-head attention variant that is reframed as a two-step attention process wi…☆50Updated 2 years ago
- Implementation of Memformer, a Memory-augmented Transformer, in Pytorch☆106Updated 4 years ago
- Curse-of-memory phenomenon of RNNs in sequence modelling☆19Updated last week
- [AAAI 2023 (Oral)] CrissCross: Self-Supervised Audio-Visual Representation Learning with Relaxed Cross-Modal Synchronicity☆23Updated last year
- Implementation of Multistream Transformers in Pytorch☆53Updated 3 years ago
- A PyTorch implementation of SimSiam based on CVPR 2021 paper "Exploring Simple Siamese Representation Learning"☆10Updated 3 years ago
- VIsually-Pivoted Audio and(N) Text☆21Updated 2 years ago
- Code for the paper: Audio-Visual Model Distillation Using Acoustic Images☆20Updated last year
- Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding☆42Updated last month
- Implementation of Cross Transformer for spatially-aware few-shot transfer, in Pytorch☆51Updated 3 years ago
- ☆31Updated 3 years ago
- ☆72Updated 3 years ago
- Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr\"om Method (NeurIPS 2021)☆59Updated 2 years ago
- A Pytorch Implementations for Various Vector Quantization Methods☆27Updated 3 years ago
- custom cuda kernel for {2, 3}d relative attention with pytorch wrapper☆43Updated 4 years ago
- This repository contains the code for our CVPR 2022 paper on "Audio-visual Generalised Zero-shot Learning with Cross-modal Attention and …☆34Updated last year
- [ICCV'21] The Right to Talk: An Audio-Visual Transformer Approach☆20Updated 3 years ago
- The Pytorch implementation of paper: Masked Spectrogram Prediction For Self-Supervised Audio Pre-Training☆39Updated last year
- Code for the C2KD paper (ICASSP 2023)☆16Updated last year
- This repository contains the code for our ECCV 2022 paper "Temporal and cross-modal attention for audio-visual zero-shot learning"☆24Updated last year
- [ICLR 2023] “ Layer Grafted Pre-training: Bridging Contrastive Learning And Masked Image Modeling For Better Representations”, Ziyu Jian…☆23Updated last year
- Implementation of OmniNet, Omnidirectional Representations from Transformers, in Pytorch☆56Updated 3 years ago
- A variant of Transformer-XL where the memory is updated not with a queue, but with attention☆46Updated 4 years ago
- Code for Discriminative Sounding Objects Localization (NeurIPS 2020)☆58Updated 2 years ago
- Sapsucker Woods 60 Audiovisual Dataset☆14Updated 2 years ago
- Pytorch implementation of Performer from the paper "Rethinking Attention with Performers".☆23Updated 4 years ago