zinengtang / TVLT
PyTorch code for “TVLT: Textless Vision-Language Transformer” (NeurIPS 2022 Oral)
☆123Updated 2 years ago
Alternatives and similar repositories for TVLT:
Users that are interested in TVLT are comparing it to the libraries listed below
- Align and Prompt: Video-and-Language Pre-training with Entity Prompts☆185Updated 2 years ago
- Code release for "MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound"☆139Updated 2 years ago
- [ACL 2023] Official PyTorch code for Singularity model in "Revealing Single Frame Bias for Video-and-Language Learning"☆132Updated last year
- ☆108Updated 2 years ago
- A PyTorch implementation of EmpiricalMVM☆40Updated last year
- ☆54Updated 2 years ago
- An official implementation for "X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval"☆152Updated 11 months ago
- MUSIC-AVQA, CVPR2022 (ORAL)☆78Updated 2 years ago
- https://layer6ai-labs.github.io/xpool/☆120Updated last year
- Official implementation of "Everything at Once - Multi-modal Fusion Transformer for Video Retrieval". CVPR 2022☆101Updated 2 years ago
- Summary about Video-to-Text datasets. This repository is part of the review paper *Bridging Vision and Language from the Video-to-Text Pe…☆121Updated last year
- multimodal video-audio-text generation and retrieval between every pair of modalities on the MUGEN dataset. The repo. contains the traini…☆39Updated last year
- [NeurIPS 2023] Self-Chained Image-Language Model for Video Localization and Question Answering☆187Updated last year
- [SIGIR 2022] CenterCLIP: Token Clustering for Efficient Text-Video Retrieval. Also, a text-video retrieval toolbox based on CLIP + fast p…☆129Updated 2 years ago
- Hierarchical Video-Moment Retrieval and Step-Captioning (CVPR 2023)☆100Updated last month
- [CVPR'23 Highlight] AutoAD: Movie Description in Context.☆94Updated 4 months ago
- [arXiv22] Disentangled Representation Learning for Text-Video Retrieval☆96Updated 2 years ago
- ☆75Updated 2 years ago
- PyTorch Implementation on Paper [CVPR2021]Distilling Audio-Visual Knowledge by Compositional Contrastive Learning☆84Updated 3 years ago
- Code for the AVLnet (Interspeech 2021) and Cascaded Multilingual (Interspeech 2021) papers.☆50Updated 2 years ago
- MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptions☆159Updated last year
- Use CLIP to represent video for Retrieval Task☆69Updated 4 years ago
- Vision Transformers are Parameter-Efficient Audio-Visual Learners☆99Updated last year
- [NeurIPS 2022] Zero-Shot Video Question Answering via Frozen Bidirectional Language Models☆156Updated 3 months ago
- [TPAMI2024] Codes and Models for VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset☆281Updated 2 months ago
- Pytorch code for Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners☆114Updated 2 years ago
- A Unified Framework for Video-Language Understanding☆57Updated last year
- ACAV100M: Automatic Curation of Large-Scale Datasets for Audio-Visual Video Representation Learning. In ICCV, 2021.☆56Updated 3 years ago
- Code on selecting an action based on multimodal inputs. Here in this case inputs are voice and text.☆70Updated 3 years ago
- Official code for "Bridging Video-text Retrieval with Multiple Choice Questions", CVPR 2022 (Oral).☆139Updated 2 years ago