pliang279 / HighMMT
[TMLR 2022] High-Modality Multimodal Transformer
☆107Updated 2 weeks ago
Related projects ⓘ
Alternatives and complementary repositories for HighMMT
- Multimodal Masked Autoencoders (M3AE): A JAX/Flax Implementation☆100Updated 7 months ago
- [NeurIPS 2023] Factorized Contrastive Learning: Going Beyond Multi-view Redundancy☆60Updated last year
- Official implementation of "Everything at Once - Multi-modal Fusion Transformer for Video Retrieval". CVPR 2022☆95Updated 2 years ago
- [ICLR 2023] MultiViz: Towards Visualizing and Understanding Multimodal Models☆91Updated 2 months ago
- Implementation of Zorro, Masked Multimodal Transformer, in Pytorch☆95Updated last year
- [NeurIPS 2023, ICMI 2023] Quantifying & Modeling Multimodal Interactions☆64Updated 3 weeks ago
- Official Implementation of "Geometric Multimodal Contrastive Representation Learning" (https://arxiv.org/abs/2202.03390)☆26Updated 2 years ago
- Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning☆130Updated 2 years ago
- ☆22Updated 2 years ago
- The Continual Learning in Multimodality Benchmark☆61Updated last year
- ☆52Updated 2 years ago
- CrossCLR: Cross-modal Contrastive Learning For Multi-modal Video Representations, ICCV 2021☆57Updated 2 years ago
- Visual Language Transformer Interpreter - An interactive visualization tool for interpreting vision-language transformers☆86Updated last year
- ☆113Updated last year
- PyTorch implementation of LIMoE☆51Updated 7 months ago
- A PyTorch implementation of Multimodal Few-Shot Learning with Frozen Language Models with OPT.☆43Updated 2 years ago
- ☆25Updated 3 years ago
- Implementation for the paper "Reliable Visual Question Answering Abstain Rather Than Answer Incorrectly" (ECCV 2022: https//arxiv.org/abs…☆32Updated last year
- PyTorch code for “TVLT: Textless Vision-Language Transformer” (NeurIPS 2022 Oral)☆120Updated last year
- PyTorch code for "VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks" (CVPR2022)☆202Updated last year
- Implementation of MaMMUT, a simple vision-encoder text-decoder architecture for multimodal tasks from Google, in Pytorch☆97Updated last year
- ViLLA: Fine-grained vision-language representation learning from real-world data☆40Updated last year
- [ICLR2024] The official implementation of paper "UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modeling", by …☆69Updated 9 months ago
- Code Example for Learning Multimodal Data Augmentation in Feature Space☆40Updated last year
- [CVPR 2024] MMSum: A Dataset for Multimodal Summarization and Thumbnail Generation of Videos☆30Updated 7 months ago
- code for "Multitask Vision-Language Prompt Tuning" https://arxiv.org/abs/2211.11720☆54Updated 5 months ago
- A curated list of vision-and-language pre-training (VLP). :-)☆56Updated 2 years ago
- Official code for our CVPR 2023 paper: Test of Time: Instilling Video-Language Models with a Sense of Time☆45Updated 5 months ago
- PyTorch Implementation on Paper [CVPR2021]Distilling Audio-Visual Knowledge by Compositional Contrastive Learning☆86Updated 3 years ago
- Official Code Release for "Diagnosing and Rectifying Vision Models using Language" (ICLR 2023)☆32Updated last year