google-deepmind / multimodal_transformers

Related projects ⓘ

Alternatives and complementary repositories for multimodal_transformers

• multimodal / multimodal
  A collection of multimodal datasets, and visual features for VQA and captionning in pytorch. Just run "pip install multimodal"
  ☆79Updated 2 years ago
• IntelLabs / VL-InterpreT
  Visual Language Transformer Interpreter - An interactive visualization tool for interpreting vision-language transformers
  ☆86Updated last year
• fawazsammani / nlxgpt
  NLX-GPT: A Model for Natural Language Explanations in Vision and Vision-Language Tasks, CVPR 2022 (Oral)
  ☆44Updated 9 months ago
• allenai / sherlock
  Code, data, models for the Sherlock corpus
  ☆55Updated last year
• McGill-NLP / imagecode
  Code and data for ImageCoDe, a contextual vison-and-language benchmark
  ☆39Updated 8 months ago
• salesforce / VD-BERT
  ☆44Updated 2 years ago
• intersun / LightningDOT
  source code and pre-trained/fine-tuned checkpoint for NAACL 2021 paper LightningDOT
  ☆73Updated last year
• facebookresearch / reliable_vqa
  Implementation for the paper "Reliable Visual Question Answering Abstain Rather Than Answer Incorrectly" (ECCV 2022: https//arxiv.org/abs…
  ☆32Updated last year
• goel-shashank / CyCLIP
  ☆113Updated last year
• pliang279 / HighMMT
  [TMLR 2022] High-Modality Multimodal Transformer
  ☆107Updated last week
• zinengtang / Perceiver_VL
  PyTorch code for "Perceiver-VL: Efficient Vision-and-Language Modeling with Iterative Latent Attention" (WACV 2023)
  ☆32Updated last year
• Weixin-Liang / MetaShift
  MetaShift: A Dataset of Datasets for Evaluating Contextual Distribution Shifts and Training Conflicts (ICLR 2022)
  ☆108Updated 2 years ago
• rowanz / merlot
  MERLOT: Multimodal Neural Script Knowledge Models
  ☆223Updated 2 years ago
• LuoweiZhou / coco-caption
  kdexd/coco-caption@de6f385
  ☆26Updated 4 years ago
• google-deepmind / brave
  A JAX implementation of Broaden Your Views for Self-Supervised Video Learning, or BraVe for short.
  ☆48Updated 4 months ago
• e-bug / iglue
  [ICML 2022] Code and data for our paper "IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages"
  ☆49Updated last year
• BierOne / Attention-Faithfulness
  [ICML 2022] This is the pytorch implementation of "Rethinking Attention-Model Explainability through Faithfulness Violation Test" (https:…
  ☆18Updated 2 years ago
• ExplainableML / CLEVR-X
  CLEVR-X: A Visual Reasoning Dataset for Natural Language Explanations
  ☆25Updated last year
• changdaeoh / multimodal-mixup
  Official implementation for NeurIPS'23 paper "Geodesic Multi-Modal Mixup for Robust Fine-Tuning"
  ☆28Updated last month
• zhjohnchan / awesome-vision-and-language-pretraining
  A curated list of vision-and-language pre-training (VLP). :-)
  ☆56Updated 2 years ago
• google-research / fnc
  ☆25Updated 3 years ago
• e-bug / cross-modal-ablation
  [EMNLP 2021] Code and data for our paper "Vision-and-Language or Vision-for-Language? On Cross-Modal Influence in Multimodal Transformers…
  ☆20Updated 2 years ago
• gchhablani / multilingual-vqa
  Repository for Multilingual-VQA task created during HuggingFace JAX/Flax community week.
  ☆34Updated 3 years ago
• bearcatt / LaBERT
  A length-controllable and non-autoregressive image captioning model.
  ☆66Updated 3 years ago
• HendrikStrobelt / miniClip
  ☆47Updated 3 years ago
• facebookresearch / data2vec_vision
  Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
  ☆78Updated 2 years ago
• YuanEZhou / satic
  ☆25Updated 3 years ago
• redcaps-dataset / redcaps-downloader
  Command-line tool for downloading and extending the RedCaps dataset.
  ☆45Updated 10 months ago
• guilk / KAT
  Research code for "KAT: A Knowledge Augmented Transformer for Vision-and-Language"
  ☆60Updated 2 years ago
• MicPie / clasp
  CLASP - Contrastive Language-Aminoacid Sequence Pretraining
  ☆142Updated 3 years ago