RaptorMai / MLLM-CompBench
[NeurIPS'25] MLLM-CompBench evaluates the comparative reasoning of MLLMs with 40K image pairs and questions across 8 dimensions of relative comparison: visual attribute, existence, state, emotion, temporality, spatiality, quantity, and quality. CompBench covers diverse visual domains, including animals, fashion, sports, and scenes
☆38Updated 2 weeks ago
Alternatives and similar repositories for MLLM-CompBench:
Users that are interested in MLLM-CompBench are comparing it to the libraries listed below
- ☆26Updated 2 years ago
- This repo implements the CVPR23 paper Trainable Projected Gradient Method for Robust Fine-tuning☆24Updated last year
- PyTorch implementation of POEM (Out-of-distribution detection with posterior sampling), ICML 2022☆28Updated 2 years ago
- Test-Time Adaptation via Conjugate Pseudo-Labels☆40Updated last year
- Code for Continuously Changing Corruptions (CCC) benchmark + evaluation☆35Updated 8 months ago
- Code for Debiasing Vision-Language Models via Biased Prompts☆57Updated last year
- [NeurIPS21] TTT++: When Does Self-supervised Test-time Training Fail or Thrive?☆67Updated 3 years ago
- ☆39Updated 2 years ago
- ☆22Updated 11 months ago
- Official Code for NeurIPS 2022 Paper: How Mask Matters: Towards Theoretical Understandings of Masked Autoencoders☆58Updated last year
- ☆18Updated 2 years ago
- LANCE: Stress-testing Visual Models by Generating Language-guided Counterfactual Images☆32Updated last year
- Metrics for "Beyond neural scaling laws: beating power law scaling via data pruning " (NeurIPS 2022 Outstanding Paper Award)☆56Updated 2 years ago
- This is the official PyTorch Implementation of "SoTTA: Robust Test-Time Adaptation on Noisy Data Streams (NeurIPS '23)" by Taesik Gong*, …☆21Updated last year
- Implementation for ECCV 2022 Paper "Class Is Invariant to Context and Vice Versa: On Learning Invariance for Out-Of-Distribution Generali…☆19Updated 2 years ago
- Official repository for the ICCV 2023 paper: "Waffling around for Performance: Visual Classification with Random Words and Broad Concepts…☆57Updated last year
- ☆41Updated last year
- [NeurIPS '24] Frustratingly easy Test-Time Adaptation of VLMs!!☆44Updated last month
- This repository is the official implementation of Dataset Condensation with Contrastive Signals (DCC), accepted at ICML 2022.☆21Updated 2 years ago
- ☆16Updated 2 years ago
- Create generated datasets and train robust classifiers☆35Updated last year
- Code for NeurIPS 2021 paper "Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learning".☆16Updated 3 years ago
- ☆43Updated last year
- ☆56Updated 4 months ago
- ☆63Updated last year
- PyTorch implementation of CIDER (How to exploit hyperspherical embeddings for out-of-distribution detection), ICLR 2023☆60Updated last year
- ☆48Updated 2 years ago
- [CVPR2022] PyTorch re-implementation of Prompt Distribution Learning☆18Updated 2 years ago
- [CVPR23] "Understanding and Improving Visual Prompting: A Label-Mapping Perspective" by Aochuan Chen, Yuguang Yao, Pin-Yu Chen, Yihua Zha…☆53Updated last year
- ☆86Updated 2 years ago