mbzuai-oryx / ALM-BenchLinks
[CVPR 2025 π₯] ALM-Bench is a multilingual multi-modal diverse cultural benchmark for 100 languages across 19 categories. It assesses the next generation of LMMs on cultural inclusitivity.
β46Updated 4 months ago
Alternatives and similar repositories for ALM-Bench
Users that are interested in ALM-Bench are comparing it to the libraries listed below
Sorting:
- (WACV 2025 - Oral) Vision-language conversation in 10 languages including English, Chinese, French, Spanish, Russian, Japanese, Arabic, Hβ¦β82Updated 2 months ago
- [ACL2025] Unsolvable Problem Detection: Robust Understanding Evaluation for Large Multimodal Modelsβ78Updated 4 months ago
- Holistic evaluation of multimodal foundation modelsβ48Updated last year
- [ECCV 2024] Official Release of SILC: Improving vision language pretraining with self-distillationβ45Updated last year
- β41Updated last year
- β53Updated 8 months ago
- Matryoshka Multimodal Modelsβ111Updated 8 months ago
- Evaluation and dataset construction code for the CVPR 2025 paper "Vision-Language Models Do Not Understand Negation"β34Updated 5 months ago
- β35Updated last year
- Official Pytorch implementation of "Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations" (ICLR '25)β86Updated 4 months ago
- Code for the paper: "No Zero-Shot Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance" [NeurIβ¦β90Updated last year
- [ACL 2024 Findings & ICLR 2024 WS] An Evaluator VLM that is open-source, offers reproducible evaluation, and inexpensive to use. Specificβ¦β77Updated last year
- β87Updated last year
- [ICCV 2025] Auto Interpretation Pipeline and many other functionalities for Multimodal SAE Analysis.β155Updated 2 weeks ago
- How Good is Google Bard's Visual Understanding? An Empirical Study on Open Challengesβ30Updated 2 years ago
- Code for "Enhancing In-context Learning via Linear Probe Calibration"β36Updated last year
- Official implementation of "Describing Differences in Image Sets with Natural Language" (CVPR 2024 Oral)β121Updated last year
- [Technical Report] Official PyTorch implementation code for realizing the technical part of Phantom of Latent representing equipped with β¦β61Updated last year
- β17Updated 3 weeks ago
- This repo contains evaluation code for the paper "BLINK: Multimodal Large Language Models Can See but Not Perceive". https://arxiv.orβ¦β141Updated last week
- Python Library to evaluate VLM models' robustness across diverse benchmarksβ215Updated this week
- Do Vision and Language Models Share Concepts? A Vector Space Alignment Studyβ16Updated 10 months ago
- Official implementation and dataset for the NAACL 2024 paper "ComCLIP: Training-Free Compositional Image and Text Matching"β35Updated last year
- ViLMA: A Zero-Shot Benchmark for Linguistic and Temporal Grounding in Video-Language Models (ICLR 2024, Official Implementation)β16Updated last year
- Official implementation of MAIA, A Multimodal Automated Interpretability Agentβ91Updated 3 months ago
- Repository for the paper: "TiC-CLIP: Continual Training of CLIP Models" ICLR 2024β105Updated last year
- An open source implementation of CLIP (With TULIP Support)β162Updated 4 months ago
- β134Updated last month
- β79Updated 11 months ago
- [CVPR 2024 Highlight] OpenBias: Open-set Bias Detection in Text-to-Image Generative Modelsβ25Updated 7 months ago