Alternatives and similar repositories for peeb

Users that are interested in peeb are comparing it to the libraries listed below

• prometheus-eval / prometheus-vision
  [ACL 2024 Findings & ICLR 2024 WS] An Evaluator VLM that is open-source, offers reproducible evaluation, and inexpensive to use. Specific…
  ☆78Updated last year
• jmhb0 / microvqa
  [CVPR 2025] MicroVQA eval and 🤖RefineBot code for "MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based Scientific Research"…
  ☆29Updated 2 weeks ago
• pliang279 / HEMM
  Holistic evaluation of multimodal foundation models
  ☆47Updated last year
• multimodal-interpretability / maia
  Official implementation of MAIA, A Multimodal Automated Interpretability Agent
  ☆99Updated last month
• mbzuai-oryx / ALM-Bench
  [CVPR 2025 🔥] ALM-Bench is a multilingual multi-modal diverse cultural benchmark for 100 languages across 19 categories. It assesses the…
  ☆45Updated 6 months ago
• anguyen8 / vision-llms-are-blind
  ☆139Updated 3 months ago
• Heidelberg-NLP / CC-SHAP-VLM
  Official code implementation for the paper "Do Vision & Language Decoders use Images and Text equally? How Self-consistent are their Expl…
  ☆12Updated 8 months ago
• wrudman / NOTICE
  ☆13Updated 8 months ago
• apple / ml-rpm-bench
  ☆41Updated last year
• google-research / silc
  [ECCV 2024] Official Release of SILC: Improving vision language pretraining with self-distillation
  ☆47Updated last year
• EvolvingLMMs-Lab / multimodal-sae
  [ICCV 2025] Auto Interpretation Pipeline and many other functionalities for Multimodal SAE Analysis.
  ☆166Updated 2 months ago
• nickjiang2378 / vlm-hallucinations
  [ICLR '25] Official Pytorch implementation of "Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations"
  ☆96Updated 2 weeks ago
• Understanding-Visual-Datasets / VisDiff
  Official implementation of "Describing Differences in Image Sets with Natural Language" (CVPR 2024 Oral)
  ☆129Updated last month
• m1k2zoo / negbench
  Evaluation and dataset construction code for the CVPR 2025 paper "Vision-Language Models Do Not Understand Negation"
  ☆42Updated 7 months ago
• ExplainableML / sae-for-vlm
  [NeurIPS 2025] Sparse Autoencoders Learn Monosemantic Features in Vision-Language Models
  ☆49Updated 2 weeks ago
• zeyofu / BLINK_Benchmark
  This repo contains evaluation code for the paper "BLINK: Multimodal Large Language Models Can See but Not Perceive". https://arxiv.or…
  ☆152Updated 2 months ago
• bethgelab / frequency_determines_performance
  Code for the paper: "No Zero-Shot Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance" [NeurI…
  ☆93Updated last year
• jmhb0 / viddiff
  [ICLR 2025] Video Action Differencing
  ☆48Updated 5 months ago
• SriramB-98 / vit-decompose
  ☆23Updated 11 months ago
• facebookresearch / unibench
  Python Library to evaluate VLM models' robustness across diverse benchmarks
  ☆220Updated last month
• yossigandelsman / second_order_lens
  Official pytorch implementation of "Interpreting the Second-Order Effects of Neurons in CLIP"
  ☆42Updated last year
• AtsuMiyai / UPD
  [ACL2025] Unsolvable Problem Detection: Robust Understanding Evaluation for Large Multimodal Models
  ☆79Updated 6 months ago
• paulgavrikov / vlm_shapebias
  Official code for "Can We Talk Models Into Seeing the World Differently?" (ICLR 2025).
  ☆27Updated 10 months ago
• mu-cai / matryoshka-mm
  Matryoshka Multimodal Models
  ☆120Updated 10 months ago
• k1rezaei / Text-to-concept
  ☆35Updated last year
• amitakamath / whatsup_vlms
  Code and datasets for "What’s “up” with vision-language models? Investigating their struggle with spatial reasoning".
  ☆67Updated last year
• AI4LIFE-GROUP / SpLiCE
  Sparse Linear Concept Embeddings
  ☆126Updated 8 months ago
• Picsart-AI-Research / OpenBias
  [CVPR 2024 Highlight] OpenBias: Open-set Bias Detection in Text-to-Image Generative Models
  ☆26Updated 10 months ago
• yuhui-zh15 / AutoConverter
  Official implementation of "Automated Generation of Challenging Multiple-Choice Questions for Vision Language Model Evaluation" (CVPR 202…
  ☆39Updated 6 months ago
• open-compass / ProSA
  [EMNLP 2024 Findings] ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs
  ☆29Updated 6 months ago