Alternatives and similar repositories for vlm-evaluation

Users that are interested in vlm-evaluation are comparing it to the libraries listed below

• bronyayang / Law_of_Vision_Representation_in_MLLMs
  [COLM'25] Official implementation of the Law of Vision Representation in MLLMs
  ☆176Updated 3 months ago
• opendatalab / HA-DPO
  Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization
  ☆100Updated 2 years ago
• TideDra / VL-RLHF
  A RLHF Infrastructure for Vision-Language Models
  ☆193Updated last year
• mu-cai / matryoshka-mm
  Matryoshka Multimodal Models
  ☆121Updated last year
• RifleZhang / LLaVA-Hound-DPO
  ☆155Updated last year
• YiyangZhou / POVID
  [Arxiv] Aligning Modalities in Vision Large Language Models via Preference Fine-tuning
  ☆90Updated last year
• tianyi-lab / HallusionBench
  [CVPR'24] HallusionBench: You See What You Think? Or You Think What You See? An Image-Context Reasoning Benchmark Challenging for GPT-4V(…
  ☆322Updated 3 months ago
• facebookresearch / unibench
  Python Library to evaluate VLM models' robustness across diverse benchmarks
  ☆220Updated 3 months ago
• allenai / unified-io-2.pytorch
  ☆80Updated last year
• junyangwang0410 / AMBER
  An LLM-free Multi-dimensional Benchmark for Multi-modal Hallucination Evaluation
  ☆153Updated 2 years ago
• UCSC-VLAA / VLAA-Thinking
  [TMLR 25] SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models
  ☆147Updated 3 months ago
• zzxslp / SoM-LLaVA
  [COLM-2024] List Items One by One: A New Data Source and Learning Paradigm for Multimodal LLMs
  ☆145Updated last year
• mertyg / vision-language-models-are-bows
  Experiments and data for the paper "When and why vision-language models behave like bags-of-words, and what to do about it?" Oral @ ICLR …
  ☆291Updated 2 years ago
• clemneo / llava-interp
  ☆79Updated last year
• vlf-silkie / VLFeedback
  ☆102Updated 2 years ago
• tsb0601 / MMVP
  ☆359Updated 2 years ago
• RifleZhang / LLaVA-Reasoner-DPO
  ☆110Updated last year
• OpenGVLab / MMT-Bench
  [ICML 2024] | MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI
  ☆116Updated last year
• FreedomIntelligence / ALLaVA
  Harnessing 1.4M GPT4V-synthesized Data for A Lite Vision-Language Model
  ☆280Updated last year
• hendryx-scale / mhal-detect
  M-HalDetect Dataset Release
  ☆26Updated 2 years ago
• zeyofu / BLINK_Benchmark
  This repo contains evaluation code for the paper "BLINK: Multimodal Large Language Models Can See but Not Perceive". https://arxiv.or…
  ☆159Updated 4 months ago
• RAIVNLab / sugar-crepe
  [NeurIPS 2023] A faithful benchmark for vision-language compositionality
  ☆89Updated last year
• yuweihao / MM-Vet
  MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities (ICML 2024)
  ☆320Updated last year
• MMStar-Benchmark / MMStar
  [NeurIPS 2024] This repo contains evaluation code for the paper "Are We on the Right Way for Evaluating Large Vision-Language Models"
  ☆203Updated last year
• TIGER-AI-Lab / Mantis
  Official code for Paper "Mantis: Multi-Image Instruction Tuning" [TMLR 2024 Best Paper]
  ☆237Updated 3 weeks ago
• AoiDragon / POPE
  [EMNLP'23] The official GitHub page for ''Evaluating Object Hallucination in Large Vision-Language Models''
  ☆104Updated 5 months ago
• RUCAIBox / POPE
  The official GitHub page for ''Evaluating Object Hallucination in Large Vision-Language Models''
  ☆244Updated 5 months ago
• yfzhang114 / LLaVA-Align
  [ACM Multimedia 2025] This is the official repo for Debiasing Large Visual Language Models, including a Post-Hoc debias method and Visual…
  ☆82Updated 11 months ago
• mlfoundations / VisIT-Bench
  ☆50Updated 2 years ago
• FreedomIntelligence / MLLM-Bench
  MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria
  ☆72Updated last year