visual-haystacks / vhs_benchmarkLinks
🔥 [ICLR 2025] Official Benchmark Toolkits for "Visual Haystacks: A Vision-Centric Needle-In-A-Haystack Benchmark"
☆29Updated 5 months ago
Alternatives and similar repositories for vhs_benchmark
Users that are interested in vhs_benchmark are comparing it to the libraries listed below
Sorting:
- Github repository for "Bring Reason to Vision: Understanding Perception and Reasoning through Model Merging" (ICML 2025)☆68Updated 2 months ago
- This repo contains evaluation code for the paper "BLINK: Multimodal Large Language Models Can See but Not Perceive". https://arxiv.or…☆133Updated last year
- Enhancing Large Vision Language Models with Self-Training on Image Comprehension.☆69Updated last year
- [Arxiv] Aligning Modalities in Vision Large Language Models via Preference Fine-tuning☆86Updated last year
- [NeurIPS 2024] Calibrated Self-Rewarding Vision Language Models☆77Updated last year
- 🔥 [ICLR 2025] Official PyTorch Model "Visual Haystacks: A Vision-Centric Needle-In-A-Haystack Benchmark"☆17Updated 5 months ago
- [EMNLP 2024] mDPO: Conditional Preference Optimization for Multimodal Large Language Models.☆79Updated 8 months ago
- [NeurIPS 2023] A faithful benchmark for vision-language compositionality☆82Updated last year
- VLM Evaluation: Benchmark for VLMs, spanning text generation tasks from VQA to Captioning☆120Updated 10 months ago
- Official implementation of the Law of Vision Representation in MLLMs☆163Updated 8 months ago
- code for "Strengthening Multimodal Large Language Model with Bootstrapped Preference Optimization"☆57Updated 11 months ago
- ☆100Updated last year
- Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization☆91Updated last year
- Official implementation of "Connect, Collapse, Corrupt: Learning Cross-Modal Tasks with Uni-Modal Data" (ICLR 2024)☆33Updated 9 months ago
- Official implementation of "Automated Generation of Challenging Multiple-Choice Questions for Vision Language Model Evaluation" (CVPR 202…☆32Updated 2 months ago
- [TACL'23] VSR: A probing benchmark for spatial undersranding of vision-language models.☆127Updated 2 years ago
- [ICML 2024 Oral] Official code repository for MLLM-as-a-Judge.☆77Updated 5 months ago
- [NeurIPS2024] Official code for (IMA) Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputs☆20Updated 9 months ago
- ☆95Updated 4 months ago
- Official Pytorch implementation of "Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations" (ICLR '25)☆76Updated 2 months ago
- [ACM Multimedia 2025] This is the official repo for Debiasing Large Visual Language Models, including a Post-Hoc debias method and Visual…☆81Updated 5 months ago
- Python Library to evaluate VLM models' robustness across diverse benchmarks☆210Updated this week
- The official code of "VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning"☆134Updated 2 months ago
- VideoHallucer, The first comprehensive benchmark for hallucination detection in large video-language models (LVLMs)☆36Updated 4 months ago
- Matryoshka Multimodal Models☆112Updated 6 months ago
- ☆152Updated 9 months ago
- ☆50Updated last year
- VLM2-Bench [ACL 2025 Main]: A Closer Look at How Well VLMs Implicitly Link Explicit Matching Visual Cues☆41Updated 2 months ago
- Code and datasets for "What’s “up” with vision-language models? Investigating their struggle with spatial reasoning".☆56Updated last year
- [TMLR] Public code repo for paper "A Single Transformer for Scalable Vision-Language Modeling"☆144Updated 8 months ago