RAIVNLab / mnmsLinks
m&ms: A Benchmark to Evaluate Tool-Use for multi-step multi-modal tasks
☆44Updated last year
Alternatives and similar repositories for mnms
Users that are interested in mnms are comparing it to the libraries listed below
Sorting:
- This is the repo for our paper "Mr-Ben: A Comprehensive Meta-Reasoning Benchmark for Large Language Models"☆52Updated last year
- ☆42Updated last year
- Code for Paper: Autonomous Evaluation and Refinement of Digital Agents [COLM 2024]☆147Updated last year
- Trial and Error: Exploration-Based Trajectory Optimization of LLM Agents (ACL 2024 Main Conference)☆159Updated last year
- [NeurIPS 2024 D&B Track] GTA: A Benchmark for General Tool Agents☆132Updated 9 months ago
- ☆53Updated 9 months ago
- Evaluation framework for paper "VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?"☆62Updated last year
- RM-R1: Unleashing the Reasoning Potential of Reward Models☆156Updated 6 months ago
- [ACL2024] Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarios☆66Updated 4 months ago
- [EMNLP 2025] WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learning☆63Updated last month
- [ACL 2024] PCA-Bench: Evaluating Multimodal Large Language Models in Perception-Cognition-Action Chain☆105Updated last year
- ☆105Updated last year
- Improving Language Understanding from Screenshots. Paper: https://arxiv.org/abs/2402.14073☆31Updated last year
- The code and data for the paper JiuZhang3.0☆49Updated last year
- [ACL 2024] The project of Symbol-LLM☆60Updated last year
- A comrephensive collection of learning from rewards in the post-training and test-time scaling of LLMs, with a focus on both reward model…☆60Updated 6 months ago
- [COLING 2025] ToolEyes: Fine-Grained Evaluation for Tool Learning Capabilities of Large Language Models in Real-world Scenarios☆71Updated 7 months ago
- [NeurIPS'24] Weak-to-Strong Search: Align Large Language Models via Searching over Small Language Models☆64Updated last year
- Official Implementation of ARPO: End-to-End Policy Optimization for GUI Agents with Experience Replay☆140Updated 7 months ago
- [ICLR 2024] Trajectory-as-Exemplar Prompting with Memory for Computer Control☆63Updated 11 months ago
- ☆32Updated last year
- ☆31Updated last year
- [ICLR'24 spotlight] Tool-Augmented Reward Modeling☆51Updated 6 months ago
- [ACL 2025] A Neural-Symbolic Self-Training Framework☆117Updated 7 months ago
- ☆100Updated 2 years ago
- An Easy-to-use Hallucination Detection Framework for LLMs.☆62Updated last year
- ☆31Updated last year
- [ICLR 2024] MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use☆101Updated last year
- This repository is maintained to release dataset and models for multimodal puzzle reasoning.☆113Updated 10 months ago
- Official Code of IdealGPT☆35Updated 2 years ago