RAIVNLab / mnms
m&ms: A Benchmark to Evaluate Tool-Use for multi-step multi-modal tasks
☆39Updated 7 months ago
Alternatives and similar repositories for mnms:
Users that are interested in mnms are comparing it to the libraries listed below
- ☆40Updated 5 months ago
- [NeurIPS 2024] A task generation and model evaluation system for multimodal language models.☆70Updated 5 months ago
- This is the repo for our paper "Mr-Ben: A Comprehensive Meta-Reasoning Benchmark for Large Language Models"☆49Updated 6 months ago
- Improving Language Understanding from Screenshots. Paper: https://arxiv.org/abs/2402.14073☆28Updated 10 months ago
- [NAACL 2025] The official implementation of paper "Learning From Failure: Integrating Negative Examples when Fine-tuning Large Language M…☆26Updated last year
- [ACL2024] Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarios☆56Updated last year
- ☆59Updated 8 months ago
- ☆29Updated 7 months ago
- [ICLR'24 spotlight] Tool-Augmented Reward Modeling☆47Updated 4 months ago
- An Easy-to-use Hallucination Detection Framework for LLMs.☆58Updated last year
- ☆40Updated this week
- ☆41Updated this week
- Evaluation framework for paper "VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?"☆55Updated 6 months ago
- Watch Every Step! LLM Agent Learning via Iterative Step-level Process Refinement (EMNLP 2024 Main Conference)☆57Updated 6 months ago
- The code and data for the paper JiuZhang3.0☆43Updated 11 months ago
- Code for Paper: Teaching Language Models to Critique via Reinforcement Learning☆94Updated 3 weeks ago
- [NeurIPS 2024] A comprehensive benchmark for evaluating critique ability of LLMs☆39Updated 5 months ago
- [NeurIPS 2024 D&B Track] GTA: A Benchmark for General Tool Agents☆88Updated last month
- Reproduction of "RLCD Reinforcement Learning from Contrast Distillation for Language Model Alignment☆69Updated last year
- [preprint] We propose a novel fine-tuning method, Separate Memory and Reasoning, which combines prompt tuning with LoRA.☆44Updated 4 months ago
- [COLING 2025] ToolEyes: Fine-Grained Evaluation for Tool Learning Capabilities of Large Language Models in Real-world Scenarios☆65Updated 5 months ago
- Official implementation for "MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?"☆43Updated 2 months ago
- Official Code of IdealGPT☆36Updated last year
- This the implementation of LeCo☆31Updated 3 months ago
- ☆30Updated last year
- A instruction data generation system for multimodal language models.☆32Updated 3 months ago
- [NAACL 2024] Making Language Models Better Tool Learners with Execution Feedback☆41Updated last year
- Trial and Error: Exploration-Based Trajectory Optimization of LLM Agents (ACL 2024 Main Conference)☆138Updated 6 months ago
- Unofficial Implementation of Chain-of-Thought Reasoning Without Prompting☆32Updated last year
- Official github repo for the paper "Compression Represents Intelligence Linearly" [COLM 2024]☆134Updated 7 months ago