RAIVNLab / mnms
m&ms: A Benchmark to Evaluate Tool-Use for multi-step multi-modal tasks
☆35Updated 4 months ago
Alternatives and similar repositories for mnms:
Users that are interested in mnms are comparing it to the libraries listed below
- [NeurIPS 2024] A task generation and model evaluation system for multimodal language models.☆62Updated 2 months ago
- This is the repo for our paper "Mr-Ben: A Comprehensive Meta-Reasoning Benchmark for Large Language Models"☆45Updated 3 months ago
- ☆58Updated 4 months ago
- Trial and Error: Exploration-Based Trajectory Optimization of LLM Agents (ACL 2024 Main Conference)☆114Updated 3 months ago
- [ACL 2024] The project of Symbol-LLM☆47Updated 6 months ago
- An Easy-to-use Hallucination Detection Framework for LLMs.☆55Updated 9 months ago
- Source code of "Reasons to Reject? Aligning Language Models with Judgments"☆58Updated 11 months ago
- ☆37Updated 2 months ago
- Unofficial Implementation of Chain-of-Thought Reasoning Without Prompting☆26Updated 10 months ago
- Evaluation framework for paper "VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?"☆48Updated 3 months ago
- Code for paper "Unraveling Cross-Modality Knowledge Conflicts in Large Vision-Language Models."☆39Updated 3 months ago
- [ACL 2024 Findings] CriticBench: Benchmarking LLMs for Critique-Correct Reasoning☆21Updated 10 months ago
- Reproduction of "RLCD Reinforcement Learning from Contrast Distillation for Language Model Alignment☆65Updated last year
- [ICLR'24 spotlight] Tool-Augmented Reward Modeling☆44Updated last month
- This the implementation of LeCo☆30Updated last week
- [ACL2024] Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarios☆51Updated 10 months ago
- A instruction data generation system for multimodal language models.☆29Updated 3 weeks ago
- Improving Language Understanding from Screenshots. Paper: https://arxiv.org/abs/2402.14073☆26Updated 6 months ago
- This is the official repository of the paper "OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI"☆91Updated last month
- Watch Every Step! LLM Agent Learning via Iterative Step-level Process Refinement (EMNLP 2024 Main Conference)☆51Updated 3 months ago
- [ACL'24] Code and data of paper "When is Tree Search Useful for LLM Planning? It Depends on the Discriminator"☆54Updated 11 months ago
- [AAAI 2025 oral] Evaluating Mathematical Reasoning Beyond Accuracy☆44Updated last month
- Evaluate the Quality of Critique☆35Updated 7 months ago
- [NeurIPS 2024] The official implementation of paper: Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs.☆90Updated 3 months ago
- Directional Preference Alignment☆54Updated 4 months ago
- Official repository for MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models [NeurIPS 2024]☆58Updated 2 months ago
- The code and data for the paper JiuZhang3.0☆40Updated 8 months ago
- In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigation (ICML 2024)☆50Updated 10 months ago
- [ICLR 2025] InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationales☆66Updated 2 months ago
- ☆22Updated 7 months ago