Alternatives and similar repositories for MCPEval

Users that are interested in MCPEval are comparing it to the libraries listed below

• fairyshine / Chain-of-Tools
  The official implementation of the paper "Chain-of-Tools: Utilizing Massive Unseen Tools in the CoT Reasoning of Frozen Language Models".
  ☆83Updated 7 months ago
• SalesforceAIResearch / CRMArena
  Official Repo for CRMArena and CRMArena-Pro
  ☆119Updated 4 months ago
• facebookresearch / matrix
  Matrix (Multi-Agent daTa geneRation Infra and eXperimentation framework) is a versatile engine for multi-agent conversational data genera…
  ☆99Updated this week
• jina-ai / jina-vdr
  Jina VDR is a multilingual, multi-domain benchmark for visual document retrieval
  ☆30Updated 2 months ago
• HishamAlyahya / semantic_backprop
  Source code of "How to Correctly do Semantic Backpropagation on Language-based Agentic Systems" 🤖
  ☆76Updated 10 months ago
• SakanaAI / natural_niches
  The code repository of the paper: Competition and Attraction Improve Model Fusion
  ☆161Updated 2 months ago
• microsoft / llm-steer-instruct
  A method for steering llms to better follow instructions
  ☆55Updated 2 months ago
• facebookresearch / collaborative-reasoner
  Source code for the collaborative reasoner research project at Meta FAIR.
  ☆103Updated 6 months ago
• letta-ai / sleep-time-compute
  accompanying material for sleep-time compute paper
  ☆117Updated 6 months ago
• padas-lab-de / ir-rag-sigir24-persona-rag
  ☆50Updated last year
• StigLidu / DualDistill
  [EMNLP 2025] The official implementation for paper "Agentic-R1: Distilled Dual-Strategy Reasoning"
  ☆101Updated 2 months ago
• huggingface / screensuite
  ScreenSuite - The most comprehensive benchmarking suite for GUI Agents!
  ☆130Updated last month
• adityasoni9998 / OpenHands-Versa
  Code for the paper "Coding Agents with Multimodal Browsing are Generalist Problem Solvers"
  ☆88Updated this week
• PathOnAIOrg / LiteWebAgent
  [NAACL2025] LiteWebAgent: The Open-Source Suite for VLM-Based Web-Agent Applications
  ☆127Updated 3 months ago
• oriyor / assistantbench
  Implementation of the paper: "AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?"
  ☆63Updated 10 months ago
• sony / talkhier
  Official Repo for The Paper "Talk Structurally, Act Hierarchically: A Collaborative Framework for LLM Multi-Agent Systems"
  ☆57Updated 8 months ago
• DeepSoftwareAnalytics / Awesome-Agent4SE
  ☆101Updated last year
• google / lmeval
  ☆232Updated 3 months ago
• miralab-ai / autoreason
  ☆40Updated 10 months ago
• LLMSELECTOR / LLMSELECTOR
  ☆79Updated last month
• microsoft / eureka-ml-insights
  A framework for standardizing evaluations of large foundation models, beyond single-score reporting and rankings.
  ☆169Updated this week
• THU-KEG / Agentic-Reward-Modeling
  [ACL 2025] Agentic Reward Modeling: Integrating Human Preferences with Verifiable Correctness Signals for Reliable Reward Systems
  ☆108Updated 4 months ago
• javyduck / KnowHalu
  ☆48Updated last year
• AlexCuadron / ThinkingAgent
  Systematic evaluation framework that automatically rates overthinking behavior in large language models.
  ☆93Updated 5 months ago
• SalesforceAIResearch / SFR-RAG
  ☆79Updated 9 months ago
• Columbia-NLP-Lab / PAPILLON
  Code for our paper PAPILLON: PrivAcy Preservation from Internet-based and Local Language MOdel ENsembles
  ☆59Updated 5 months ago
• pgasawa / BARE
  Leveraging Base Language Models for Few-Shot Synthetic Data Generation
  ☆36Updated last week
• diffbot / diffbot-llm-inference
  DIffbot LLM Inference Server
  ☆201Updated 2 months ago
• google-deepmind / latent-multi-hop-reasoning
  [ACL 2024] Do Large Language Models Latently Perform Multi-Hop Reasoning?
  ☆79Updated 7 months ago
• OpenPipe / deductive-reasoning
  Train your own SOTA deductive reasoning model
  ☆109Updated 7 months ago