Alternatives and similar repositories for DevEval:

Users that are interested in DevEval are comparing it to the libraries listed below

• thunlp / DebugBench
  The repository for paper "DebugBench: "Evaluating Debugging Capability of Large Language Models".
  ☆66Updated 8 months ago
• zjunlp / AutoAct
  [ACL 2024] AutoAct: Automatic Agent Learning from Scratch for QA via Self-Planning
  ☆213Updated 2 months ago
• THUDM / NaturalCodeBench
  NaturalCodeBench (Findings of ACL 2024)
  ☆62Updated 4 months ago
• THUNLP-MT / StableToolBench
  A new tool learning benchmark aiming at well-balanced stability and reality, based on ToolBench.
  ☆132Updated last week
• SALT-NLP / DyLAN
  Official Implementation of Dynamic LLM-Agent Network: An LLM-agent Collaboration Framework with Agent Team Optimization
  ☆132Updated 9 months ago
• zjunlp / WorfBench
  [ICLR 2025] Benchmarking Agentic Workflow Generation
  ☆58Updated 3 weeks ago
• InfiAgent / InfiAgent
  InfiAgent-DABench: Evaluating Agents on Data Analysis Tasks (ICML 2024)
  ☆111Updated 2 months ago
• facebookresearch / cruxeval
  CRUXEval: Code Reasoning, Understanding, and Execution Evaluation
  ☆131Updated 5 months ago
• xlang-ai / Spider2-V
  [NeurIPS 2024] Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows?
  ☆118Updated 6 months ago
• Yifan-Song793 / ETO
  Trial and Error: Exploration-Based Trajectory Optimization of LLM Agents (ACL 2024 Main Conference)
  ☆127Updated 4 months ago
• ntunlp / xCodeEval
  xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval
  ☆78Updated 5 months ago
• GAIR-NLP / ProX
  Offical Repo for "Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale"
  ☆226Updated 3 weeks ago
• code-rag-bench / code-rag-bench
  CodeRAG-Bench: Can Retrieval Augment Code Generation?
  ☆117Updated 3 months ago
• diagram-of-thought / diagram-of-thought
  Official implementation of paper "On the Diagram of Thought" (https://arxiv.org/abs/2409.10038)
  ☆174Updated 5 months ago
• bytarnish / AGILE
  ☆103Updated last month
• Open-Source-O1 / o1_Reasoning_Patterns_Study
  ☆101Updated 3 months ago
• WooooDyy / MathCritique
  Implementation for the research paper "Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervision".
  ☆51Updated 3 months ago
• OS-Copilot / OS-Genesis
  Code and data for OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis
  ☆111Updated last week
• zorazrw / awesome-tool-llm
  ☆214Updated 6 months ago
• GAIR-NLP / ReAlign
  Reformatted Alignment
  ☆114Updated 5 months ago
• Leolty / repobench
  ✨ RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems - ICLR 2024
  ☆146Updated 6 months ago
• LeapLabTHU / ExpeL
  ☆123Updated 2 months ago
• AlphaPav / mem-kk-logic
  On Memorization of Large Language Models in Logical Reasoning
  ☆53Updated 4 months ago
• Lordog / R-Judge
  R-Judge: Benchmarking Safety Risk Awareness for LLM Agents (EMNLP Findings 2024)
  ☆66Updated last month
• THUDM / VisualAgentBench
  Towards Large Multimodal Models as Visual Foundation Agents
  ☆192Updated last month