Alternatives and similar repositories for DevEval:

Users that are interested in DevEval are comparing it to the libraries listed below

• thunlp / DebugBench
  The repository for paper "DebugBench: "Evaluating Debugging Capability of Large Language Models".
  ☆59Updated 5 months ago
• zjunlp / AutoAct
  [ACL 2024] AUTOACT: Automatic Agent Learning from Scratch for QA via Self-Planning
  ☆186Updated 2 months ago
• ZubinGou / math-evaluation-harness
  A simple toolkit for benchmarking LLMs on mathematical reasoning tasks. 🧮✨
  ☆117Updated 7 months ago
• THUNLP-MT / StableToolBench
  A new tool learning benchmark aiming at well-balanced stability and reality, based on ToolBench.
  ☆121Updated 3 months ago
• tongyx361 / Awesome-LLM4Math
  Curation of resources for LLM mathematical reasoning, most of which are screened by @tongyx361 to ensure high quality and accompanied wit…
  ☆92Updated 5 months ago
• code-rag-bench / code-rag-bench
  CodeRAG-Bench: Can Retrieval Augment Code Generation?
  ☆90Updated last month
• GAIR-NLP / ProX
  Offical Repo for "Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale"
  ☆201Updated 2 months ago
• Junjie-Ye / ToolEyes
  [COLING 2025] ToolEyes: Fine-Grained Evaluation for Tool Learning Capabilities of Large Language Models in Real-world Scenarios
  ☆64Updated 2 weeks ago
• hkust-nlp / llm-compression-intelligence
  Official github repo for the paper "Compression Represents Intelligence Linearly" [COLM 2024]
  ☆127Updated 2 months ago
• bigai-nlco / LooGLE
  ACL 2024 | LooGLE: Long Context Evaluation for Long-Context Language Models
  ☆170Updated 2 months ago
• KbsdJames / Omni-MATH
  The official repository of the Omni-MATH benchmark.
  ☆59Updated last month
• THUDM / NaturalCodeBench
  NaturalCodeBench (Findings of ACL 2024)
  ☆59Updated 2 months ago
• TIGER-AI-Lab / LongICLBench
  Code and Data for "Long-context LLMs Struggle with Long In-context Learning"
  ☆92Updated 5 months ago
• xlang-ai / Spider2-V
  [NeurIPS 2024] Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows?
  ☆112Updated 3 months ago
• FlagOpen / TACO
  ☆149Updated 4 months ago
• facebookresearch / cruxeval
  CRUXEval: Code Reasoning, Understanding, and Execution Evaluation
  ☆116Updated 2 months ago
• NingMiao / SelfCheck
  Code for the paper <SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoning>
  ☆46Updated last year
• RUCAIBox / Slow_Thinking_with_LLMs
  A series of technical report on Slow Thinking with LLM
  ☆23Updated this week
• open-compass / MathBench
  [ACL 2024 Findings] MathBench: A Comprehensive Multi-Level Difficulty Mathematics Evaluation Dataset
  ☆86Updated 5 months ago
• open-compass / Ada-LEval
  The official implementation of "Ada-LEval: Evaluating long-context LLMs with length-adaptable benchmarks"
  ☆51Updated 7 months ago
• zorazrw / awesome-tool-llm
  ☆200Updated 4 months ago
• sail-sg / CPO
  [NeurIPS 2024] The official implementation of paper: Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs.
  ☆73Updated last month
• dvlab-research / Mr-Ben
  This is the repo for our paper "Mr-Ben: A Comprehensive Meta-Reasoning Benchmark for Large Language Models"
  ☆43Updated last month
• OFA-Sys / gsm8k-ScRel
  Codes and Data for Scaling Relationship on Learning Mathematical Reasoning with Large Language Models
  ☆225Updated 3 months ago
• SkyworkAI / skywork-o1-prm-inference
  ☆37Updated 3 weeks ago
• ozyyshr / RepoGraph
  Enhancing AI Software Engineering with Repository-level Code Graph
  ☆101Updated 3 months ago
• sail-sg / sdft
  [ACL 2024] The official codebase for the paper "Self-Distillation Bridges Distribution Gap in Language Model Fine-tuning".
  ☆105Updated last month
• LightChen233 / reasoning-boundary
  ☆34Updated 2 months ago
• GAIR-NLP / OlympicArena
  This is the official repository of the paper "OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI"
  ☆89Updated this week
• MTU-Bench-Team / MTU-Bench
  MTU-Bench: A Multi-granularity Tool-Use Benchmark for Large Language Models
  ☆21Updated last month