Alternatives and similar repositories for DevEval:

Users that are interested in DevEval are comparing it to the libraries listed below

• zjunlp / AutoAct
  [ACL 2024] AutoAct: Automatic Agent Learning from Scratch for QA via Self-Planning
  ☆215Updated 2 months ago
• ganler / code-r1
  Reproducing R1 for Code with Reliable Rewards
  ☆132Updated 3 weeks ago
• THUDM / T1
  Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling
  ☆95Updated 2 months ago
• THUNLP-MT / StableToolBench
  A new tool learning benchmark aiming at well-balanced stability and reality, based on ToolBench.
  ☆135Updated 2 weeks ago
• bigai-nlco / LooGLE
  ACL 2024 | LooGLE: Long Context Evaluation for Long-Context Language Models
  ☆179Updated 5 months ago
• hkust-nlp / AgentBoard
  An Analytical Evaluation Board of Multi-turn LLM Agents [NeurIPS 2024 Oral]
  ☆291Updated 10 months ago
• kyegomez / Lets-Verify-Step-by-Step
  "Improving Mathematical Reasoning with Process Supervision" by OPENAI
  ☆108Updated 3 weeks ago
• GAIR-NLP / ProX
  Offical Repo for "Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale"
  ☆229Updated last month
• thunlp / DebugBench
  The repository for paper "DebugBench: "Evaluating Debugging Capability of Large Language Models".
  ☆68Updated 8 months ago
• GAIR-NLP / ReAlign
  Reformatted Alignment
  ☆115Updated 6 months ago
• sail-sg / oat-zero
  A lightweight reproduction of DeepSeek-R1-Zero with indepth analysis of self-reflection behavior.
  ☆212Updated this week
• CodeEditorBench / CodeEditorBench
  ☆45Updated 9 months ago
• InfiAgent / InfiAgent
  InfiAgent-DABench: Evaluating Agents on Data Analysis Tasks (ICML 2024)
  ☆113Updated 3 months ago
• qishenghu / InstructCoder
  InstructCoder: Instruction Tuning Large Language Models for Code Editing | Oral ACL-2024 srw
  ☆58Updated 5 months ago
• StonyBrookNLP / appworld
  🌍 Repository for "AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agent", ACL'24 Best Resource Pap…
  ☆165Updated this week
• code-rag-bench / code-rag-bench
  CodeRAG-Bench: Can Retrieval Augment Code Generation?
  ☆119Updated 4 months ago
• tongyx361 / Awesome-LLM4Math
  Curation of resources for LLM mathematical reasoning, most of which are screened by @tongyx361 to ensure high quality and accompanied wit…
  ☆118Updated 8 months ago
• microsoft / simulated-trial-and-error
  ☆120Updated 9 months ago
• xlang-ai / DS-1000
  [ICML 2023] Data and code release for the paper "DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation".
  ☆238Updated 4 months ago
• zorazrw / awesome-tool-llm
  ☆217Updated 7 months ago
• cmu-l3 / l1
  L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning
  ☆148Updated this week
• ZubinGou / math-evaluation-harness
  A simple toolkit for benchmarking LLMs on mathematical reasoning tasks. 🧮✨
  ☆189Updated 10 months ago
• zjunlp / WorfBench
  [ICLR 2025] Benchmarking Agentic Workflow Generation
  ☆62Updated last month
• Ber666 / ToolkenGPT
  ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddings - NeurIPS 2023 (oral)
  ☆261Updated 11 months ago
• TIGER-AI-Lab / Program-of-Thoughts
  Data and Code for Program of Thoughts (TMLR 2023)
  ☆263Updated 10 months ago
• Berkeley-NLP / Agent-Eval-Refine
  Code for Paper: Autonomous Evaluation and Refinement of Digital Agents [COLM 2024]
  ☆130Updated 3 months ago
• SkyworkAI / skywork-o1-prm-inference
  ☆60Updated 3 months ago
• AlphaPav / mem-kk-logic
  On Memorization of Large Language Models in Logical Reasoning
  ☆56Updated 4 months ago
• bytedance / SandboxFusion
  ☆160Updated last month
• hkust-nlp / llm-compression-intelligence
  Official github repo for the paper "Compression Represents Intelligence Linearly" [COLM 2024]
  ☆130Updated 6 months ago