Alternatives and similar repositories for SWEBench-verified-mini

Users that are interested in SWEBench-verified-mini are comparing it to the libraries listed below

• R2E-Gym / R2E-Gym
  Official repository for R2E-Gym: Procedural Environment Generation and Hybrid Verifiers for Scaling Open-Weights SWE Agents
  ☆128Updated last week
• gso-bench / gso
  GSO: Challenging Software Optimization Tasks for Evaluating SWE-Agents
  ☆30Updated last week
• Leolty / repobench
  ✨ RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems - ICLR 2024
  ☆168Updated 11 months ago
• qishenghu / InstructCoder
  InstructCoder: Instruction Tuning Large Language Models for Code Editing | Oral ACL-2024 srw
  ☆61Updated 9 months ago
• facebookresearch / cruxeval
  CRUXEval: Code Reasoning, Understanding, and Execution Evaluation
  ☆149Updated 9 months ago
• amazon-science / llm-code-preference
  Training and Benchmarking LLMs for Code Preference.
  ☆34Updated 8 months ago
• princeton-nlp / intercode
  [NeurIPS 2023 D&B] Code repository for InterCode benchmark https://arxiv.org/abs/2306.14898
  ☆222Updated last year
• princeton-nlp / USACO
  Can Language Models Solve Olympiad Programming?
  ☆119Updated 6 months ago
• aorwall / SWE-bench-docker
  ☆99Updated last year
• TransluceAI / observatory
  A toolkit for describing model features and intervening on those features to steer behavior.
  ☆194Updated 8 months ago
• UFO-101 / auto-circuit
  A library for efficient patching and automatic circuit discovery.
  ☆72Updated this week
• METR / RE-Bench
  ☆92Updated 3 months ago
• r2e-project / r2e
  r2e: turn any github repository into a programming agent environment
  ☆128Updated 3 months ago
• microsoft / SWE-bench-Live
  🚀 SWE-bench Goes Live!
  ☆100Updated last week
• amazon-science / cceval
  CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completion (NeurIPS 2023)
  ☆150Updated 11 months ago
• MLE-Dojo / MLE-Dojo
  ☆57Updated this week
• scicode-bench / SciCode
  A benchmark that challenges language models to code solutions for scientific problems
  ☆127Updated this week
• SWE-bench / experiments
  Open sourced predictions, execution logs, trajectories, and results from model inference + evaluation runs on the SWE-bench task.
  ☆194Updated 2 weeks ago
• shunzh / Code-AI-Tree-Search
  ☆119Updated last year
• bigcode-project / bigcodebench-annotation
  BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions
  ☆23Updated 11 months ago
• protagolabs / odyssey-math
  ☆84Updated 6 months ago
• huggingface / ioi
  ☆35Updated 3 months ago
• crux-eval / eval-arena
  ☆28Updated last week
• kanishkg / stream-of-search
  Repository for the paper Stream of Search: Learning to Search in Language
  ☆149Updated 5 months ago
• zorazrw / odex
  [EMNLP'23] Execution-Based Evaluation for Open Domain Code Generation
  ☆48Updated last year
• SalesforceAIResearch / LaTRO
  ☆117Updated 5 months ago
• logic-star-ai / swt-bench
  [NeurIPS 2024] Evaluation harness for SWT-Bench, a benchmark for evaluating LLM repository-level test-generation
  ☆51Updated last month
• SalesforceAIResearch / swecomm
  ☆27Updated 6 months ago
• evalplus / repoqa
  RepoQA: Evaluating Long-Context Code Understanding
  ☆112Updated 8 months ago
• ucl-dark / llm_debate
  Code release for "Debating with More Persuasive LLMs Leads to More Truthful Answers"
  ☆112Updated last year