Alternatives and similar repositories for SWEBench-verified-mini

Users that are interested in SWEBench-verified-mini are comparing it to the libraries listed below

• R2E-Gym / R2E-Gym
  Official repository for R2E-Gym: Procedural Environment Generation and Hybrid Verifiers for Scaling Open-Weights SWE Agents
  ☆77Updated 2 weeks ago
• nuprl / CanItEdit
  Can It Edit? Evaluating the Ability of Large Language Models to Follow Code Editing Instructions
  ☆43Updated 10 months ago
• METR / RE-Bench
  ☆87Updated 2 months ago
• BigComputer-Project / SWE-Arena
  SWE Arena
  ☆34Updated 2 months ago
• hughbzhang / o1_inference_scaling_laws
  Replicating O1 inference-time scaling laws
  ☆87Updated 6 months ago
• aorwall / SWE-bench-docker
  ☆97Updated 11 months ago
• zorazrw / odex
  [EMNLP'23] Execution-Based Evaluation for Open Domain Code Generation
  ☆48Updated last year
• logic-star-ai / swt-bench
  [NeurIPS 2024] Evaluation harness for SWT-Bench, a benchmark for evaluating LLM repository-level test-generation
  ☆50Updated 2 weeks ago
• ApolloResearch / e2e_sae
  Sparse Autoencoder Training Library
  ☆52Updated last month
• gonglinyuan / safim
  ☆36Updated last month
• SalesforceAIResearch / LaTRO
  ☆115Updated 4 months ago
• shunzh / Code-AI-Tree-Search
  ☆119Updated 11 months ago
• gso-bench / gso
  GSO: Challenging Software Optimization Tasks for Evaluating SWE-Agents
  ☆27Updated 3 weeks ago
• zorazrw / agent-skill-induction
  Agent Skill Induction: "Inducing Programmatic Skills for Agentic Tasks"
  ☆22Updated 2 months ago
• r2e-project / r2e
  r2e: turn any github repository into a programming agent environment
  ☆125Updated 2 months ago
• scicode-bench / SciCode
  A benchmark that challenges language models to code solutions for scientific problems
  ☆123Updated this week
• qishenghu / InstructCoder
  InstructCoder: Instruction Tuning Large Language Models for Code Editing | Oral ACL-2024 srw
  ☆61Updated 8 months ago
• MLE-Dojo / MLE-Dojo
  ☆53Updated last week
• ucl-dark / llm_debate
  Code release for "Debating with More Persuasive LLMs Leads to More Truthful Answers"
  ☆109Updated last year
• huggingface / ioi
  ☆35Updated 3 months ago
• princeton-nlp / USACO
  Can Language Models Solve Olympiad Programming?
  ☆117Updated 5 months ago
• bigcode-project / bigcodebench-annotation
  BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions
  ☆23Updated 10 months ago
• evalplus / repoqa
  RepoQA: Evaluating Long-Context Code Understanding
  ☆109Updated 7 months ago
• SalesforceAIResearch / swecomm
  ☆27Updated 5 months ago
• amazon-science / llm-code-preference
  Training and Benchmarking LLMs for Code Preference.
  ☆33Updated 7 months ago
• EleutherAI / elk-generalization
  Investigating the generalization behavior of LM probes trained to predict truth labels: (1) from one annotator to another, and (2) from e…
  ☆27Updated last year
• rmshin / llm-mcts
  ☆41Updated last year
• Aider-AI / aider-swe-bench
  Harness used to benchmark aider against SWE Bench benchmarks
  ☆72Updated 11 months ago
• UFO-101 / auto-circuit
  A library for efficient patching and automatic circuit discovery.
  ☆67Updated 2 months ago
• LoryPack / LLM-LieDetector
  Code for the ICLR 2024 paper "How to catch an AI liar: Lie detection in black-box LLMs by asking unrelated questions"
  ☆70Updated last year