Alternatives and similar repositories for swe-bench.github.io

Users that are interested in swe-bench.github.io are comparing it to the libraries listed below

• Aider-AI / aider-swe-bench
  Harness used to benchmark aider against SWE Bench benchmarks
  ☆79Updated last year
• aorwall / moatless-testbeds
  Moatless Testbeds allows you to create isolated testbed environments in a Kubernetes cluster where you can apply code changes through git…
  ☆14Updated 10 months ago
• OpenHands / trajectory-visualizer
  ☆45Updated 3 weeks ago
• SWE-bench / sb-cli
  Run SWE-bench evaluations remotely
  ☆51Updated 5 months ago
• dxhou / CoAct
  ☆32Updated last year
• a-antoniades / swe-search
  ☆12Updated last year
• aorwall / moatless-tree-search
  ☆132Updated 8 months ago
• agential-ai / agential
  🔔🧠 Easily experiment with popular language agents across diverse reasoning/decision-making benchmarks!
  ☆53Updated 6 months ago
• allenai / olmo-cookbook
  OLMost every training recipe you need to perform data interventions with the OLMo family of models.
  ☆64Updated this week
• BigComputer-Project / SWE-Arena
  SWE Arena
  ☆35Updated 7 months ago
• kagnlp / CodeGenerator
  This repository contains popular code generation frameworks such as MapCoder, CodeSIM.
  ☆69Updated 7 months ago
• clinicalml / realhumaneval
  ☆22Updated last year
• goncalorafaria / qalign
  QAlign is a new test-time alignment approach that improves language model performance by using Markov chain Monte Carlo methods.
  ☆26Updated last month
• sunblaze-ucb / omega
  ☆46Updated 7 months ago
• THUDM / SWE-Dev
  [ACL25' Findings] SWE-Dev is an SWE agent with a scalable test case construction pipeline.
  ☆58Updated 6 months ago
• SalesforceAIResearch / swecomm
  ☆28Updated 2 months ago
• VITA-Group / ChainCoder
  [ICML 2023] "Outline, Then Details: Syntactically Guided Coarse-To-Fine Code Generation", Wenqing Zheng, S P Sharan, Ajay Kumar Jaiswal, …
  ☆43Updated 2 years ago
• evalplus / repoqa
  RepoQA: Evaluating Long-Context Code Understanding
  ☆128Updated last year
• sunblaze-ucb / reasoning_ladder
  ☆35Updated 8 months ago
• oriyor / assistantbench
  Implementation of the paper: "AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?"
  ☆69Updated last year
• SWE-bench / experiments
  Open sourced predictions, execution logs, trajectories, and results from model inference + evaluation runs on the SWE-bench task.
  ☆246Updated last week
• METR / eval-analysis-public
  Public repository containing METR's DVC pipeline for eval data analysis
  ☆199Updated last week
• sorendunn / Agentless-Lite
  Agentless Lite: RAG-based SWE-Bench software engineering scaffold
  ☆45Updated 9 months ago
• SalesforceAIResearch / CodeTree
  Code for the paper: CodeTree: Agent-guided Tree Search for Code Generation with Large Language Models
  ☆31Updated 10 months ago
• Ablustrund / APPS_Plus
  StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback
  ☆74Updated last year
• schauppi / Self-Rewarding-Language-Models
  ☆48Updated last year
• AlexCuadron / ThinkingAgent
  Systematic evaluation framework that automatically rates overthinking behavior in large language models.
  ☆96Updated 8 months ago
• yueqis / API-Based-Agent
  ☆61Updated 7 months ago
• benpry / why-think-step-by-step
  Code and data for the paper "Why think step by step? Reasoning emerges from the locality of experience"
  ☆62Updated 10 months ago
• ZeroSumEval / ZeroSumEval
  A framework for pitting LLMs against each other in an evolving library of games ⚔
  ☆35Updated 9 months ago