Alternatives and similar repositories for devin-swebench-results

Users that are interested in devin-swebench-results are comparing it to the libraries listed below

• Aider-AI / aider-swe-bench
  Harness used to benchmark aider against SWE Bench benchmarks
  ☆78Updated last year
• teknium1 / LLM-Benchmark-Logs
  Just a bunch of benchmark logs for different LLMs
  ☆119Updated last year
• automix-llm / automix
  Mixing Language Models with Self-Verification and Meta-Verification
  ☆111Updated last year
• reasoning-machines / prompt-lib
  A set of utilities for running few-shot prompting experiments on large-language models
  ☆126Updated 2 years ago
• InflectionAI / Inflection-Benchmarks
  Public Inflection Benchmarks
  ☆68Updated last year
• allenai / CommonGen-Eval
  Evaluating LLMs with CommonGen-Lite
  ☆93Updated last year
• asappresearch / webagents-step
  ☆41Updated last year
• togethercomputer / Llama-2-7B-32K-Instruct
  ☆85Updated 2 years ago
• Ag2S1 / Sibyl-System
  ☆125Updated last year
• Anni-Zou / Meta-CoT
  Meta-CoT: Generalizable Chain-of-Thought Prompting in Mixed-task Scenarios with Large Language Models
  ☆100Updated 2 years ago
• aorwall / moatless-tree-search
  ☆129Updated 7 months ago
• posgnu / rci-agent
  A codebase for "Language Models can Solve Computer Tasks"
  ☆239Updated last year
• NL2Code / CodeR
  ☆159Updated last year
• agiresearch / Formal-LLM
  Formal-LLM: Integrating Formal Language and Natural Language for Controllable LLM-based Agents
  ☆132Updated last year
• allenai / recoma
  Reasoning by Communicating with Agents
  ☆29Updated 8 months ago
• cursor / eval
  ☆142Updated 2 years ago
• neulab / gemini-benchmark
  ☆150Updated 2 years ago
• McGill-NLP / weblinx
  WebLINX is a benchmark for building web navigation agents with conversational capabilities
  ☆156Updated 11 months ago
• HazyResearch / eclair-agents
  Automating enterprise workflows with multimodal agents
  ☆114Updated last year
• aymeric-roucher / agent_reasoning_benchmark
  🔧 Compare how Agent systems perform on several benchmarks. 📊🚀
  ☆103Updated 5 months ago
• LLM360 / crystalcoder-train
  Pre-training code for CrystalCoder 7B LLM
  ☆56Updated last year
• PrimeIntellect-ai / genesys
  ☆135Updated 9 months ago
• SWE-bench / sb-cli
  Run SWE-bench evaluations remotely
  ☆50Updated 5 months ago
• SWE-bench / experiments
  Open sourced predictions, execution logs, trajectories, and results from model inference + evaluation runs on the SWE-bench task.
  ☆237Updated this week
• Aider-AI / refactor-benchmark
  Aider's refactoring benchmark exercises based on popular python repos
  ☆78Updated last year
• wang-research-lab / agentinstruct
  Code repo for "Agent Instructs Large Language Models to be General Zero-Shot Reasoners"
  ☆120Updated 2 months ago
• All-Hands-AI / openhands-resolver
  A system that tries to resolve all issues on a github repo with OpenHands.
  ☆117Updated last year
• uclaml / Rephrase-and-Respond
  Official repo of Respond-and-Respond: data, code, and evaluation
  ☆103Updated last year
• aymeric-roucher / GAIA
  Beating the GAIA benchmark with Transformers Agents. 🚀
  ☆144Updated 10 months ago
• allenai / clin
  ☆86Updated 2 years ago