Alternatives and similar repositories for devin-swebench-results

Users that are interested in devin-swebench-results are comparing it to the libraries listed below

• Aider-AI / aider-swe-bench
  Harness used to benchmark aider against SWE Bench benchmarks
  ☆76Updated last year
• allenai / CommonGen-Eval
  Evaluating LLMs with CommonGen-Lite
  ☆91Updated last year
• teknium1 / LLM-Benchmark-Logs
  Just a bunch of benchmark logs for different LLMs
  ☆119Updated last year
• automix-llm / automix
  Mixing Language Models with Self-Verification and Meta-Verification
  ☆110Updated 9 months ago
• SWE-bench / sb-cli
  Run SWE-bench evaluations remotely
  ☆42Updated last month
• aorwall / moatless-tree-search
  ☆112Updated 3 months ago
• aymeric-roucher / agent_reasoning_benchmark
  🔧 Compare how Agent systems perform on several benchmarks. 📊🚀
  ☆102Updated last month
• SWE-bench / experiments
  Open sourced predictions, execution logs, trajectories, and results from model inference + evaluation runs on the SWE-bench task.
  ☆212Updated this week
• LLM360 / crystalcoder-train
  Pre-training code for CrystalCoder 7B LLM
  ☆55Updated last year
• Ag2S1 / Sibyl-System
  ☆124Updated last year
• togethercomputer / Llama-2-7B-32K-Instruct
  ☆85Updated 2 years ago
• DeepSoftwareAnalytics / Awesome-Agent4SE
  ☆99Updated last year
• asappresearch / webagents-step
  ☆41Updated last year
• Anni-Zou / Meta-CoT
  Meta-CoT: Generalizable Chain-of-Thought Prompting in Mixed-task Scenarios with Large Language Models
  ☆97Updated last year
• reasoning-machines / prompt-lib
  A set of utilities for running few-shot prompting experiments on large-language models
  ☆122Updated last year
• VITA-Group / ChainCoder
  [ICML 2023] "Outline, Then Details: Syntactically Guided Coarse-To-Fine Code Generation", Wenqing Zheng, S P Sharan, Ajay Kumar Jaiswal, …
  ☆41Updated last year
• agiresearch / Formal-LLM
  Formal-LLM: Integrating Formal Language and Natural Language for Controllable LLM-based Agents
  ☆128Updated last year
• stunningpixels / lou-eval
  Track the progress of LLM context utilisation
  ☆55Updated 5 months ago
• NL2Code / CodeR
  ☆159Updated last year
• aymeric-roucher / GAIA
  Beating the GAIA benchmark with Transformers Agents. 🚀
  ☆135Updated 7 months ago
• young-geng / koala_data_pipeline
  The data processing pipeline for the Koala chatbot language model
  ☆118Updated 2 years ago
• InternLM / SWE-Fixer
  ☆117Updated 4 months ago
• amazon-science / CodeSage
  CodeSage: Code Representation Learning At Scale (ICLR 2024)
  ☆112Updated 10 months ago
• LLM360 / Analysis360
  Open Implementations of LLM Analyses
  ☆107Updated 11 months ago
• TheDuckAI / DuckTrack
  Multimodal computer agent data collection program
  ☆146Updated last year
• evalplus / repoqa
  RepoQA: Evaluating Long-Context Code Understanding
  ☆117Updated 10 months ago
• oriyor / assistantbench
  Implementation of the paper: "AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?"
  ☆62Updated 9 months ago
• xingyaoww / LeTI
  Official repo for NAACL 2024 Findings paper "LeTI: Learning to Generate from Textual Interactions."
  ☆64Updated 2 years ago
• McGill-NLP / weblinx
  WebLINX is a benchmark for building web navigation agents with conversational capabilities
  ☆158Updated 7 months ago
• METR / eval-analysis-public
  Public repository containing METR's DVC pipeline for eval data analysis
  ☆108Updated 5 months ago