SWE-bench / sb-cliLinks
Run SWE-bench evaluations remotely
☆21Updated last month
Alternatives and similar repositories for sb-cli
Users that are interested in sb-cli are comparing it to the libraries listed below
Sorting:
- Implementation of the paper: "AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?"☆57Updated 6 months ago
- [EMNLP'23] Execution-Based Evaluation for Open Domain Code Generation☆48Updated last year
- ☆41Updated last year
- [ACL'24] Code and data of paper "When is Tree Search Useful for LLM Planning? It Depends on the Discriminator"☆54Updated last year
- Reasoning by Communicating with Agents☆29Updated last month
- ☆86Updated 2 weeks ago
- Scalable Meta-Evaluation of LLMs as Evaluators☆42Updated last year
- ☆43Updated 2 months ago
- Source code for paper: INTERVENOR : Prompt the Coding Ability of Large Language Models with the Interactive Chain of Repairing☆26Updated 7 months ago
- ☆27Updated 5 months ago
- InstructCoder: Instruction Tuning Large Language Models for Code Editing | Oral ACL-2024 srw☆61Updated 8 months ago
- A framework for pitting LLMs against each other in an evolving library of games ⚔☆32Updated 2 months ago
- Middleware for LLMs: Tools Are Instrumental for Language Agents in Complex Environments (EMNLP'2024)☆37Updated 5 months ago
- Official repo for NAACL 2024 Findings paper "LeTI: Learning to Generate from Textual Interactions."☆65Updated last year
- Code for paper "LEVER: Learning to Verifiy Language-to-Code Generation with Execution" (ICML'23)☆88Updated last year
- 🔔🧠 Easily experiment with popular language agents across diverse reasoning/decision-making benchmarks!☆52Updated 3 months ago
- This repository includes a benchmark and code for the paper "Evaluating LLMs at Detecting Errors in LLM Responses".☆29Updated 10 months ago
- A set of utilities for running few-shot prompting experiments on large-language models☆121Updated last year
- Learning to Retrieve by Trying - Source code for Grounding by Trying: LLMs with Reinforcement Learning-Enhanced Retrieval☆39Updated 7 months ago
- Training and Benchmarking LLMs for Code Preference.☆33Updated 7 months ago
- Small, simple agent task environments for training and evaluation☆18Updated 7 months ago
- [NeurIPS 2024] Evaluation harness for SWT-Bench, a benchmark for evaluating LLM repository-level test-generation☆50Updated 3 weeks ago
- Functional Benchmarks and the Reasoning Gap☆87Updated 8 months ago
- Code and Dataset for Learning to Solve Complex Tasks by Talking to Agents☆24Updated 3 years ago
- Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory☆62Updated last month
- ☆75Updated 3 months ago
- Moatless Testbeds allows you to create isolated testbed environments in a Kubernetes cluster where you can apply code changes through git…☆13Updated 2 months ago
- Repository for NPHardEval, a quantified-dynamic benchmark of LLMs☆54Updated last year
- Code for the paper: CodeTree: Agent-guided Tree Search for Code Generation with Large Language Models☆22Updated 2 months ago
- ☆26Updated 11 months ago