SWE-bench / sb-cliLinks
Run SWE-bench evaluations remotely
☆17Updated 2 weeks ago
Alternatives and similar repositories for sb-cli
Users that are interested in sb-cli are comparing it to the libraries listed below
Sorting:
- This repository includes a benchmark and code for the paper "Evaluating LLMs at Detecting Errors in LLM Responses".☆29Updated 9 months ago
- Official repo for NAACL 2024 Findings paper "LeTI: Learning to Generate from Textual Interactions."☆65Updated last year
- Implementation of the paper: "AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?"☆56Updated 5 months ago
- Reasoning by Communicating with Agents☆28Updated last month
- ☆51Updated 7 months ago
- Code and Dataset for Learning to Solve Complex Tasks by Talking to Agents☆24Updated 3 years ago
- [EMNLP'23] Execution-Based Evaluation for Open Domain Code Generation☆48Updated last year
- Code, datasets, models for the paper "Automatic Evaluation of Attribution by Large Language Models"☆56Updated last year
- ☆42Updated 2 months ago
- Middleware for LLMs: Tools Are Instrumental for Language Agents in Complex Environments (EMNLP'2024)☆36Updated 5 months ago
- ☆53Updated 11 months ago
- Moatless Testbeds allows you to create isolated testbed environments in a Kubernetes cluster where you can apply code changes through git…☆11Updated last month
- Aioli: A unified optimization framework for language model data mixing☆25Updated 4 months ago
- Learning to Retrieve by Trying - Source code for Grounding by Trying: LLMs with Reinforcement Learning-Enhanced Retrieval☆34Updated 7 months ago
- [ACL'24] Code and data of paper "When is Tree Search Useful for LLM Planning? It Depends on the Discriminator"☆54Updated last year
- A set of utilities for running few-shot prompting experiments on large-language models☆121Updated last year
- A framework for pitting LLMs against each other in an evolving library of games ⚔☆32Updated last month
- Functional Benchmarks and the Reasoning Gap☆86Updated 8 months ago
- ☆26Updated 2 months ago
- Pre-training code for CrystalCoder 7B LLM☆54Updated last year
- ☆57Updated 8 months ago
- [EMNLP 2024] A Retrieval Benchmark for Scientific Literature Search☆89Updated 6 months ago
- ☆45Updated 9 months ago
- Scalable Meta-Evaluation of LLMs as Evaluators☆42Updated last year
- ☆83Updated last month
- LLMs as Collaboratively Edited Knowledge Bases☆45Updated last year
- ReBase: Training Task Experts through Retrieval Based Distillation☆29Updated 3 months ago
- Code and Data for "Language Modeling with Editable External Knowledge"☆33Updated 11 months ago
- Small, simple agent task environments for training and evaluation☆18Updated 7 months ago
- [ICLR'25] "Attention in Large Language Models Yields Efficient Zero-Shot Re-Rankers"☆19Updated 2 months ago