princeton-nlp / intercode
[NeurIPS 2023 D&B] Code repository for InterCode benchmark https://arxiv.org/abs/2306.14898
☆209Updated 10 months ago
Alternatives and similar repositories for intercode:
Users that are interested in intercode are comparing it to the libraries listed below
- Accepted by Transactions on Machine Learning Research (TMLR)☆126Updated 5 months ago
- Code for paper "LEVER: Learning to Verifiy Language-to-Code Generation with Execution" (ICML'23)☆84Updated last year
- 🌍 Repository for "AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agent", ACL'24 Best Resource Pap…☆162Updated 3 months ago
- A set of utilities for running few-shot prompting experiments on large-language models☆118Updated last year
- CRUXEval: Code Reasoning, Understanding, and Execution Evaluation☆131Updated 5 months ago
- ☆86Updated 8 months ago
- A codebase for "Language Models can Solve Computer Tasks"☆233Updated 10 months ago
- [ICLR'24 Spotlight] A language model (LM)-based emulation framework for identifying the risks of LM agents with tool use☆131Updated last year
- A hard gym for programming☆149Updated 8 months ago
- [ICML 2023] Data and code release for the paper "DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation".☆238Updated 4 months ago
- ✨ RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems - ICLR 2024☆147Updated 7 months ago
- A benchmark that challenges language models to code solutions for scientific problems☆111Updated last week
- ☆115Updated 8 months ago
- [NeurIPS 2022] 🛒WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents☆311Updated 6 months ago
- CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completion (NeurIPS 2023)☆135Updated 7 months ago
- ☆217Updated 7 months ago
- Code for the paper 🌳 Tree Search for Language Model Agents☆185Updated 7 months ago
- Functional Benchmarks and the Reasoning Gap☆84Updated 5 months ago
- Code and data for "Lumos: Learning Agents with Unified Data, Modular Design, and Open-Source LLMs"☆463Updated last year
- An Analytical Evaluation Board of Multi-turn LLM Agents [NeurIPS 2024 Oral]☆291Updated 10 months ago
- Official Repo for ICLR 2024 paper MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback by Xingyao Wang*, Ziha…☆117Updated 9 months ago
- A simple unified framework for evaluating LLMs☆204Updated last week
- An extensible benchmark for evaluating large language models on planning☆329Updated this week
- ToolBench, an evaluation suite for LLM tool manipulation capabilities.☆150Updated last year
- ☆130Updated 4 months ago
- Archon provides a modular framework for combining different inference-time techniques and LMs with just a JSON config file.☆164Updated 2 weeks ago
- This is the repo for the paper Shepherd -- A Critic for Language Model Generation☆218Updated last year
- Synthetic question-answering dataset to formally analyze the chain-of-thought output of large language models on a reasoning task.☆138Updated 5 months ago
- Reasoning with Language Model is Planning with World Model☆161Updated last year
- Repository for the paper Stream of Search: Learning to Search in Language☆142Updated last month