[ICML 2023] Data and code release for the paper "DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation".
☆278Oct 30, 2024Updated last year
Alternatives and similar repositories for DS-1000
Users that are interested in DS-1000 are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆12Dec 2, 2022Updated 3 years ago
- ☆34Oct 14, 2026Updated 2 weeks ago
- ☆55Aug 25, 2023Updated 3 years ago
- A framework for the evaluation of autoregressive code generation language models.☆1,063Jul 22, 2025Updated last year
- [ICLR 2023] Code for our paper "Selective Annotation Makes Language Models Better Few-Shot Learners"☆109Jul 15, 2023Updated 3 years ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- Rigourous evaluation of LLM-synthesized code - NeurIPS 2023 & COLM 2024☆1,837Oct 2, 2025Updated last year
- [EMNLP'23] Execution-Based Evaluation for Open Domain Code Generation☆49Dec 22, 2023Updated 2 years ago
- ☆17Dec 9, 2022Updated 3 years ago
- [ICLR 2023] Code for the paper "Binding Language Models in Symbolic Languages"☆325Aug 25, 2023Updated 3 years ago
- Lyra: A Benchmark for Turducken-Style Code Generation☆15Apr 22, 2022Updated 4 years ago
- 🐙 OctoPack: Instruction Tuning Code Large Language Models☆480Feb 5, 2025Updated last year
- Code for the paper "Evaluating Large Language Models Trained on Code"☆3,417Oct 16, 2026Updated last week
- ☆14Aug 18, 2022Updated 4 years ago
- [ICLR'25] BigCodeBench: Benchmarking Code Generation Towards AGI☆516Jan 3, 2026Updated 9 months ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- CRUXEval: Code Reasoning, Understanding, and Execution Evaluation☆174Oct 11, 2024Updated 2 years ago
- Code for the ICLR 2019 paper "Learning to Represent Edits"☆14Dec 8, 2022Updated 3 years ago
- code for "Natural Language to Code Translation with Execution"☆41Nov 2, 2022Updated 3 years ago
- Code for generating the JuICe dataset.☆37Oct 27, 2021Updated 5 years ago
- APPS: Automated Programming Progress Standard (NeurIPS 2021)☆542Jun 19, 2024Updated 2 years ago
- 🧭 SailCompass: Towards Reproducible and Robust Evaluation for Southeast Asian Languages☆13Feb 12, 2025Updated last year
- [EMNLP 2022] Unifying and multi-tasking structured knowledge grounding with language models☆565Aug 22, 2023Updated 3 years ago
- Paper collections of methods that using language to interact with environment, including interact with real world, simulated world or WWW…☆128Jul 26, 2023Updated 3 years ago
- Mapping Language to Code in a Programmatic Context☆80Jan 27, 2021Updated 5 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- CliniQG4QA: Generating Diverse Questions for Domain Adaptation of Clinical Question Answering☆23Feb 26, 2021Updated 5 years ago
- A MBTI test on Large Language Model like GPT-3.☆28May 2, 2022Updated 4 years ago
- ☆58May 28, 2024Updated 2 years ago
- Official repository for the paper "LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code"☆970Jul 16, 2025Updated last year
- A multi-programming language benchmark for LLMs☆317Apr 12, 2026Updated 6 months ago
- Contests based Dataset for Code Generation☆13Dec 11, 2022Updated 3 years ago
- ☆10Apr 15, 2023Updated 3 years ago
- We introduce FixEval , a dataset for competitive programming bug fixing along with a comprehensive test suite and show the necessity of e…☆26Aug 31, 2022Updated 4 years ago
- [ACL '24] Source code for paper: INTERVENOR : Prompt the Coding Ability of Large Language Models with the Interactive Chain of Repairing☆28Nov 25, 2024Updated last year
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- ☆20Aug 9, 2024Updated 2 years ago
- [ACL 2024] Code for the paper "ALaRM: Align Language Models via Hierarchical Rewards Modeling"☆25Mar 28, 2024Updated 2 years ago
- ☆677Nov 1, 2024Updated last year
- A containerized HTTP server that executes code in many languages☆11Jul 19, 2024Updated 2 years ago
- [ICLR 2024] Lemur: Open Foundation Models for Language Agents☆555Oct 28, 2023Updated 3 years ago
- This is the official code for the paper CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning (Neur…☆579Jun 2, 2026Updated 4 months ago
- CodeXGLUE☆1,842Apr 23, 2024Updated 2 years ago