ntunlp / ExecEvalLinks
A distributed, extensible, secure solution for evaluating machine generated code with unit tests in multiple programming languages.
☆59Updated last year
Alternatives and similar repositories for ExecEval
Users that are interested in ExecEval are comparing it to the libraries listed below
Sorting:
- xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval☆87Updated last year
- CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completion (NeurIPS 2023)☆164Updated 3 months ago
- CRUXEval: Code Reasoning, Understanding, and Execution Evaluation☆161Updated last year
- Accepted by Transactions on Machine Learning Research (TMLR)☆136Updated last year
- ☆127Updated 2 years ago
- ✨ RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems - ICLR 2024☆181Updated last year
- InstructCoder: Instruction Tuning Large Language Models for Code Editing | Oral ACL-2024 srw☆64Updated last year
- ☆54Updated last year
- [ICML 2023] Data and code release for the paper "DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation".☆259Updated last year
- StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback☆73Updated last year
- Code for the TMLR 2023 paper "PPOCoder: Execution-based Code Generation using Deep Reinforcement Learning"☆117Updated last year
- Repoformer: Selective Retrieval for Repository-Level Code Completion (ICML 2024)☆61Updated 5 months ago
- NaturalCodeBench (Findings of ACL 2024)☆67Updated last year
- Code for paper "LEVER: Learning to Verifiy Language-to-Code Generation with Execution" (ICML'23)☆90Updated 2 years ago
- ☆111Updated last year
- ☆41Updated last year
- Astraios: Parameter-Efficient Instruction Tuning Code Language Models☆63Updated last year
- evol augment any dataset online☆61Updated 2 years ago
- ☆28Updated last week
- The repository for paper "DebugBench: "Evaluating Debugging Capability of Large Language Models".☆84Updated last year
- A multi-programming language benchmark for LLMs☆283Updated 2 weeks ago
- Reinforcement Learning for Repository-Level Code Completion☆42Updated last year
- An open-source library for contamination detection in NLP datasets and Large Language Models (LLMs).☆57Updated last year
- [ICLR 2024] Evaluating Large Language Models at Evaluating Instruction Following☆134Updated last year
- Official Repo for ICLR 2024 paper MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback by Xingyao Wang*, Ziha…☆134Updated last year
- Training and Benchmarking LLMs for Code Preference.☆37Updated last year
- ToolBench, an evaluation suite for LLM tool manipulation capabilities.☆164Updated last year
- A dataset of LLM-generated chain-of-thought steps annotated with mistake location.☆84Updated last year
- ☆33Updated 2 months ago
- Open Source WizardCoder Dataset☆162Updated 2 years ago