We introduce FixEval , a dataset for competitive programming bug fixing along with a comprehensive test suite and show the necessity of execution based evaluation compared to suboptimal match based evaluation metrics like BLEU, CodeBLEU, Syntax Match, Exact Match etc.
☆26Aug 31, 2022Updated 3 years ago
Alternatives and similar repositories for FixEval
Users that are interested in FixEval are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Contains the code and data for our #ICSE2022 paper titled as "CodeFill: Multi-token Code Completion by Jointly Learning from Structure an…☆15May 18, 2022Updated 3 years ago
- The RunBugRun dataset of executable bugs☆24Sep 24, 2025Updated 6 months ago
- MODIT: On Multi-Modal Learning of Editing Source Code.☆20Apr 24, 2021Updated 4 years ago
- This is the official implement for the paper 'Domain Adaptive Code Completion via Language Models and Decoupled Domain Databases''☆14Oct 4, 2023Updated 2 years ago
- ☆17Dec 12, 2021Updated 4 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆43Jan 1, 2025Updated last year
- ☆14Aug 21, 2020Updated 5 years ago
- Contests based Dataset for Code Generation☆13Dec 11, 2022Updated 3 years ago
- Information about the CodedotAI reading group sessions.☆12Aug 16, 2021Updated 4 years ago
- ☆22Nov 17, 2021Updated 4 years ago
- An empirical study on patch correctness☆15Nov 5, 2022Updated 3 years ago
- code for "Implant Global and Local Hierarchy Information to Sequence based Code Representation Models"☆12Dec 13, 2024Updated last year
- Vision Transformer-Inspired Automated Vulnerability Repair☆19May 13, 2025Updated 11 months ago
- DataSet and source code for PyART☆11Nov 27, 2022Updated 3 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Ying Zhang, Ya Xiao, Md Mahir Asef Kabir, Daphne Yao, Na Meng. Example-Based Vulnerability Detection and Repair in Java Code 30th IEEE/AC…☆10Jul 31, 2022Updated 3 years ago
- ☆14Feb 2, 2023Updated 3 years ago
- 基于CodeBert预训练模型，微调后/直接对目标数据集进行测试☆14Oct 19, 2021Updated 4 years ago
- Data and scripts extending the ManyBugs and Defects4J benchmarks for evaluating applicability of automated program repair techniques.☆15Dec 10, 2017Updated 8 years ago
- open science repo of "Neural Transfer Learning for Repairing Security Vulnerabilities in C Code" https://arxiv.org/pdf/2104.08308☆63Feb 23, 2024Updated 2 years ago
- ☆18Aug 11, 2022Updated 3 years ago
- ICSE 2021 Artifact for: Shipwright: A Human-in-the-Loop System for Dockerfile Repair.☆23May 11, 2021Updated 4 years ago
- Source codes for paper ”ReACC: A Retrieval-Augmented Code Completion Framework“☆65Apr 18, 2022Updated 3 years ago
- Open-science repo for our experimental results of automatic software repair on the Defects4J benchmark of Java bugs☆37Nov 12, 2025Updated 5 months ago
- Wordpress hosting with auto-scaling - Free Trial • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Source code embeddings for various programming languages☆17Jul 11, 2018Updated 7 years ago
- Trials of pre-trained BERT models for the medical domain in Japanese.☆13Nov 21, 2020Updated 5 years ago
- Magnum-NLC2CMD is the winning solution for the NeurIPS 2020 NLC2CMD challenge.☆33Mar 6, 2023Updated 3 years ago
- code and data for paper "Automatic Generation and Summarization of Shellcode via Transformer and Dual Learning", which accepted in SANER …☆12May 8, 2022Updated 3 years ago
- Code and data for XLCoST: A Benchmark Dataset for Cross-lingual Code Intelligence☆94Jan 21, 2025Updated last year
- A multi-lingual program repair benchmark set based on the Quixey Challenge☆141Aug 29, 2022Updated 3 years ago
- Improving Machine Translation Systems via Isotopic Replacement☆12Apr 14, 2023Updated 3 years ago
- pytorch实现bert做seq2seq任务，使用unilm方案。☆10Apr 1, 2020Updated 6 years ago
- Program Transformation Tool for Java Methods☆11Sep 16, 2022Updated 3 years ago
- Wordpress hosting with auto-scaling - Free Trial • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Replication package for EMNLP2022 paper- RACE: Retrieval-Augmented Commit Message Generation☆19Oct 21, 2022Updated 3 years ago
- ☆127Apr 22, 2023Updated 2 years ago
- Code Generation as a Dual Task of Code Summarization.☆30Jun 28, 2021Updated 4 years ago
- ☆16Nov 3, 2022Updated 3 years ago
- JEMMA: An Extensible Java dataset for Many ML4Code Applications☆19Dec 12, 2022Updated 3 years ago
- ☆41Jan 13, 2023Updated 3 years ago
- Mining tool and large-scale datasets of single statement bug fixes in Python☆19Nov 29, 2023Updated 2 years ago