We introduce FixEval , a dataset for competitive programming bug fixing along with a comprehensive test suite and show the necessity of execution based evaluation compared to suboptimal match based evaluation metrics like BLEU, CodeBLEU, Syntax Match, Exact Match etc.
☆26Aug 31, 2022Updated 4 years ago
Alternatives and similar repositories for FixEval
Users that are interested in FixEval are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Contests based Dataset for Code Generation☆13Dec 11, 2022Updated 3 years ago
- Contains the code and data for our #ICSE2022 paper titled as "CodeFill: Multi-token Code Completion by Jointly Learning from Structure an…☆15May 18, 2022Updated 4 years ago
- LLM inference with HuggingFace (experimental)☆11Feb 6, 2025Updated last year
- Aix-bench, the Java benchmark for code synthesis problem.☆52Aug 19, 2022Updated 4 years ago
- Replication package for EMNLP2022 paper- RACE: Retrieval-Augmented Commit Message Generation☆19Oct 21, 2022Updated 4 years ago
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- pytorch版损失函数，改写自科学空间文章，【通过互信息思想来缓解类别不平衡问题】、【将“softmax+交叉熵”推广到多标签分类问题】☆12Aug 22, 2021Updated 5 years ago
- The RunBugRun dataset of executable bugs☆27Sep 24, 2025Updated last year
- This is the official implement for the paper 'Domain Adaptive Code Completion via Language Models and Decoupled Domain Databases''☆14Oct 4, 2023Updated 3 years ago
- ☆43Jul 13, 2026Updated 3 months ago
- ☆16Dec 12, 2021Updated 4 years ago
- [COLING25] CodeJudge Eval: Can Large Language Models be Good Judges in Code Understanding?☆12Dec 3, 2024Updated last year
- ☆14Aug 21, 2020Updated 6 years ago
- ☆39Jan 13, 2023Updated 3 years ago
- An empirical study on patch correctness☆15Nov 5, 2022Updated 3 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Information about the CodedotAI reading group sessions.☆13Aug 16, 2021Updated 5 years ago
- ☆10Mar 22, 2022Updated 4 years ago
- Data and code for "Chain-of-Thought in Neural Code Generation: From and For Lightweight Language Models", which accepted in TSE.☆16Jul 3, 2024Updated 2 years ago
- code for "Implant Global and Local Hierarchy Information to Sequence based Code Representation Models"☆12Dec 13, 2024Updated last year
- Towards converting multilingual source code into one language-agnostic graph representation.☆49Mar 22, 2023Updated 3 years ago
- Code of our paper Applying CodeBERT for Automated Program Repair of Java Simple Bugs which is accepted to MSR 2021.☆52Nov 27, 2022Updated 3 years ago
- ☆22Nov 17, 2021Updated 4 years ago
- Code of the COLING22 paper "uChecker: Masked Pretrained Language Models as Unsupervised Chinese Spelling Checkers"☆19Aug 17, 2022Updated 4 years ago
- Efficient Pre-training of Masked Language Model via Concept-based Curriculum Masking☆13Feb 5, 2023Updated 3 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Ying Zhang, Ya Xiao, Md Mahir Asef Kabir, Daphne Yao, Na Meng. Example-Based Vulnerability Detection and Repair in Java Code 30th IEEE/AC…☆10Jul 31, 2022Updated 4 years ago
- ☆13Feb 2, 2023Updated 3 years ago
- Vision Transformer-Inspired Automated Vulnerability Repair☆19May 13, 2025Updated last year
- DataSet and source code for PyART☆12Nov 27, 2022Updated 3 years ago
- ☆17Nov 3, 2022Updated 4 years ago
- This is the project repository of our ASE22 paper: Natural Test Generation for Precise Testing of Question Answering Software☆14Dec 1, 2022Updated 3 years ago
- A collection of practical code generation tasks and tests from open source projects. Complementary to HumanEval by OpenAI.☆24Jan 28, 2023Updated 3 years ago
- 基于CodeBert预训练模型，微调后/直接对目标数据集进行测试☆14Oct 19, 2021Updated 5 years ago
- Data and scripts extending the ManyBugs and Defects4J benchmarks for evaluating applicability of automated program repair techniques.☆16Dec 10, 2017Updated 8 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Source codes for paper ”ReACC: A Retrieval-Augmented Code Completion Framework“☆67Apr 18, 2022Updated 4 years ago
- A multi-lingual program repair benchmark set based on the Quixey Challenge☆148Aug 29, 2022Updated 4 years ago
- open science repo of "Neural Transfer Learning for Repairing Security Vulnerabilities in C Code" https://arxiv.org/pdf/2104.08308☆64Feb 23, 2024Updated 2 years ago
- code and data for paper "Automatic Generation and Summarization of Shellcode via Transformer and Dual Learning", which accepted in SANER …☆12May 8, 2022Updated 4 years ago
- ☆18Aug 11, 2022Updated 4 years ago
- This is the tool released in ICSE 2022 paper "Static Inference Meets Deep Learning: A Hybrid Type Inference Approach for Python"☆45Oct 19, 2023Updated 3 years ago
- ☆17Dec 9, 2022Updated 3 years ago