mahimanzum / FixEval

We introduce FixEval , a dataset for competitive programming bug fixing along with a comprehensive test suite and show the necessity of execution based evaluation compared to suboptimal match based evaluation metrics like BLEU, CodeBLEU, Syntax Match, Exact Match etc.
22Updated 2 years ago

Related projects

Alternatives and complementary repositories for FixEval