xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval
☆87Sep 17, 2024Updated last year
Alternatives and similar repositories for xCodeEval
Users that are interested in xCodeEval are comparing it to the libraries listed below
Sorting:
- A distributed, extensible, secure solution for evaluating machine generated code with unit tests in multiple programming languages.☆62Oct 21, 2024Updated last year
- A high-throughput and memory-efficient inference and serving engine for LLMs☆17Jun 3, 2024Updated last year
- This repository contains code for cleaning your training data of benchmark data to help combat data snooping.☆27Apr 21, 2023Updated 2 years ago
- Utilities for efficient fine-tuning, inference and evaluation of code generation models☆21Oct 3, 2023Updated 2 years ago
- Official repository for the paper "COAST: Enhancing the Code Debugging Ability of LLMs through Communicative Agent Based Data Synthesis".☆18Feb 19, 2025Updated last year
- ☆14Jul 18, 2025Updated 7 months ago
- [EMNLP'23] Execution-Based Evaluation for Open Domain Code Generation☆49Dec 22, 2023Updated 2 years ago
- ☆15Jun 18, 2024Updated last year
- This is a new metric that can be used to evaluate faithfulness of text generated by LLMs. The work behind this repository can be found he…☆31Aug 25, 2023Updated 2 years ago
- A collection of practical code generation tasks and tests in open source projects. Complementary to HumanEval by OpenAI.☆154Dec 25, 2024Updated last year
- Yet another coding assistant powered by LLM.☆16Sep 11, 2024Updated last year
- Understanding the correlation between different LLM benchmarks☆29Jan 11, 2024Updated 2 years ago
- Training and Benchmarking LLMs for Code Preference.☆38Nov 15, 2024Updated last year
- ☆16Nov 26, 2024Updated last year
- In-Context Alignment: Chat with Vanilla Language Models Before Fine-Tuning☆35Aug 9, 2023Updated 2 years ago
- code for "Natural Language to Code Translation with Execution"☆41Nov 2, 2022Updated 3 years ago
- code2vec: Learning Distributed Representations of Code☆14Jun 27, 2018Updated 7 years ago
- Knowledge transfer from high-resource to low-resource programming languages for Code LLMs☆16Aug 12, 2025Updated 6 months ago
- Training LLaMA language model with MMEngine! It supports LoRA fine-tuning!☆41Apr 2, 2023Updated 2 years ago
- O-GIA is an umbrella for research, infrastructure and projects ecosystem that should provide open source, reproducible datasets, models, …☆87Feb 19, 2023Updated 3 years ago
- Repository for ICSE'22 paper "Recommending Good First Issues in GitHub OSS Projects"☆15Apr 7, 2024Updated last year
- CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completion (NeurIPS 2023)☆174Aug 15, 2025Updated 6 months ago
- No More Manual Tests? Evaluating and Improving ChatGPT for Unit Test Generation☆19Jun 28, 2023Updated 2 years ago
- Code for the TMLR 2023 paper "PPOCoder: Execution-based Code Generation using Deep Reinforcement Learning"☆117Jan 9, 2024Updated 2 years ago
- List of research papers of ICSE, FSE, ASE, and ISSTA since 2020.☆34Dec 29, 2025Updated 2 months ago
- ☆51Jun 21, 2025Updated 8 months ago
- ☆18Apr 15, 2024Updated last year
- Rigourous evaluation of LLM-synthesized code - NeurIPS 2023 & COLM 2024☆1,688Oct 2, 2025Updated 4 months ago
- Code for the NLP4Prog workshop paper "Reading StackOverflow Encourages Cheating: Adding Question TextImproves Extractive Code Generation"☆21Aug 10, 2021Updated 4 years ago
- ☆23Oct 4, 2024Updated last year
- ☆85Jun 13, 2023Updated 2 years ago
- Can It Edit? Evaluating the Ability of Large Language Models to Follow Code Editing Instructions☆48Sep 13, 2025Updated 5 months ago
- Efficient and Scalable Estimation of Tool Representations in Vector Space☆29Sep 5, 2024Updated last year
- ☆49May 13, 2024Updated last year
- Code for paper "LEVER: Learning to Verifiy Language-to-Code Generation with Execution" (ICML'23)☆90Jul 5, 2023Updated 2 years ago
- Mapping Language to Code in a Programmatic Context☆80Jan 27, 2021Updated 5 years ago
- Re-implementation of "CODE2SEQ: GENERATING SEQUENCES FROM STRUCTURED REPRESENTATIONS OF CODE"☆45Jul 25, 2024Updated last year
- Code supporting the paper Graph-Embedding Empowered Entity Retrieval☆24Apr 11, 2025Updated 10 months ago
- Replication of the paper "Structured Neural Summarization" which uses Graph Neural Networks and Seq2Seq models to summarize natural langu…☆21Mar 15, 2019Updated 6 years ago