A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories
☆43Sep 4, 2024Updated 2 years ago
Alternatives and similar repositories for DevEval
Users that are interested in DevEval are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- An Evolving Code Generation Benchmark Aligned with Real-world Code Repositories☆72Aug 15, 2024Updated 2 years ago
- ☆67Jun 19, 2024Updated 2 years ago
- ☆28Jul 20, 2024Updated 2 years ago
- Code and data for the ICSE 2025 paper "HumanEvo: An Evolving-aware Benchmark for More Realistic Evaluation of Repository-level Code Gener…☆13Feb 10, 2025Updated last year
- A collection of practical code generation tasks and tests in open source projects. Complementary to HumanEval by OpenAI.☆160Dec 25, 2024Updated last year
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- SolEval is the 1st evaluation framework designed to benchmark Large Language Models (LLMs) for generating Solidity smart contracts at the…☆14Feb 10, 2026Updated 8 months ago
- A Comprehensive Benchmark for Software Development.☆139May 30, 2024Updated 2 years ago
- [ACL25] FEA-Bench: A Benchmark for Evaluating Repository-Level Code Generation for Feature Implementation☆63Jan 28, 2026Updated 9 months ago
- ☆14Jul 26, 2025Updated last year
- [NAACL 2025] Benchmark for Repository-Level Code Generation, focus on Executability, Correctness from Test Cases and Usage of Contexts fr…☆164Jan 8, 2026Updated 9 months ago
- Benchmark ClassEval for class-level code generation.☆154Oct 24, 2024Updated 2 years ago
- ☆14Jul 22, 2021Updated 5 years ago
- Prism: Revealing Hidden Functional Clusters from Massive Instances in Cloud Systems [ASE'2023]☆10Jan 10, 2024Updated 2 years ago
- "Why do I feel offended?" - Korean Dataset for Offensive Language Identification (EACL2023 Findings)☆16May 14, 2023Updated 3 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Official implementation of paper How to Understand Whole Repository? New SOTA on SWE-bench Lite (21.3%)☆97Mar 26, 2025Updated last year
- ☆27Sep 23, 2024Updated 2 years ago
- ☆13Jul 21, 2026Updated 3 months ago
- ☆11May 29, 2023Updated 3 years ago
- ☆11May 18, 2024Updated 2 years ago
- The Infibench variant of bigcode-evaluation-harness --- a framework for the evaluation of autoregressive code generation language models.☆14Oct 19, 2024Updated 2 years ago
- A recommender system that recommends cryptocurrencies to investors.☆10Mar 26, 2021Updated 5 years ago
- Source code for EMNLP'25 paper "CodeRAG: Finding Relevant and Necessary Knowledge for Retrieval-Augmented Repository-Level Code Completio…☆25Apr 15, 2026Updated 6 months ago
- ☆31Oct 26, 2026Updated last week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆33Jan 14, 2025Updated last year
- FORTE (Full-cycle Office Real-world Task Evaluation) is a general agent benchmark for evaluating AI agents on daily office productivity a…☆20Jun 30, 2026Updated 4 months ago
- ☆38Apr 8, 2026Updated 6 months ago
- [IJCAI 2020] Metamorphic Testing and Certi fied Mitigation of Fairness Violations in NLP Models☆10Jul 17, 2020Updated 6 years ago
- CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completion (NeurIPS 2023)☆184Aug 15, 2025Updated last year
- obsidian的自用插件☆15Jul 23, 2026Updated 3 months ago
- ☆32Oct 28, 2023Updated 3 years ago
- The official codes for our paper at COLING 2022: Semantic-Preserving Adversarial Code Comprehension☆12Oct 23, 2022Updated 4 years ago
- [ICSE 2024] Code Search is All You Need? Improving Code Suggestions With Code Search☆13Apr 24, 2024Updated 2 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- [NeurIPS 2024] OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI☆106Mar 6, 2025Updated last year
- ☆28Jun 2, 2026Updated 5 months ago
- Official code for the paper "CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modules"☆48Jun 2, 2026Updated 5 months ago
- Source code for EMSE 2023 paper "Zero-Shot Code Representation Learning via Prompt Tuning"☆13Feb 15, 2023Updated 3 years ago
- 学习过程中积累的一些笔记☆34Mar 9, 2023Updated 3 years ago
- Official implementation of Concept-as-Tree: A Controllable Synthetic Data Framework Makes Stronger Personalized VLMs.☆12Jun 28, 2026Updated 4 months ago
- MCP server for comparing prices across Taiwan e-commerce platforms (Coupang, momo, PChome)☆18Sep 8, 2026Updated last month