evo-eval / evoevalLinks
EvoEval: Evolving Coding Benchmarks via LLM
☆80Updated last year
Alternatives and similar repositories for evoeval
Users that are interested in evoeval are comparing it to the libraries listed below
Sorting:
- Can It Edit? Evaluating the Ability of Large Language Models to Follow Code Editing Instructions☆48Updated 3 months ago
- ✨ RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems - ICLR 2024☆182Updated last year
- RepoQA: Evaluating Long-Context Code Understanding☆127Updated last year
- CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completion (NeurIPS 2023)☆164Updated 4 months ago
- CRUXEval: Code Reasoning, Understanding, and Execution Evaluation☆163Updated last year
- [NeurIPS 2024] Evaluation harness for SWT-Bench, a benchmark for evaluating LLM repository-level test-generation☆64Updated last week
- The repository for paper "DebugBench: "Evaluating Debugging Capability of Large Language Models".☆85Updated last year
- Benchmark ClassEval for class-level code generation.☆146Updated last year
- Repilot, a patch generation tool introduced in the ESEC/FSE'23 paper "Copiloting the Copilots: Fusing Large Language Models with Completi…☆137Updated 2 years ago
- Enhancing AI Software Engineering with Repository-level Code Graph☆240Updated 8 months ago
- [ICML '24] R2E: Turn any GitHub Repository into a Programming Agent Environment☆138Updated 8 months ago
- Dianshu-Liao / AAA-Code-Generation-Framework-for-Code-Repository-Local-Aware-Global-Aware-Third-Party-Aware☆24Updated 2 years ago
- InstructCoder: Instruction Tuning Large Language Models for Code Editing | Oral ACL-2024 srw☆64Updated last year
- An Evolving Code Generation Benchmark Aligned with Real-world Code Repositories☆67Updated last year
- ☆78Updated last year
- [LREC-COLING'24] HumanEval-XL: A Multilingual Code Generation Benchmark for Cross-lingual Natural Language Generalization☆40Updated 9 months ago
- ☆127Updated 2 years ago
- Open sourced predictions, execution logs, trajectories, and results from model inference + evaluation runs on the SWE-bench task.☆231Updated last week
- TDD-Bench-Verified is a new benchmark for generating test cases for test-driven development (TDD)☆25Updated 3 months ago
- Dataflow-guided retrieval augmentation for repository-level code completion, ACL 2024 (main)☆30Updated 9 months ago
- We introduce FixEval , a dataset for competitive programming bug fixing along with a comprehensive test suite and show the necessity of e…☆24Updated 3 years ago
- XFT: Unlocking the Power of Code Instruction Tuning by Simply Merging Upcycled Mixture-of-Experts☆35Updated last year
- ☆112Updated last year
- Official code for the paper "CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modules"☆48Updated last month
- ☆68Updated last year
- A distributed, extensible, secure solution for evaluating machine generated code with unit tests in multiple programming languages.☆61Updated last year
- Artifact repository for the paper "Lost in Translation: A Study of Bugs Introduced by Large Language Models while Translating Code", In P…☆51Updated 8 months ago
- Code and data for XLCoST: A Benchmark Dataset for Cross-lingual Code Intelligence☆89Updated 11 months ago
- ☆36Updated 2 years ago
- Releasing code for "ReCode: Robustness Evaluation of Code Generation Models"☆56Updated last year