microsoft / prose-benchmarks
PROSE Public Benchmark Suite
☆24Updated 6 months ago
Alternatives and similar repositories for prose-benchmarks:
Users that are interested in prose-benchmarks are comparing it to the libraries listed below
- Astraios: Parameter-Efficient Instruction Tuning Code Language Models☆57Updated 11 months ago
- NaturalCodeBench (Findings of ACL 2024)☆62Updated 5 months ago
- [EMNLP'23] Execution-Based Evaluation for Open Domain Code Generation☆47Updated last year
- InstructCoder: Instruction Tuning Large Language Models for Code Editing | Oral ACL-2024 srw☆58Updated 6 months ago
- ☆23Updated 5 months ago
- ☆44Updated 10 months ago
- ☆75Updated last week
- xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval☆79Updated 6 months ago
- Training and Benchmarking LLMs for Code Preference.☆33Updated 4 months ago
- We introduce FixEval , a dataset for competitive programming bug fixing along with a comprehensive test suite and show the necessity of e…☆22Updated 2 years ago
- Code and dataset for EMNLP 2022 Findings paper "Benchmarking Language Models for Code Syntax Understanding"☆14Updated 2 years ago
- A dataset of LLM-generated chain-of-thought steps annotated with mistake location.☆80Updated 7 months ago
- ☆59Updated 10 months ago
- ☆36Updated 9 months ago
- [ICML 2023] "Outline, Then Details: Syntactically Guided Coarse-To-Fine Code Generation", Wenqing Zheng, S P Sharan, Ajay Kumar Jaiswal, …☆40Updated last year
- [EACL 2024] ICE-Score: Instructing Large Language Models to Evaluate Code☆72Updated 9 months ago
- Language Models of Code are Few-Shot Commonsense Learners (EMNLP 2022)☆86Updated 2 years ago
- Lightweight tool to identify Data Contamination in LLMs evaluation☆49Updated last year
- code for "Natural Language to Code Translation with Execution"☆40Updated 2 years ago
- ☆19Updated 4 months ago
- StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback☆64Updated 7 months ago
- Scripts for downloading and pre-processing the `proof-pile`, a high quality dataset of mathematical text and code.☆19Updated 2 years ago
- ☆26Updated 2 months ago
- CodeUltraFeedback: aligning large language models to coding preferences☆71Updated 9 months ago
- Transformers at any scale☆41Updated last year
- ☆28Updated 4 months ago
- A extension of Transformers library to include T5ForSequenceClassification class.☆38Updated last year
- ☆37Updated 3 months ago
- The LM Contamination Index is a manually created database of contamination evidences for LMs.☆78Updated 11 months ago
- ☆34Updated last year