FSoft-AI4Code / CodeMMLU
🚀 CodeMMLU Evaluator: A framework for evaluating LM models on CodeMMLU MCQs benchmark.
☆13Updated 2 weeks ago
Related projects ⓘ
Alternatives and complementary repositories for CodeMMLU
- [EMNLP 2023] The Vault: A Comprehensive Multilingual Dataset for Advancing Code Understanding and Generation☆84Updated 2 months ago
- Official code for the paper "CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modules"☆34Updated 11 months ago
- Releasing code for "ReCode: Robustness Evaluation of Code Generation Models"☆48Updated 7 months ago
- [ACL 2024] Novel reranking method to select the best solutions for code generation☆14Updated 5 months ago
- ☆25Updated this week
- Graph-based method for end-to-end code completion with context awareness on repository☆44Updated 2 months ago
- Code for paper "LEVER: Learning to Verifiy Language-to-Code Generation with Execution" (ICML'23)☆79Updated last year
- ☆43Updated this week
- Code for the TMLR 2023 paper "PPOCoder: Execution-based Code Generation using Deep Reinforcement Learning"☆96Updated 10 months ago
- ☆99Updated 3 months ago
- Astraios: Parameter-Efficient Instruction Tuning Code Language Models☆57Updated 7 months ago
- [EMNLP'22] Code for 'Exploring Representation-level Augmentation for Code Search'☆25Updated last year
- Replication package for ISSTA2023 paper - Towards Efficient Fine-tuning of Pre-trained Code Models: An Experimental Study and Beyond☆20Updated last year
- A Lightweight Visual Understanding and Reasoning Benchmark for Evaluating Large Multimodal Models through Coding Tasks☆13Updated 2 weeks ago
- evol augment any dataset online☆55Updated last year
- Repoformer: Selective Retrieval for Repository-Level Code Completion (ICML 2024)☆38Updated 4 months ago
- [EACL 2024] ICE-Score: Instructing Large Language Models to Evaluate Code☆69Updated 4 months ago
- ⚒️ Tree-sitter custom toolkit for extracting function and class from raw source file☆39Updated 4 months ago
- A collection of recent papers, benchmarks and datasets of AI4Code domain.☆54Updated 6 months ago
- [NeurIPS 2024 Main Track] Code for the paper titled "Instruction Tuning With Loss Over Instructions"☆25Updated 5 months ago
- Codes for papers on Large Language Models Personalization (LaMP)☆124Updated last month
- InstructCoder: Instruction Tuning Large Language Models for Code Editing | Oral ACL-2024 srw☆52Updated last month
- [NAACL 2024 Outstanding Paper] Source code for the NAACL 2024 paper entitled "R-Tuning: Instructing Large Language Models to Say 'I Don't…☆83Updated 4 months ago
- LongHeads: Multi-Head Attention is Secretly a Long Context Processor☆28Updated 7 months ago
- A distributed, extensible, secure solution for evaluating machine generated code with unit tests in multiple programming languages.☆41Updated 3 weeks ago
- The Paper List on Data Contamination for Large Language Models Evaluation.☆74Updated this week
- Code and data accompanying our paper on arXiv "Faithful Chain-of-Thought Reasoning".☆156Updated 6 months ago
- Critique-out-Loud Reward Models☆36Updated 3 weeks ago
- A repository to perform self-instruct with a model on HF Hub☆31Updated last year
- XFT: Unlocking the Power of Code Instruction Tuning by Simply Merging Upcycled Mixture-of-Experts☆27Updated 4 months ago