FSoft-AI4Code / CodeMMLULinks
[ICLR 2025] 🚀 CodeMMLU Evaluator: A framework for evaluating LM models on CodeMMLU MCQs benchmark.
☆24Updated 5 months ago
Alternatives and similar repositories for CodeMMLU
Users that are interested in CodeMMLU are comparing it to the libraries listed below
Sorting:
- ☆99Updated 11 months ago
- Systematic evaluation framework that automatically rates overthinking behavior in large language models.☆93Updated 4 months ago
- Training and Benchmarking LLMs for Code Preference.☆36Updated 10 months ago
- [EMNLP'23] Execution-Based Evaluation for Open Domain Code Generation☆49Updated last year
- RepoQA: Evaluating Long-Context Code Understanding☆117Updated 11 months ago
- InstructCoder: Instruction Tuning Large Language Models for Code Editing | Oral ACL-2024 srw☆62Updated last year
- [EMNLP 2023] The Vault: A Comprehensive Multilingual Dataset for Advancing Code Understanding and Generation☆100Updated last year
- Evaluating LLMs with fewer examples☆161Updated last year
- Replicating O1 inference-time scaling laws☆90Updated 10 months ago
- Code for the paper "Fishing for Magikarp"☆170Updated 4 months ago
- Astraios: Parameter-Efficient Instruction Tuning Code Language Models☆62Updated last year
- Official code for the paper "CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modules"☆46Updated 8 months ago
- ☆41Updated last year
- Code for paper "LEVER: Learning to Verifiy Language-to-Code Generation with Execution" (ICML'23)☆90Updated 2 years ago
- ☆40Updated 3 months ago
- [FORGE 2025] Graph-based method for end-to-end code completion with context awareness on repository☆66Updated last year
- [NeurIPS 2024] Goldfish Loss: Mitigating Memorization in Generative LLMs☆92Updated 10 months ago
- CRUXEval: Code Reasoning, Understanding, and Execution Evaluation☆153Updated 11 months ago
- ☆47Updated 6 months ago
- StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback☆71Updated last year
- Code for reproducing our paper "Are Sparse Autoencoders Useful? A Case Study in Sparse Probing"☆29Updated 6 months ago
- ☆39Updated 6 months ago
- ☆128Updated last year
- Official repository for "Scaling Retrieval-Based Langauge Models with a Trillion-Token Datastore".☆216Updated 2 months ago
- A distributed, extensible, secure solution for evaluating machine generated code with unit tests in multiple programming languages.☆56Updated 11 months ago
- Code for PHATGOOSE introduced in "Learning to Route Among Specialized Experts for Zero-Shot Generalization"☆89Updated last year
- ☆28Updated this week
- ☆123Updated 7 months ago
- Public code repo for paper "SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales"☆109Updated last year
- ☆74Updated last month