FSoft-AI4Code / CodeMMLU
🚀 CodeMMLU Evaluator: A framework for evaluating LM models on CodeMMLU MCQs benchmark.
☆17Updated last month
Alternatives and similar repositories for CodeMMLU:
Users that are interested in CodeMMLU are comparing it to the libraries listed below
- [EMNLP 2023] The Vault: A Comprehensive Multilingual Dataset for Advancing Code Understanding and Generation☆86Updated 4 months ago
- Graph-based method for end-to-end code completion with context awareness on repository☆54Updated 4 months ago
- Benchmark for Repository-Level Code Generation, focus on Executability, Correctness from Test Cases and Usage of Contexts from Cross-file…☆22Updated last month
- ☆50Updated 2 months ago
- Training and Benchmarking LLMs for Code Preference.☆28Updated 2 months ago
- ☆116Updated 3 months ago
- [EMNLP'23] Execution-Based Evaluation for Open Domain Code Generation☆45Updated last year
- ☆75Updated last year
- CodeSage: Code Representation Learning At Scale (ICLR 2024)☆89Updated 2 months ago
- Pre-training code for CrystalCoder 7B LLM☆55Updated 8 months ago
- Replicating O1 inference-time scaling laws☆70Updated last month
- Astraios: Parameter-Efficient Instruction Tuning Code Language Models☆57Updated 9 months ago
- ☆38Updated 3 months ago
- CodeUltraFeedback: aligning large language models to coding preferences☆66Updated 6 months ago
- InstructCoder: Instruction Tuning Large Language Models for Code Editing | Oral ACL-2024 srw☆54Updated 3 months ago
- Official repository for "Scaling Retrieval-Based Langauge Models with a Trillion-Token Datastore".☆153Updated last month
- [ACL 2024] Novel reranking method to select the best solutions for code generation☆14Updated 7 months ago
- Code for paper "LEVER: Learning to Verifiy Language-to-Code Generation with Execution" (ICML'23)☆81Updated last year
- Code for the paper "Fishing for Magikarp"☆140Updated this week
- A package dedicated for running benchmark agreement testing☆15Updated last month
- CRUXEval: Code Reasoning, Understanding, and Execution Evaluation☆119Updated 3 months ago
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models"☆58Updated 3 months ago
- ☆52Updated 2 weeks ago
- StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback☆62Updated 4 months ago
- Official code for the paper "CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modules"☆39Updated this week
- [NeurIPS 2024] Goldfish Loss: Mitigating Memorization in Generative LLMs☆81Updated 2 months ago
- This is the official repository for Inheritune.☆109Updated 3 months ago
- The official implementation of the paper "Demystifying the Compression of Mixture-of-Experts Through a Unified Framework".☆52Updated 2 months ago
- B-STAR: Monitoring and Balancing Exploration and Exploitation in Self-Taught Reasoners☆66Updated 2 weeks ago
- ☆22Updated this week