[ICLR 2025] 🚀 CodeMMLU Evaluator: A framework for evaluating LM models on CodeMMLU MCQs benchmark.
☆30Apr 21, 2025Updated last year
Alternatives and similar repositories for CodeMMLU
Users that are interested in CodeMMLU are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ⚒️ Tree-sitter custom toolkit for extracting function and class from raw source file☆53Jul 1, 2024Updated 2 years ago
- [ACL 2024] Novel reranking method to select the best solutions for code generation☆16Jun 9, 2024Updated 2 years ago
- Dataset and Code for NeurIPS 2023 paper "Language-driven Scene Synthesis using Multi-conditional Diffusion Model."☆47Aug 8, 2024Updated 2 years ago
- [EMNLP 2023] The Vault: A Comprehensive Multilingual Dataset for Advancing Code Understanding and Generation☆105Aug 21, 2024Updated 2 years ago
- Generalist Software Agents to Solve Soware Engineering Tasks☆256Dec 10, 2024Updated last year
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- [NAACL 2025] Benchmark for Repository-Level Code Generation, focus on Executability, Correctness from Test Cases and Usage of Contexts fr…☆164Jan 8, 2026Updated 9 months ago
- Dependency call extraction tool for Python modules (function, class, import statement)☆13Aug 7, 2024Updated 2 years ago
- ☆26Jul 19, 2022Updated 4 years ago
- [NeurIPS 2026] Official implementaion for Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs☆18Oct 7, 2026Updated 3 weeks ago
- Treasury and Liquidity Management (TLM) - This is a highly complex, critical back-office function. Agentic systems can optimize cash flow…☆11Jul 7, 2025Updated last year
- [EMNLP 2026] A multilingual benchmark for repository-level unit test generation, evaluating LLMs across 5 programming languages and real-…☆168Aug 27, 2026Updated 2 months ago
- Language Model for Mainframe Modernization☆74Aug 23, 2024Updated 2 years ago
- [FORGE 2025] Incorporating Agile methodology into agents to create complex real-world softwares☆467Oct 15, 2024Updated 2 years ago
- A collection of recent papers, benchmarks and datasets of AI4Code domain.☆61Apr 23, 2024Updated 2 years ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- [TOSEM 2026] Datasets and models for TD detection in Java using comment and source code☆127Feb 10, 2026Updated 8 months ago
- Code implementation for CoTexT: Multi-task Learning with Code-Text Transformer☆35Sep 14, 2021Updated 5 years ago
- Awesome paper list for reasoning generalization in LLMs.☆160Oct 13, 2026Updated 2 weeks ago
- A Comparative Study of Various Code Embeddings in Software Semantic Matching☆18Dec 8, 2022Updated 3 years ago
- ☆10Aug 25, 2020Updated 6 years ago
- [FSE 2026] Do Not Treat Code as Natural Language: Implications for Repository-Level Code Generation and Beyond☆140Jun 4, 2026Updated 4 months ago
- Repository of the paper 'CodeQueries: A Dataset of Semantic Queries over Code' published in ISEC 2024☆13Apr 21, 2024Updated 2 years ago
- Program Translator AI built on Pytorch☆15Dec 19, 2019Updated 6 years ago
- IT jobs in South Korea☆10Jul 5, 2016Updated 10 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆17Jun 18, 2024Updated 2 years ago
- Programs for producing static and dynamic (runtime) call graphs for Java programs☆15Aug 23, 2021Updated 5 years ago
- The dataset, source code and the results of our ESEC/FSE 2023 paper "An Extensive Study on Adversarial Attack against Pre-trained Models …☆15Feb 19, 2024Updated 2 years ago
- Generating Adversarial Examples for Holding Robustness of Source Code Processing Models☆17Dec 2, 2021Updated 4 years ago
- ☆143Jun 6, 2025Updated last year
- Sketch Driven Regular Expression Generation.☆18Apr 26, 2023Updated 3 years ago
- Code for Doubly deformable aggregation of covariance matrices for few-shot segmentation☆16Oct 25, 2022Updated 4 years ago
- Preserving Generalization of Language Models in Few-shot Continual Relation Extraction (EMNLP2024)☆17Nov 21, 2024Updated last year
- Code and data for XLCoST: A Benchmark Dataset for Cross-lingual Code Intelligence☆93Jan 21, 2025Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Visualize expert firing frequencies across sentences in the Mixtral MoE model☆18Dec 22, 2023Updated 2 years ago
- Set of tools to help working with "Big Code"☆42Apr 28, 2022Updated 4 years ago
- AutoPruner: Transformer-based Call Graph Pruning (ESEC/FSE 2022, Research Track)☆22Dec 7, 2023Updated 2 years ago
- ☆23Sep 18, 2023Updated 3 years ago
- ☆17Mar 20, 2025Updated last year
- Pytorch implementation for "Video Joint Modelling Based on Hierarchical Transformer for Co-summarization"☆15Aug 24, 2025Updated last year
- Generating Sentences from Disentangled Syntactic and Semantic Spaces☆11Jun 24, 2019Updated 7 years ago