Public repository containing METR's DVC pipeline for eval data analysis
☆296Mar 6, 2026Updated 4 months ago
Alternatives and similar repositories for eval-analysis-public
Users that are interested in eval-analysis-public are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Vivaria is METR's tool for running evaluations and conducting agent elicitation research.☆139May 18, 2026Updated last month
- NeurIPS 2024: SciFIBench: Benchmarking Large Multimodal Models for Scientific Figure Interpretation☆13May 24, 2025Updated last year
- ☆27Apr 1, 2026Updated 3 months ago
- ☆25May 25, 2024Updated 2 years ago
- ☆11Jun 2, 2021Updated 5 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆49May 17, 2026Updated last month
- A Kubernetes sandbox environment for use with inspect_ai☆32Updated this week
- [ICML 2025] EffiCoder: Enhancing Code Generation in Large Language Models through Efficiency-Aware Fine-tuning☆16May 24, 2025Updated last year
- ☆34Jun 4, 2025Updated last year
- ☆73Jun 16, 2026Updated 2 weeks ago
- A lightweight library for Bayesian analysis of LLM evals (ICML 2025 Spotlight Position Paper)☆25May 28, 2025Updated last year
- ☆271Apr 22, 2026Updated 2 months ago
- UQ: Assessing Language Models on Unsolved Questions☆30Aug 26, 2025Updated 10 months ago
- ☆10Mar 13, 2023Updated 3 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- An alignment auditing agent capable of quickly exploring alignment hypothesis☆1,242Jun 26, 2026Updated last week
- Original code base for On Pretraining Data Diversity for Self-Supervised Learning☆14Dec 30, 2024Updated last year
- Python package to download and use the SSB datasets☆11Aug 3, 2023Updated 2 years ago
- Code for Preventing Language Models From Hiding Their Reasoning, which evaluates defenses against LLM steganography.☆25Jan 26, 2024Updated 2 years ago
- Repository to create traveling waves integrate special information through time☆58Aug 8, 2025Updated 10 months ago
- Indexing framework designed for the automated creation of structured knowledge bases in Azure AI Search☆15Jun 10, 2026Updated 3 weeks ago
- METR Task Standard☆181Feb 3, 2025Updated last year
- Reward Learning by Simulating the Past☆46May 9, 2019Updated 7 years ago
- Vision Large Language Models trained on M3IT instruction tuning dataset☆17Aug 16, 2023Updated 2 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Official code for the paper: "Metadata Archaeology"☆19May 10, 2023Updated 3 years ago
- ☆116Jul 2, 2024Updated 2 years ago
- Byte-sized text games for code generation tasks on virtual environments☆20Jul 8, 2024Updated last year
- [NAACL'25] "Revealing the Barriers of Language Agents in Planning"☆13Jun 22, 2025Updated last year
- Library for sequence-to-sequence numeric prediction, applicable to any tokenizable input, and allows pretraining and fine-tuning over mul…☆347Jun 24, 2026Updated last week
- Code for the paper "Refining Language Model with Compositional Explanation" (NeurIPS 2021)☆11Oct 25, 2021Updated 4 years ago
- ☆18Jun 23, 2026Updated last week
- This repo contains the dataset and code for the paper "SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software E…☆1,437Jul 18, 2025Updated 11 months ago
- A suite of open-ended, non-imitative tasks involving generalizable skills for large language model chatbots and agents to enable bootstra…☆49Jan 31, 2025Updated last year
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- [LREC-Coling 2024] PECC: Problem Extraction and Coding Challenges☆14May 30, 2024Updated 2 years ago
- Ranking LLMs on agentic tasks☆222May 21, 2026Updated last month
- Finding trojans in aligned LLMs. Official repository for the competition hosted at SaTML 2024.☆118Jun 13, 2024Updated 2 years ago
- A manuscript exploring the effects of taxonomic bias on microbiome differential-abundance analysis.☆12Oct 30, 2023Updated 2 years ago
- ☆126Jun 10, 2026Updated 3 weeks ago
- Python script to download all historical FX data from www.truefx.com☆10May 22, 2023Updated 3 years ago
- Codebase for fine-tuning Llama2 70B to generate math test questions and answers.☆11Aug 30, 2024Updated last year