METR / eval-analysis-public
Public repository containing METR's DVC pipeline for eval data analysis
☆56Updated last month
Alternatives and similar repositories for eval-analysis-public
Users that are interested in eval-analysis-public are comparing it to the libraries listed below
Sorting:
- A framework for pitting LLMs against each other in an evolving library of games ⚔☆31Updated 3 weeks ago
- Interaction-first method for generating demonstrations for web-agents on any website☆38Updated 2 weeks ago
- A better way of testing, inspecting, and analyzing AI Agent traces.☆35Updated last week
- ☆18Updated 7 months ago
- ☆14Updated last month
- Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optuna☆52Updated 3 months ago
- LMQL implementation of tree of thoughts☆34Updated last year
- Automatic Prompt Optimization☆35Updated last year
- ☆46Updated this week
- The official evaluation suite and dynamic data release for MixEval.☆11Updated 7 months ago
- ReDel is a toolkit for researchers and developers to build, iterate on, and analyze recursive multi-agent systems. (EMNLP 2024 Demo)☆77Updated last month
- [ACL 2024] Do Large Language Models Latently Perform Multi-Hop Reasoning?☆65Updated last month
- ☆22Updated this week
- A tree-based prefix cache library that allows rapid creation of looms: hierarchal branching pathways of LLM generations.☆68Updated 3 months ago
- ☆37Updated 3 months ago
- ☆39Updated last week
- Source code of "How to Correctly do Semantic Backpropagation on Language-based Agentic Systems" 🤖☆67Updated 5 months ago
- accompanying material for sleep-time compute paper☆82Updated 2 weeks ago
- ☆41Updated 5 months ago
- Mixing Language Models with Self-Verification and Meta-Verification☆104Updated 5 months ago
- Train your own SOTA deductive reasoning model☆92Updated 2 months ago
- Matrix (Multi-Agent daTa geneRation Infra and eXperimentation framework) is a versatile engine for multi-agent conversational data genera…☆33Updated this week
- QAlign is a new test-time alignment approach that improves language model performance by using Markov chain Monte Carlo methods.☆23Updated last month
- ☆38Updated 9 months ago
- oda-r is a professional-grade compiler for Declarative Self-improving Python (DSPy), featuring comprehensive error handling, logging, and…☆36Updated 3 months ago
- ☆31Updated 3 weeks ago
- ☆45Updated 7 months ago
- Just a bunch of benchmark logs for different LLMs☆119Updated 9 months ago
- ☆28Updated 7 months ago
- Creating Generative AI Apps which work☆17Updated last month