METR / eval-analysis-publicLinks
Public repository containing METR's DVC pipeline for eval data analysis
☆86Updated 4 months ago
Alternatives and similar repositories for eval-analysis-public
Users that are interested in eval-analysis-public are comparing it to the libraries listed below
Sorting:
- Source code of "How to Correctly do Semantic Backpropagation on Language-based Agentic Systems" 🤖☆72Updated 8 months ago
- Official homepage for "Self-Harmonized Chain of Thought" (NAACL 2025)☆91Updated 6 months ago
- A framework for pitting LLMs against each other in an evolving library of games ⚔☆34Updated 3 months ago
- QAlign is a new test-time alignment approach that improves language model performance by using Markov chain Monte Carlo methods.☆23Updated 4 months ago
- Matrix (Multi-Agent daTa geneRation Infra and eXperimentation framework) is a versatile engine for multi-agent conversational data genera…☆81Updated last week
- Analysis code for paper "SciArena: An Open Evaluation Platform for Foundation Models in Scientific Literature Tasks"☆45Updated this week
- An attribution library for LLMs☆42Updated 10 months ago
- Train your own SOTA deductive reasoning model☆103Updated 5 months ago
- A DSPy-based implementation of the tree of thoughts method (Yao et al., 2023) for generating persuasive arguments☆87Updated 10 months ago
- A framework for optimizing DSPy programs with RL☆96Updated this week
- ReDel is a toolkit for researchers and developers to build, iterate on, and analyze recursive multi-agent systems. (EMNLP 2024 Demo)☆83Updated 4 months ago
- Code for "Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs"☆54Updated 5 months ago
- LLM reads a paper and produce a working prototype☆58Updated 3 months ago
- Automating enterprise workflows with multimodal agents☆108Updated 10 months ago
- A library for benchmarking the Long Term Memory and Continual learning capabilities of LLM based agents. With all the tests and code you…☆76Updated 7 months ago
- accompanying material for sleep-time compute paper☆99Updated 3 months ago
- ☆40Updated 7 months ago
- ☆73Updated 5 months ago
- ☆102Updated last week
- Automated Capability Discovery via Foundation Model Self-Exploration☆59Updated 5 months ago
- ☆44Updated 3 months ago
- Training an LLM to use a calculator with multi-turn reinforcement learning, achieving a **62% absolute increase in evaluation accuracy**.☆45Updated 3 months ago
- Prompts used in the Automated Auditing Blog Post☆78Updated 2 weeks ago
- Code for our paper PAPILLON: PrivAcy Preservation from Internet-based and Local Language MOdel ENsembles☆53Updated 3 months ago
- The Granite Guardian models are designed to detect risks in prompts and responses.☆93Updated this week
- Experimental Code for StructuredRAG: JSON Response Formatting with Large Language Models☆111Updated 3 months ago
- ☆71Updated last week
- A better way of testing, inspecting, and analyzing AI Agent traces.☆39Updated last month
- ☆65Updated 2 months ago
- A framework for pitting LLMs against each other in an evolving library of games ⚔☆32Updated 3 months ago