METR / eval-analysis-publicLinks
Public repository containing METR's DVC pipeline for eval data analysis
☆124Updated 6 months ago
Alternatives and similar repositories for eval-analysis-public
Users that are interested in eval-analysis-public are comparing it to the libraries listed below
Sorting:
- Source code for the collaborative reasoner research project at Meta FAIR.☆103Updated 6 months ago
- Train your own SOTA deductive reasoning model☆109Updated 7 months ago
- accompanying material for sleep-time compute paper☆117Updated 6 months ago
- Code for "Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs"☆76Updated 8 months ago
- Matrix (Multi-Agent daTa geneRation Infra and eXperimentation framework) is a versatile engine for multi-agent conversational data genera…☆97Updated last week
- Experimental Code for StructuredRAG: JSON Response Formatting with Large Language Models☆110Updated 6 months ago
- Source code of "How to Correctly do Semantic Backpropagation on Language-based Agentic Systems" 🤖☆76Updated 10 months ago
- Super basic implementation (gist-like) of RLMs with REPL environments.☆204Updated last week
- A DSPy-based implementation of the tree of thoughts method (Yao et al., 2023) for generating persuasive arguments☆90Updated 3 weeks ago
- A library for benchmarking the Long Term Memory and Continual learning capabilities of LLM based agents. With all the tests and code you…☆79Updated 10 months ago
- ☆172Updated last week
- ReDel is a toolkit for researchers and developers to build, iterate on, and analyze recursive multi-agent systems. (EMNLP 2024 Demo)☆88Updated last month
- ☆59Updated 9 months ago
- Archon provides a modular framework for combining different inference-time techniques and LMs with just a JSON config file.☆188Updated 7 months ago
- Training setup for Langchain's Open Deep Research☆67Updated 2 months ago
- Official homepage for "Self-Harmonized Chain of Thought" (NAACL 2025)☆91Updated 9 months ago
- SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?☆202Updated this week
- Prompts used in the Automated Auditing Blog Post☆114Updated 3 months ago
- LLM reads a paper and produce a working prototype☆57Updated 6 months ago
- Analysis code for Neurips 2025 paper "SciArena: An Open Evaluation Platform for Foundation Models in Scientific Literature Tasks"☆53Updated 2 months ago
- Mixing Language Models with Self-Verification and Meta-Verification☆109Updated 10 months ago
- ☆135Updated 7 months ago
- A framework for pitting LLMs against each other in an evolving library of games ⚔☆33Updated 6 months ago
- ☆40Updated 10 months ago
- Just a bunch of benchmark logs for different LLMs☆118Updated last year
- Training an LLM to use a calculator with multi-turn reinforcement learning, achieving a **62% absolute increase in evaluation accuracy**.☆57Updated 5 months ago
- Code for our paper PAPILLON: PrivAcy Preservation from Internet-based and Local Language MOdel ENsembles☆59Updated 5 months ago
- A framework for optimizing DSPy programs with RL☆208Updated last week
- ☆68Updated 5 months ago
- The Granite Guardian models are designed to detect risks in prompts and responses.☆119Updated 3 weeks ago