METR / eval-analysis-publicLinks
Public repository containing METR's DVC pipeline for eval data analysis
☆104Updated 4 months ago
Alternatives and similar repositories for eval-analysis-public
Users that are interested in eval-analysis-public are comparing it to the libraries listed below
Sorting:
- accompanying material for sleep-time compute paper☆105Updated 3 months ago
- Analysis code for paper "SciArena: An Open Evaluation Platform for Foundation Models in Scientific Literature Tasks"☆47Updated 3 weeks ago
- Source code of "How to Correctly do Semantic Backpropagation on Language-based Agentic Systems" 🤖☆73Updated 8 months ago
- Official homepage for "Self-Harmonized Chain of Thought" (NAACL 2025)☆92Updated 7 months ago
- Train your own SOTA deductive reasoning model☆104Updated 5 months ago
- Source code for the collaborative reasoner research project at Meta FAIR.☆103Updated 4 months ago
- A DSPy-based implementation of the tree of thoughts method (Yao et al., 2023) for generating persuasive arguments☆88Updated 10 months ago
- ReDel is a toolkit for researchers and developers to build, iterate on, and analyze recursive multi-agent systems. (EMNLP 2024 Demo)☆84Updated 5 months ago
- Matrix (Multi-Agent daTa geneRation Infra and eXperimentation framework) is a versatile engine for multi-agent conversational data genera…☆88Updated last week
- ☆46Updated 7 months ago
- ☆40Updated 8 months ago
- ☆54Updated 9 months ago
- LLM reads a paper and produce a working prototype☆57Updated 4 months ago
- A framework for pitting LLMs against each other in an evolving library of games ⚔☆33Updated 4 months ago
- Just a bunch of benchmark logs for different LLMs☆120Updated last year
- QAlign is a new test-time alignment approach that improves language model performance by using Markov chain Monte Carlo methods.☆23Updated this week
- ☆119Updated 2 weeks ago
- A library for benchmarking the Long Term Memory and Continual learning capabilities of LLM based agents. With all the tests and code you…☆76Updated 8 months ago
- j1-micro (1.7B) & j1-nano (600M) are absurdly tiny but mighty reward models.☆95Updated last month
- Training an LLM to use a calculator with multi-turn reinforcement learning, achieving a **62% absolute increase in evaluation accuracy**.☆46Updated 3 months ago
- Archon provides a modular framework for combining different inference-time techniques and LMs with just a JSON config file.☆179Updated 5 months ago
- ☆133Updated 5 months ago
- DevQualityEval: An evaluation benchmark 📈 and framework to compare and evolve the quality of code generation of LLMs.☆180Updated 3 months ago
- ☆55Updated 2 months ago
- Mixing Language Models with Self-Verification and Meta-Verification☆105Updated 8 months ago
- look how they massacred my boy☆64Updated 10 months ago
- A system that tries to resolve all issues on a github repo with OpenHands.☆112Updated 9 months ago
- [ACL 2024] Do Large Language Models Latently Perform Multi-Hop Reasoning?☆74Updated 5 months ago
- Experimental Code for StructuredRAG: JSON Response Formatting with Large Language Models☆111Updated 4 months ago
- Interaction-first method for generating demonstrations for web-agents on any website☆44Updated 4 months ago