METR / eval-analysis-publicLinks
Public repository containing METR's DVC pipeline for eval data analysis
☆108Updated 5 months ago
Alternatives and similar repositories for eval-analysis-public
Users that are interested in eval-analysis-public are comparing it to the libraries listed below
Sorting:
- accompanying material for sleep-time compute paper☆111Updated 4 months ago
- Training an LLM to use a calculator with multi-turn reinforcement learning, achieving a **62% absolute increase in evaluation accuracy**.☆49Updated 4 months ago
- Matrix (Multi-Agent daTa geneRation Infra and eXperimentation framework) is a versatile engine for multi-agent conversational data genera…☆95Updated this week
- Train your own SOTA deductive reasoning model☆106Updated 6 months ago
- A library for benchmarking the Long Term Memory and Continual learning capabilities of LLM based agents. With all the tests and code you…☆78Updated 9 months ago
- Official homepage for "Self-Harmonized Chain of Thought" (NAACL 2025)☆92Updated 7 months ago
- ReDel is a toolkit for researchers and developers to build, iterate on, and analyze recursive multi-agent systems. (EMNLP 2024 Demo)☆86Updated last week
- Source code for the collaborative reasoner research project at Meta FAIR.☆103Updated 5 months ago
- ☆39Updated last year
- ☆95Updated last month
- Just a bunch of benchmark logs for different LLMs☆119Updated last year
- Source code of "How to Correctly do Semantic Backpropagation on Language-based Agentic Systems" 🤖☆75Updated 9 months ago
- A DSPy-based implementation of the tree of thoughts method (Yao et al., 2023) for generating persuasive arguments☆89Updated 11 months ago
- LLM reads a paper and produce a working prototype☆56Updated 5 months ago
- A framework for pitting LLMs against each other in an evolving library of games ⚔☆33Updated 4 months ago
- QAlign is a new test-time alignment approach that improves language model performance by using Markov chain Monte Carlo methods.☆24Updated last week
- ☆55Updated 7 months ago
- Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optuna☆55Updated 7 months ago
- The Prime Intellect CLI provides a powerful command-line interface for managing GPU resources across various providers☆84Updated this week
- ☆56Updated 2 months ago
- ☆133Updated 6 months ago
- Archon provides a modular framework for combining different inference-time techniques and LMs with just a JSON config file.☆182Updated 6 months ago
- Automated Capability Discovery via Foundation Model Self-Exploration☆63Updated 7 months ago
- ☆68Updated 3 months ago
- A framework for optimizing DSPy programs with RL☆177Updated this week
- Code for "Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs"☆54Updated 6 months ago
- OpenCoconut implements a latent reasoning paradigm where we generate thoughts before decoding.☆172Updated 8 months ago
- ☆129Updated last week
- look how they massacred my boy☆64Updated 11 months ago
- [ACL 2024] Do Large Language Models Latently Perform Multi-Hop Reasoning?☆77Updated 6 months ago