Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.
☆142Nov 16, 2026Updated this week
Alternatives and similar repositories for every_eval_ever
Users that are interested in every_eval_ever are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆28Apr 21, 2026Updated 6 months ago
- ☆11Sep 9, 2026Updated 2 months ago
- Code for 'Answer Matching Outperforms Multiple Choice for Language Model Evaluation' paper☆20Jul 4, 2025Updated last year
- A Model Agnostic function to directly remove specified layers from the LLM☆11May 23, 2024Updated 2 years ago
- utilities for batched llm calls with retries☆55Updated this week
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- Fluid Language Model Benchmarking☆29Sep 16, 2025Updated last year
- Mapping out the "memory" of neural nets with data attribution☆113Updated this week
- This is the official repository for "CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data An…☆25Oct 26, 2023Updated 3 years ago
- Hubble is a suite of fully open-source large language models (LLMs) for the scientific study of LLM memorization.☆24Apr 15, 2026Updated 7 months ago
- James' cookbook of evaluations and finetuning experiments☆37Feb 19, 2026Updated 8 months ago
- A sample pattern for running CI tests on Modal☆21Apr 12, 2025Updated last year
- ☆14Feb 5, 2022Updated 4 years ago
- LLM checkpointing for DeepSpeed/Megatron☆27Nov 30, 2025Updated 11 months ago
- A python library built on top of Inspect AI to support Control research and evaluations, by Redwood and EquiStamp☆46Updated this week
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Closed-form polynomial approximations to neural networks☆13Jan 31, 2025Updated last year
- Training-free feature discovery for LLM activations via streaming exemplar partitioning.☆14Sep 17, 2026Updated 2 months ago
- Transcript analysis for long-horizon agent evaluations☆74Nov 10, 2026Updated last week
- ☆14Dec 1, 2025Updated 11 months ago
- Inspect Flow is a workflow stack built on Inspect AI that enables research organisations to run AI evaluations at scale.☆23Updated this week
- SAEs in Jax☆12Dec 16, 2024Updated last year
- torch_remat fine-grained activation checkpointing API☆26Nov 11, 2026Updated last week
- ☆16Oct 27, 2026Updated 3 weeks ago
- Tool for exporting Apple Neural Engine-accelerated versions of transformers models on HuggingFace Hub.☆18Jun 25, 2026Updated 4 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- An Emulator for Fine-tuning Large Language Models using Small Language Models☆11Apr 14, 2024Updated 2 years ago
- Convolutional Monge Mapping Normalization☆10Nov 16, 2023Updated 3 years ago
- ScienceMeter: Tracking Scientific Knowledge Updates in Language Models, COLM 2026☆17Jun 28, 2025Updated last year
- The API gateway for the Public AI Inference Utility, based on Zuplo☆14Oct 26, 2026Updated 3 weeks ago
- ☆25Aug 27, 2026Updated 2 months ago
- ☆29Jul 8, 2026Updated 4 months ago
- Inspect: A framework for large language model evaluations☆3,277Updated this week
- [ICLR 2026] MicroVerse: A Preliminary Exploration Toward a Micro-World Simulation☆12Jul 12, 2026Updated 4 months ago
- ☆41May 7, 2026Updated 6 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Run Inspect AI evals in the cloud☆119Updated this week
- Source code of NAACL 2025 Findings "Scaling Up Membership Inference: When and How Attacks Succeed on Large Language Models"☆16Dec 16, 2025Updated 11 months ago
- Explorations into the proposed SDFT, Self-Distillation Enables Continual Learning, from Shenfeld et al. of MIT☆33Feb 6, 2026Updated 9 months ago
- A Streamlit app to add structured tags to a dataset card☆23Jun 30, 2022Updated 4 years ago
- ☆284Mar 26, 2025Updated last year
- Code repo for the model organisms and convergent directions of EM papers.☆90Sep 22, 2025Updated last year
- Delphi was the home of a temple to Phoebus Apollo, which famously had the inscription, 'Know Thyself.' This library lets language models …☆282Nov 9, 2026Updated last week
Are these results useful?