☆330Jul 2, 2024Updated last year
Alternatives and similar repositories for evals
Users that are interested in evals are comparing it to the libraries listed below
Sorting:
- Human preference data for "Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback"☆1,818Jun 17, 2025Updated 8 months ago
- ☆25Sep 5, 2024Updated last year
- ☆251Dec 21, 2022Updated 3 years ago
- ☆1,072Mar 6, 2024Updated last year
- (Model-written) LLM evals library☆18Dec 13, 2024Updated last year
- A library for bridging Python and HTML/Javascript (via Svelte) for creating interactive visualizations☆205Dec 22, 2021Updated 4 years ago
- Keeping language models honest by directly eliciting knowledge encoded in their activations.☆217Feb 23, 2026Updated last week
- ☆22Sep 9, 2021Updated 4 years ago
- Notebooks accompanying Anthropic's "Toy Models of Superposition" paper☆137Sep 14, 2022Updated 3 years ago
- datasets from the paper "Towards Understanding Sycophancy in Language Models"☆105Oct 25, 2023Updated 2 years ago
- LLM experiments done during SERI MATS - focusing on activation steering / interpreting activation spaces☆102Sep 21, 2023Updated 2 years ago
- ControlArena is a collection of settings, model organisms and protocols - for running control experiments.☆158Updated this week
- METR Task Standard☆177Feb 3, 2025Updated last year
- Implementation of Influence Function approximations for differently sized ML models, using PyTorch☆16Sep 15, 2023Updated 2 years ago
- ☆120Jan 19, 2026Updated last month
- Interactive Composition Explorer: a debugger for compositional language model programs☆567Jan 5, 2026Updated last month
- Contains random samples referenced in the paper "Sleeper Agents: Training Robustly Deceptive LLMs that Persist Through Safety Training".☆131Mar 9, 2024Updated last year
- A library for mechanistic interpretability of GPT-style language models☆3,133Updated this week
- The AI that helps you achieve your goals☆11Feb 4, 2024Updated 2 years ago
- ☆552Feb 5, 2024Updated 2 years ago
- gpt completions in vscode☆35Mar 24, 2023Updated 2 years ago
- ☆91Mar 4, 2024Updated last year
- Measuring if attention is explanation with ROAR☆22Mar 3, 2023Updated 3 years ago
- The nnsight package enables interpreting and manipulating the internals of deep learned models.☆825Feb 23, 2026Updated last week
- Representation Engineering: A Top-Down Approach to AI Transparency☆953Aug 14, 2024Updated last year
- Used for adaptive human in the loop evaluation of language and embedding models.☆308Mar 1, 2023Updated 3 years ago
- Erasing concepts from neural representations with provable guarantees☆243Jan 27, 2025Updated last year
- Inspect: A framework for large language model evaluations☆1,783Updated this week
- ☆2,548May 19, 2024Updated last year
- Tools for understanding how transformer predictions are built layer-by-layer☆567Aug 7, 2025Updated 6 months ago
- For experiments involving instruct gpt. Currently used for documenting open research questions.☆71Nov 8, 2022Updated 3 years ago
- A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)☆4,741Jan 8, 2024Updated 2 years ago
- ☆13Mar 22, 2023Updated 2 years ago
- ☆23Jan 27, 2026Updated last month
- Experiments with representation engineering☆13Feb 28, 2024Updated 2 years ago
- Benchmarking LLM Inference Speeds☆13Feb 4, 2026Updated last month
- Code and data for the NAACL 2021 paper: "XFORMAL: A Benchmark for Multilingual Formality Style Transfer"☆12Jun 7, 2021Updated 4 years ago
- ☆12Oct 23, 2022Updated 3 years ago
- ☆20Nov 15, 2024Updated last year