safety-research / safety-toolingView external linksLinks
Inference API for many LLMs and other useful tools for empirical research
☆104Updated this week
Alternatives and similar repositories for safety-tooling
Users that are interested in safety-tooling are comparing it to the libraries listed below
Sorting:
- ☆25Nov 11, 2025Updated 3 months ago
- ControlArena is a collection of settings, model organisms and protocols - for running control experiments.☆153Updated this week
- ☆20May 25, 2024Updated last year
- Open Source Replication of Anthropic's Alignment Faking Paper☆54Apr 4, 2025Updated 10 months ago
- Open source replication of Anthropic's Crosscoders for Model Diffing☆64Oct 27, 2024Updated last year
- Official Code for our paper: "Language Models Learn to Mislead Humans via RLHF""☆18Oct 11, 2024Updated last year
- Official codebase for "Analyzing the Generalization and Reliability of Steering Vectors"☆19Dec 14, 2024Updated last year
- Vivaria is METR's tool for running evaluations and conducting agent elicitation research.☆134Feb 8, 2026Updated last week
- Codebase for "On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback". This repo implements a generative multi-tur…☆23Dec 3, 2024Updated last year
- A CLI tool that helps AI researchers share datasets responsibly.☆20Sep 15, 2025Updated 5 months ago
- A collection of different ways to implement accessing and modifying internal model activations for LLMs☆20Oct 18, 2024Updated last year
- The AI that helps you achieve your goals☆11Feb 4, 2024Updated 2 years ago
- Improving transparency of large language models' reasoning☆14Nov 25, 2025Updated 2 months ago
- Code repo for the model organisms and convergent directions of EM papers.☆49Sep 22, 2025Updated 4 months ago
- Codebase for Obfuscated Activations Bypass LLM Latent-Space Defenses☆28Feb 11, 2025Updated last year
- Code for the "Overcoming Sparsity Artifacts in Crosscoders to Interpret Chat-Tuning" paper.☆16Nov 21, 2025Updated 2 months ago
- A toolkit that provides a range of model diffing techniques including a UI to visualize them interactively.☆57Feb 9, 2026Updated last week
- Multi-Layer Sparse Autoencoders (ICLR 2025)☆29Feb 6, 2026Updated last week
- Machine Learning for Alignment Bootcamp☆81Apr 27, 2022Updated 3 years ago
- ☆24Oct 3, 2025Updated 4 months ago
- Code to reproduce key results accompanying "SAEs (usually) Transfer Between Base and Chat Models"☆13Jul 18, 2024Updated last year
- A tiny easily hackable implementation of a feature dashboard.☆15Oct 21, 2025Updated 3 months ago
- ☆21Jun 22, 2025Updated 7 months ago
- ☆13Jun 4, 2024Updated last year
- Code for Tangent Model Composition for Ensembling and Continual Fine-tuning (ICCV 2023) and Tangent Transformers for Composition, Privacy…☆13May 14, 2024Updated last year
- Attribution-based Parameter Decomposition☆33Jun 11, 2025Updated 8 months ago
- Shaping capabilities with token-level pretraining data filtering☆81Jan 28, 2026Updated 2 weeks ago
- ☆22Updated this week
- ☆35May 9, 2025Updated 9 months ago
- A benchmark for mechanistic discovery of circuits in Transformers☆16Dec 15, 2024Updated last year
- Learning Safety Constraints for Large Language Models (ICML2025)☆29Aug 4, 2025Updated 6 months ago
- A TinyStories LM with SAEs and transcoders☆14Apr 3, 2025Updated 10 months ago
- Code for my NeurIPS 2024 ATTRIB paper titled "Attribution Patching Outperforms Automated Circuit Discovery"☆47May 31, 2024Updated last year
- Automatically create Anki cards from text using language models☆20Jan 7, 2023Updated 3 years ago
- ☆71Updated this week
- Code release for "Debating with More Persuasive LLMs Leads to More Truthful Answers"☆124Mar 22, 2024Updated last year
- ☆88Dec 18, 2025Updated last month
- ☆929Feb 4, 2026Updated last week
- A library for efficient patching and automatic circuit discovery.☆90Dec 31, 2025Updated last month