andyrdt / refusal_directionLinks
Code and results accompanying the paper "Refusal in Language Models Is Mediated by a Single Direction".
☆257Updated 2 months ago
Alternatives and similar repositories for refusal_direction
Users that are interested in refusal_direction are comparing it to the libraries listed below
Sorting:
- Improving Alignment and Robustness with Circuit Breakers☆226Updated 11 months ago
- Steering Llama 2 with Contrastive Activation Addition☆174Updated last year
- LLM experiments done during SERI MATS - focusing on activation steering / interpreting activation spaces☆96Updated last year
- [ICLR 2025] General-purpose activation steering library☆94Updated 3 weeks ago
- Steering vectors for transformer language models in Pytorch / Huggingface☆122Updated 6 months ago
- ☆162Updated 9 months ago
- For OpenMOSS Mechanistic Interpretability Team's Sparse Autoencoder (SAE) research.☆143Updated last week
- Delphi was the home of a temple to Phoebus Apollo, which famously had the inscription, 'Know Thyself.' This library lets language models …☆206Updated last week
- Stanford NLP Python library for benchmarking the utility of LLM interpretability methods☆124Updated 2 months ago
- Röttger et al. (NAACL 2024): "XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models"☆110Updated 6 months ago
- Code for In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering☆181Updated 6 months ago
- A toolkit for describing model features and intervening on those features to steer behavior.☆197Updated 9 months ago
- Using sparse coding to find distributed representations used by neural networks.☆262Updated last year
- Code release for "Debating with More Persuasive LLMs Leads to More Truthful Answers"☆114Updated last year
- ☆223Updated last year
- A simple unified framework for evaluating LLMs☆240Updated 4 months ago
- A resource repository for representation engineering in large language models☆130Updated 9 months ago
- ☆184Updated last month
- ☆187Updated 4 months ago
- Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimization☆30Updated last year
- ☆89Updated last month
- Function Vectors in Large Language Models (ICLR 2024)☆176Updated 4 months ago
- WMDP is a LLM proxy benchmark for hazardous knowledge in bio, cyber, and chemical security. We also release code for RMU, an unlearning m…☆135Updated 2 months ago
- Sparsify transformers with SAEs and transcoders☆608Updated last week
- A curated list of LLM Interpretability related material - Tutorial, Library, Survey, Paper, Blog, etc..☆265Updated 5 months ago
- ☆185Updated 9 months ago
- This repository contains the code and data for the paper "SelfIE: Self-Interpretation of Large Language Model Embeddings" by Haozhe Chen,…☆50Updated 8 months ago
- Algebraic value editing in pretrained language models☆65Updated last year
- We jailbreak GPT-3.5 Turbo’s safety guardrails by fine-tuning it on only 10 adversarially designed examples, at a cost of less than $0.20…☆317Updated last year
- AI Logging for Interpretability and Explainability🔬☆125Updated last year