andyrdt / refusal_direction
Code and results accompanying the paper "Refusal in Language Models Is Mediated by a Single Direction".
☆117Updated last month
Related projects ⓘ
Alternatives and complementary repositories for refusal_direction
- Steering Llama 2 with Contrastive Activation Addition☆94Updated 5 months ago
- Improving Alignment and Robustness with Circuit Breakers☆152Updated last month
- ☆102Updated last month
- Steering vectors for transformer language models in Pytorch / Huggingface☆64Updated last month
- ☆99Updated this week
- LLM experiments done during SERI MATS - focusing on activation steering / interpreting activation spaces☆77Updated last year
- Code for reproducing our paper "Not All Language Model Features Are Linear"☆60Updated last month
- Code for NeurIPS'24 paper 'Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization'☆160Updated last month
- ☆89Updated 4 months ago
- A toolkit for describing model features and intervening on those features to steer behavior.☆69Updated this week
- Code for In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering☆142Updated 3 weeks ago
- Using sparse coding to find distributed representations used by neural networks.☆181Updated 11 months ago
- Benchmarking LLMs with Challenging Tasks from Real Users☆194Updated this week
- Create feature-centric and prompt-centric visualizations for sparse autoencoders (like those from Anthropic's published research).☆157Updated last month
- Code release for "Debating with More Persuasive LLMs Leads to More Truthful Answers"☆83Updated 7 months ago
- A simple unified framework for evaluating LLMs☆138Updated this week
- ☆101Updated last month
- datasets from the paper "Towards Understanding Sycophancy in Language Models"☆62Updated last year
- Function Vectors in Large Language Models (ICLR 2024)☆116Updated 3 weeks ago
- ☆96Updated 3 months ago
- Algebraic value editing in pretrained language models☆57Updated last year
- WMDP is a LLM proxy benchmark for hazardous knowledge in bio, cyber, and chemical security. We also release code for RMU, an unlearning m…☆78Updated 6 months ago
- ☆168Updated 8 months ago
- ☆186Updated last month
- Sparse autoencoders☆333Updated 2 weeks ago
- ☆78Updated last year
- Investigating the generalization behavior of LM probes trained to predict truth labels: (1) from one annotator to another, and (2) from e…☆25Updated 5 months ago
- ☆70Updated 3 months ago
- ☆43Updated 4 months ago
- Röttger et al. (2023): "XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models"☆61Updated 10 months ago