EleutherAI / elk
Keeping language models honest by directly eliciting knowledge encoded in their activations.
☆182Updated this week
Related projects: ⓘ
- ☆229Updated 2 months ago
- Erasing concepts from neural representations with provable guarantees☆202Updated 3 months ago
- Mechanistic Interpretability Visualizations using React☆175Updated 2 months ago
- ☆246Updated 6 months ago
- ☆174Updated 4 months ago
- Tools for understanding how transformer predictions are built layer-by-layer☆408Updated 3 months ago
- Emergent world representations: Exploring a sequence model trained on a synthetic task☆165Updated last year
- Create feature-centric and prompt-centric visualizations for sparse autoencoders (like those from Anthropic's published research).☆133Updated last month
- Mechanistic Interpretability for Transformer Models☆48Updated 2 years ago
- ☆24Updated 5 months ago
- Steering Llama 2 with Contrastive Activation Addition☆83Updated 3 months ago
- 🧠 Starter templates for doing interpretability research☆59Updated last year
- ☆99Updated 10 months ago
- Training Sparse Autoencoders on Language Models☆367Updated this week
- Resources for skilling up in AI alignment research engineering. Covers basics of deep learning, mechanistic interpretability, and RL.☆185Updated 7 months ago
- ☆91Updated last month
- A puzzle to learn about prompting☆106Updated last year
- ☆91Updated last month
- Extract full next-token probabilities via language model APIs☆226Updated 6 months ago
- Vivaria is METR's tool for running evaluations and conducting agent elicitation research.☆50Updated this week
- ☆75Updated this week
- Steering vectors for transformer language models in Pytorch / Huggingface☆52Updated last month
- Utilities for the HuggingFace transformers library☆55Updated last year
- LLM experiments done during SERI MATS - focusing on activation steering / interpreting activation spaces☆73Updated 11 months ago
- The GitHub repo for Goal Driven Discovery of Distributional Differences via Language Descriptions☆68Updated last year
- Using sparse coding to find distributed representations used by neural networks.☆162Updated 10 months ago
- A dataset of alignment research and code to reproduce it☆68Updated last year
- ☆58Updated this week
- datasets from the paper "Towards Understanding Sycophancy in Language Models"☆59Updated 10 months ago
- Python library which enables complex compositions of language models such as scratchpads, chain of thought, tool use, selection-inference…☆190Updated 3 months ago