dit7ya / awesome-ai-alignment
A curated list of awesome resources for Artificial Intelligence Alignment research
☆70Updated last year
Alternatives and similar repositories for awesome-ai-alignment:
Users that are interested in awesome-ai-alignment are comparing it to the libraries listed below
- 🧠 Starter templates for doing interpretability research☆70Updated last year
- Keeping language models honest by directly eliciting knowledge encoded in their activations.☆199Updated this week
- Tools for studying developmental interpretability in neural networks.☆88Updated 3 months ago
- ☆54Updated 7 months ago
- ☆68Updated last year
- datasets from the paper "Towards Understanding Sycophancy in Language Models"☆74Updated last year
- Machine Learning for Alignment Bootcamp☆25Updated last year
- Measuring the situational awareness of language models☆34Updated last year
- A dataset of alignment research and code to reproduce it☆77Updated last year
- ☆26Updated last year
- Redwood Research's transformer interpretability tools☆14Updated 3 years ago
- A collection of different ways to implement accessing and modifying internal model activations for LLMs☆15Updated 6 months ago
- Mechanistic Interpretability for Transformer Models☆50Updated 2 years ago
- A puzzle to learn about prompting☆127Updated last year
- we got you bro☆35Updated 8 months ago
- Contains random samples referenced in the paper "Sleeper Agents: Training Robustly Deceptive LLMs that Persist Through Safety Training".☆102Updated last year
- Notebooks accompanying Anthropic's "Toy Models of Superposition" paper☆120Updated 2 years ago
- Machine Learning for Alignment Bootcamp☆72Updated 2 years ago
- ☆133Updated 5 months ago
- unofficial re-implementation of "Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets"☆78Updated 2 years ago
- ☆26Updated last year
- LLM experiments done during SERI MATS - focusing on activation steering / interpreting activation spaces☆91Updated last year
- RuLES: a benchmark for evaluating rule-following in language models☆221Updated 2 months ago
- ☆114Updated 8 months ago
- Code and Data Repo for the CoNLL Paper -- Future Lens: Anticipating Subsequent Tokens from a Single Hidden State☆18Updated last year
- Emergent world representations: Exploring a sequence model trained on a synthetic task☆181Updated last year
- Open source interpretability artefacts for R1.☆82Updated this week
- ☆219Updated 6 months ago
- METR Task Standard☆146Updated 2 months ago
- ☆266Updated 9 months ago