dit7ya / awesome-ai-alignmentLinks
A curated list of awesome resources for Artificial Intelligence Alignment research
ā71Updated 2 years ago
Alternatives and similar repositories for awesome-ai-alignment
Users that are interested in awesome-ai-alignment are comparing it to the libraries listed below
Sorting:
- Keeping language models honest by directly eliciting knowledge encoded in their activations.ā209Updated this week
- š§ Starter templates for doing interpretability researchā74Updated 2 years ago
- we got you broā36Updated last year
- Resources for skilling up in AI alignment research engineering. Covers basics of deep learning, mechanistic interpretability, and RL.ā227Updated last month
- ā57Updated this week
- ā138Updated 2 months ago
- unofficial re-implementation of "Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets"ā79Updated 3 years ago
- A dataset of alignment research and code to reproduce itā77Updated 2 years ago
- datasets from the paper "Towards Understanding Sycophancy in Language Models"ā92Updated last year
- A puzzle to learn about promptingā135Updated 2 years ago
- Tools for studying developmental interpretability in neural networks.ā103Updated 3 months ago
- ā276Updated last year
- RuLES: a benchmark for evaluating rule-following in language modelsā232Updated 7 months ago
- Emergent world representations: Exploring a sequence model trained on a synthetic taskā191Updated 2 years ago
- ā74Updated 2 years ago
- ā300Updated last year
- ā242Updated 11 months ago
- Measuring the situational awareness of language modelsā38Updated last year
- Contains random samples referenced in the paper "Sleeper Agents: Training Robustly Deceptive LLMs that Persist Through Safety Training".ā116Updated last year
- ā27Updated 2 years ago
- This repository collects all relevant resources about interpretability in LLMsā372Updated 10 months ago
- Investigating the generalization behavior of LM probes trained to predict truth labels: (1) from one annotator to another, and (2) from eā¦ā28Updated last year
- Tools for understanding how transformer predictions are built layer-by-layerā525Updated last month
- Mechanistic Interpretability Visualizations using Reactā289Updated 9 months ago
- Redwood Research's transformer interpretability toolsā14Updated 3 years ago
- Erasing concepts from neural representations with provable guaranteesā234Updated 8 months ago
- LLM experiments done during SERI MATS - focusing on activation steering / interpreting activation spacesā97Updated 2 years ago
- Aligning AI With Shared Human Values (ICLR 2021)ā298Updated 2 years ago
- Notebooks accompanying Anthropic's "Toy Models of Superposition" paperā129Updated 3 years ago
- Inference API for many LLMs and other useful tools for empirical researchā71Updated last week