marcus-jw / Targeted-Manipulation-and-Deception-in-LLMs
Codebase for "On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback". This repo implements a generative multi-turn RL environment with support for agent, user, user feedback, transition and veto models. It also implements KTO and expert iteration for training on user preferences.
☆13Updated 3 months ago
Alternatives and similar repositories for Targeted-Manipulation-and-Deception-in-LLMs:
Users that are interested in Targeted-Manipulation-and-Deception-in-LLMs are comparing it to the libraries listed below
- A library for efficient patching and automatic circuit discovery.☆54Updated 2 weeks ago
- Code for my NeurIPS 2024 ATTRIB paper titled "Attribution Patching Outperforms Automated Circuit Discovery"☆29Updated 9 months ago
- Sparse Autoencoder Training Library☆42Updated 4 months ago
- ☆21Updated 5 months ago
- ☆35Updated last year
- ☆78Updated 6 months ago
- ☆19Updated this week
- Algebraic value editing in pretrained language models☆62Updated last year
- A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity.☆64Updated 3 months ago
- ☆110Updated 6 months ago
- ☆88Updated 2 weeks ago
- Align your LM to express calibrated verbal statements of confidence in its long-form generations.☆22Updated 8 months ago
- [NeurIPS'24 Spotlight] Observational Scaling Laws☆51Updated 5 months ago
- Rewarded soups official implementation☆54Updated last year
- This is code for most of the experiments in the paper Understanding the Effects of RLHF on LLM Generalisation and Diversity☆40Updated last year
- ☆57Updated 3 months ago
- ☆39Updated last month
- Code release for "Debating with More Persuasive LLMs Leads to More Truthful Answers"☆101Updated 11 months ago
- Source codes for "Preference-grounded Token-level Guidance for Language Model Fine-tuning" (NeurIPS 2023).☆15Updated last month
- This is an official implementation of the paper ``Building Math Agents with Multi-Turn Iterative Preference Learning'' with multi-turn DP…☆20Updated 2 months ago
- Directional Preference Alignment☆56Updated 5 months ago
- This repository contains the code used for the experiments in the paper "Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity…☆23Updated 11 months ago
- Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision☆118Updated 5 months ago
- NeuroSurgeon is a package that enables researchers to uncover and manipulate subnetworks within models in Huggingface Transformers☆39Updated 2 weeks ago
- ☆34Updated 4 months ago
- Sparse probing paper full code.☆53Updated last year
- ☆144Updated this week
- Code for the ICLR 2024 paper "How to catch an AI liar: Lie detection in black-box LLMs by asking unrelated questions"☆65Updated 8 months ago
- ☆18Updated last year
- Official repository for our paper, Transformers Learn Higher-Order Optimization Methods for In-Context Learning: A Study with Linear Mode…☆16Updated 3 months ago