agencyenterprise / PromptInject
PromptInject is a framework that assembles prompts in a modular fashion to provide a quantitative analysis of the robustness of LLMs to adversarial prompt attacks. π Best Paper Awards @ NeurIPS ML Safety Workshop 2022
β293Updated 6 months ago
Related projects: β
- β357Updated last month
- This repository provides implementation to formalize and benchmark Prompt Injection attacks and defensesβ125Updated 2 weeks ago
- Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks [arXiv, Apr 2024]β181Updated last month
- Persuasive Jailbreaker: we can persuade LLMs to jailbreak them!β232Updated 5 months ago
- HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalβ275Updated last month
- TAP: An automated jailbreaking method for black-box LLMsβ106Updated 6 months ago
- Official repo for GPTFUZZER : Red Teaming Large Language Models with Auto-Generated Jailbreak Promptsβ366Updated 5 months ago
- An Open Robustness Benchmark for Jailbreaking Language Models [arXiv 2024]β169Updated last month
- Papers about red teaming LLMs and Multimodal models.β66Updated this week
- The official implementation of our ICLR2024 paper "AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models".β203Updated last month
- Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMsβ156Updated 3 months ago
- β31Updated last month
- Curation of prompts that are known to be adversarial to large language modelsβ170Updated last year
- β‘ Vigil β‘ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputsβ299Updated 7 months ago
- A Comprehensive Assessment of Trustworthiness in GPT Modelsβ250Updated this week
- A benchmark for prompt injection detection systems.β80Updated last week
- A repository of Language Model Vulnerabilities and Exposures (LVEs).β103Updated 6 months ago
- π§ LLMFuzzer - Fuzzing Framework for Large Language Models π§ LLMFuzzer is the first open-source fuzzing framework specifically designed β¦β218Updated 7 months ago
- Red-Teaming Language Models with DSPyβ116Updated 5 months ago
- A trace analysis tool for AI agents.β97Updated this week
- We jailbreak GPT-3.5 Turboβs safety guardrails by fine-tuning it on only 10 adversarially designed examples, at a cost of less than $0.20β¦β219Updated 6 months ago
- LLM security and privacyβ38Updated 5 months ago
- Finding trojans in aligned LLMs. Official repository for the competition hosted at SaTML 2024.β100Updated 3 months ago
- Official implementation of AdvPrompter https//arxiv.org/abs/2404.16873β110Updated 4 months ago
- SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelsβ441Updated 2 months ago
- Dropbox LLM Security research code and resultsβ210Updated 3 months ago
- Papers and resources related to the security and privacy of LLMs π€β393Updated last week
- Every practical and proposed defense against prompt injection.β310Updated 3 months ago
- A collection of automated evaluators for assessing jailbreak attempts.β55Updated 2 months ago
- Dataset for the Tensor Trust projectβ29Updated 6 months ago