jailbreak

A jailbreak is an adversarial prompt crafted to bypass the intended safety policies, role-instructions, or guardrails of a model system, such as a large language model, so that it produces disallowed or unintended behavior.

In practice, jailbreaks may take the form of carefully structured instructions, role-play scenarios, obfuscation or encoding tricks, or multi-step workflows that cause the system to misinterpret or override its internal constraints. A jailbreak is normally supplied by whoever is prompting the model, and that direct channel is what separates it from a prompt injection, where a third party smuggles instructions to the model through untrusted content that the system ingests, such as a retrieved web page, document, or tool result. The two overlap, since attacker-planted content can carry jailbreak-style framing, which is why some security taxonomies treat jailbreaking as one technique inside the broader prompt injection family.

Effective mitigation requires layered defense, and the useful layers differ by threat model. Against jailbreaks, where the person prompting the model is the adversary, defenses include screening user input, hardening the system prompt so that it states its boundaries and how to refuse, enforcing a hierarchy in which lower-trust messages can’t override higher-trust policy, throttling accounts that repeatedly probe the guardrails, and red-teaming the application before deployment. Against instructions that arrive inside untrusted content, defenses include instruction isolation, provenance checks, allow and deny lists for tool invocation, and execution under least privilege. Continuous evaluation and monitoring apply to both.