LLM Jailbreak: Attack Taxonomy, Techniques, and Defense Reality

An LLM jailbreak is any technique that causes a model to produce output it was trained or instructed not to produce — harmful content, capability demonstrations, policy violations — by manipulating the input rather than modifying model weights. The phrase has evolved from a loose description of “DAN” prompts on Reddit into a formal research category with its own benchmarks, taxonomies, and peer-reviewed success metrics. Published attack rates against frontier models remain alarmingly high: across multiple 2024–2025 studies, black-box attacks on proprietary models like GPT-4 and Gemini achieve 80–90%+ success on standardized harmful-request benchmarks. The gap between “aligned” and “safe” is measurable, and it matters for every team deploying an LLM in a production context.

Jailbreak or Prompt Injection?

The two terms are routinely conflated, and the conflation hides a boundary worth keeping. Prompt injection is the broader class: any attacker-controlled text that ends up steering the model. Jailbreaking is the subset whose goal is specifically to bypass safety alignment, and where the attacker is typically the user. OWASP LLM01:2025 files both under one risk, but the distinction is operational. A jailbreak unlocks disallowed content from inside an authorized session; a prompt injection usually weaponizes data flowing in from a third party.

That determines what an engagement targets. Testing a chat assistant in isolation is mostly jailbreak hunting. Testing an agent that browses, ingests email, or queries a vector store puts the indirect injection class formalized by Greshake et al. first — and there the boundary between “jailbreak” and “remote takeover” collapses quickly, since an attacker controlling under 2% of input tokens (a hidden paragraph in a fetched page, a comment in a PDF, image alt text) can override the system prompt of an LLM-integrated application.

Why Alignment Breaks

Safety alignment rests on two foundations, and both have structural gaps attackers work in. Reinforcement learning from human feedback trains a reward model on preference data and optimizes the LLM against it, so the reward model’s finite coverage means inputs far enough from the training distribution produce unexpected behavior. Supervised refusal fine-tuning is keyed to surface patterns — particular phrasings, topic areas, combinations — which is why rephrasing the same underlying request can evade a trained refusal. The model is not reasoning about intent. It is pattern-matching against a learned distribution, and that distribution has holes.

The January 2026 unified jailbreaking survey attributes susceptibility to three root causes: incomplete training data that never covered adversarial input distributions; linguistic ambiguity that opens semantic gaps between attacker intent and safety-classifier interpretation; and generative uncertainty, the inherent stochasticity of autoregressive sampling that lets the same prompt produce different compliance behavior across runs. No alignment technique closes all three at once.

Attack Taxonomy: Four Categories Worth Knowing

The most useful organizing framework for LLM jailbreak attacks, laid out in Yi et al.’s comprehensive 2024 survey, divides attacks along two axes: attacker visibility (black-box vs. white-box) and prompt structure (single-turn vs. multi-turn). White-box attacks assume gradient access and the ability to optimize adversarial suffixes directly against the model — GCG and its variants live here. They achieve the highest raw success rates but require open weights, and their operational significance comes from transfer rather than direct use; that mechanism, along with attacker-LLM pipelines like PAIR and TAP, is covered separately in automated jailbreak attacks and the transfer problem. Black-box attacks assume only API access, which is the relevant category for production engagements.

Within the black-box space, five technique families account for the majority of current attack traffic:

Roleplay and persona assignment. The model is asked to assume a character — a fictional AI with no restrictions, a creative writing assistant, a historical figure — and the prohibited request is embedded in the character’s voice. An empirical study testing 78 distinct jailbreak prompts across 31,200 queries found roleplay present in 97.44% of successful attempts. The success rate against GPT-4 in roleplay-heavy categories consistently exceeds 70% on realistic scenario distributions. Models are more compliant when framing constructs the request as continuation rather than as a novel decision.

Encoding and obfuscation. Input filters keyed to prohibited keywords can be bypassed by delivering the request in base64, Morse code, pig Latin, token substitutions, or Unicode lookalikes. The model decodes and responds in plain text. Attack success rates using encoding-based evasion reach 76% in recent red team evaluations, and the extreme case is trivially cheap: FlipAttack, which simply reverses character order in the harmful instruction, is reported at roughly 98% attack success against GPT-4o in published black-box tests. The defense surface is wide because virtually any encoding scheme the model can interpret can be used.

Implication chaining and logic traps. A series of individually-acceptable premises is constructed so that the final step implies the target content. The model follows the logical chain because each individual step is low-violation, and safety training did not generalize to the composed output. Multi-step structured reasoning is particularly vulnerable because the model’s tendency toward coherence with its own prior text works against its safety alignment. In the 1,400-prompt evaluation, logic traps reached 81.4% attack success and showed the highest cross-family portability of any category tested, at 64.1% transfer from GPT-4 to Claude 2.

Fine-tuning attacks. Rather than working in prompt space at all, the attacker degrades alignment by injecting adversarial data into a model’s fine-tuning phase. As fine-tuning APIs commoditize across closed and open-weight providers, this surface expands, and published results show that even small quantities of adversarial data degrade refusal behavior across categories unrelated to the injected examples.

Multi-turn conversational escalation. Discussed in detail below — this is where the most actionable research has appeared recently.

Two Techniques That Changed the Calculus

Many-Shot Jailbreaking

Anthropic’s many-shot jailbreaking research, published at NeurIPS 2024, identified a class of attacks that did not exist against earlier models because they require long context windows — 100K+ tokens — that only became standard in 2023–2024.

The attack is straightforward: fill the context window with a large number of fake Q&A exchanges in which an AI assistant helpfully answers harmful questions. Then append the real harmful question. Effective sample counts range from tens to hundreds; Anthropic tested up to 256. The attack success rate follows a power law with the number of shots — more shots means higher compliance — and it generalizes across model families. Claude 2.0, GPT-4, Llama 2 (70B), GPT-3.5, and Mistral 7B were all successfully jailbroken in Anthropic’s testing.

What makes many-shot operationally significant is that it requires zero optimization. There is no adversarial suffix search, no iterative refinement loop. An attacker assembles the fake Q&A dialogue once and can replay it against any sufficiently large-context target. Long-context expansion — a capability marketed as a feature — created an attack class that did not exist before it.

A dedicated breakdown of the technique — the scaling behaviour, the adjacent attacks that share its shape, and how to scope it in a test plan — is in many-shot jailbreaking.

Mitigations are limited. Prompt length monitoring can flag anomalously large inputs, but legitimate enterprise uses (document summarization, code review, RAG pipelines) also produce large contexts. Anthropic’s proposed approach includes training-time interventions that reduce the power-law scaling — the model complies less steeply as shot count increases — but this is not a binary fix. The strongest reported result is preprocessing: classifying and reformatting the prompt before the model sees it dropped many-shot success from 61% to roughly 2% on the tested set, at the cost of latency and false positives on legitimate long-context work.

Crescendo: Multi-Turn Escalation

Crescendo, accepted at USENIX Security 2025, attacks the multi-turn conversation surface rather than single prompts. The attack begins with benign questions tangentially related to the target topic and increments turn by turn — each question slightly more proximate to the goal — until the model produces the prohibited content. The mechanism exploits a genuine property of autoregressive generation: a model that has already generated text on a topic weights the accumulated context heavily and continues in that direction. Crescendo constructs the context deliberately.

The automated implementation, Crescendomation, outperformed leading single-turn jailbreak methods by 29–61% on GPT-4 and 49–71% on Gemini Pro on the AdvBench benchmark. The attack completes in under five interaction turns on average and transferred across all tested models — GPT-4, Gemini Pro/Ultra, Llama 2/3, and Anthropic Chat.

For a red team engagement, this has a direct procedural implication: evaluation frameworks that test prompts in isolation measure the wrong surface. A deployment can pass every per-request safety check in a Crescendo sequence and still produce prohibited content by turn six. The model is not bypassed; it is walked.

Defense requires conversation-level monitoring — semantic drift detection across a session, topic escalation fingerprinting, and tracking whether model-generated output is being used to prime subsequent requests. Per-request filtering is insufficient by design. Documented escalation chains across prohibited content categories are maintained at jailbreakdb.com and jailbreaks.fyi.

The Multimodal Surface

Most jailbreak literature is text-only, but the attack surface follows the modalities the model accepts. The 2026 unified survey covers vision-language models specifically and identifies three attack classes with no text equivalent: typographic prompt injection, where the model reads text rendered inside an image as instruction; combined text-image perturbations that split a harmful request across modalities so neither component alone trips a safety filter; and proxy model transfer, where adversarial perturbations generated against a surrogate VLM carry over to the target.

This matters for any deployment accepting image, audio, or document input — RAG pipelines ingesting external files, agents with tool use, or customer-facing assistants that process user uploads. The surface scales with the number of input modalities, not with the size of the text interface. For a catalog of documented prompts across both text and multimodal categories, jailbreakdb.com maintains an indexed database with reproduction steps, and adversarialml.dev covers gradient-based attack implementations at the ML layer.

What Defense Actually Looks Like

Honest accounting from published literature: no single defensive measure stops all attack classes. Alignment training reduces success rates relative to unaligned models but does not eliminate jailbreaks. Constitutional AI and RLHF-based refusal training improve aggregate compliance but leave category-specific gaps. Keyword and pattern filters cover known bad prompts but are trivially bypassed by encoding or multi-turn approaches that produce prohibited content without any single prohibited input.

Layered defense is the realistic posture:

• Input normalization to canonicalize encoding, strip zero-width characters, and decode common obfuscation before the model sees the prompt. This is what shrinks the steganographic surface, and it is cheaper than classification.
• Input guardrails for known-pattern detection on top of normalized text
• System prompt hardening with explicit refusal instructions, tested against known attack templates. Raises the baseline; replaces nothing.
• Conversation-level semantic monitoring for escalation and drift, especially for agentic systems where multi-turn interaction is the primary mode
• Output filtering as a backstop, with classification against prohibited content categories independent of how the input was structured
• Model diversity for high-stakes deployments, routing requests through a second model as a safety judge so that a model-specific jailbreak is not a single point of failure
• Rate limiting and session analysis for detecting iterative probing patterns

The 2026 unified survey goes one step further and proposes variant-consistency detection and gradient-sensitivity analysis as design principles for defenses that generalize past the known variant list — worth understanding before committing to a production guardrail stack, because every control above is keyed to attacks that have already been published.

What to Add to Your Testing Playbook

The right corpus depends on what the target is. To rehearse the payload-crafting loop against a safe target before pointing it at a client system, the interactive AI Red Team Gym runs the same escalation pattern in the browser.

A chat assistant. Run role-play scaffolds, encoded payloads (base64, character reversal, Unicode homoglyphs), and a long-context many-shot set. Score more than refusal versus compliance: partial leakage and off-by-one answers, where the model refuses the literal request and answers a paraphrase, are findings.

An agent or RAG system. Indirect injection through every ingestion path — crawled pages, uploaded documents, tool outputs, calendar invites, embedded image text. Test what happens when the model summarizes attacker-controlled content, and what happens when one tool’s output becomes another tool’s input. Confirm whether the system trusts retrieved text on the same footing as the user’s prompt. Most still do.

A model that claims to be defended. Run a transfer test with universal adversarial suffixes from the public sets, then a many-shot attack at the longest context the API will accept. Those two surface most of the easy wins before you spend budget on anything bespoke.

The DAN era trained a generation of practitioners to think jailbreaks are about clever wording. They are not. They exploit the gap between what a model is trained to refuse and what it is trained to do, and that gap appears everywhere the model trusts input it should not.

Teams building production LLM systems should assume the deployed model will be jailbroken by a determined attacker and build compensating controls at the application layer rather than trusting model-layer alignment as the sole defense. GuardML provides a comparative view of production guardrail architectures against different attack classes. For ongoing disclosure of new techniques and real-world incidents, ai-alert.org tracks documented jailbreak disclosures as they emerge.

The volume of published research — multiple conference papers per year across CCS, USENIX Security, NeurIPS, and EMNLP — means the attack surface is expanding faster than any single mitigation can close. Keeping the engagement toolkit current requires tracking the literature, not waiting for vendor announcements.

---

→ For automated attack generation and why jailbreaks port across model families, see Automated Jailbreak Attacks and the Transfer Problem. For a GPT-4-specific breakdown of IRIS self-refinement, Crescendo, and classic pattern attacks, see GPT-4 Jailbreak Techniques: A Red Teamer’s Technical Reference.