AI Security Topics
Every offensive AI security topic covered here: prompt injection, jailbreaks, agent and tool-use exploitation, adversarial ML, and red team methodology.
Tags
- #llm-security 28
- #prompt-injection 26
- #red-team 26
- #agent-security 14
- #jailbreak 13
- #adversarial-ml 12
- #indirect-injection 4
- #tooling 4
- #attack-vectors 3
- #owasp 3
- #spoke 3
- #chatgpt 2
- #long-context 2
- #membership-inference 2
- #model-extraction 2
- #prompt-engineering 2
- #rag 2
- #tool-use 2
- #adversarial-training 1
- #agents 1
- #ai-red-team 1
- #ai-security 1
- #alignment 1
- #application-security 1
- #attack-techniques 1
- #automated-attacks 1
- #behavioral-evaluation 1
- #bypass-techniques 1
- #ceh 1
- #custom-gpts 1
- #cve 1
- #defense 1
- #detection 1
- #editorial-policy 1
- #evasion 1
- #faq 1
- #garak 1
- #gcg 1
- #governance 1
- #gpt-4 1
- #gpt-security 1
- #guardrails 1
- #hub 1
- #insecure-output-handling 1
- #interpretability 1
- #knowledge-corruption 1
- #llm-bypass 1
- #llm-monitoring 1
- #llm-security-vulnerabilities 1
- #many-shot-jailbreaking 1
- #methodology 1
- #model-inversion 1
- #model-theft 1
- #multi-turn 1
- #multimodal 1
- #openai 1
- #oscp 1
- #owasp-llm01 1
- #payload-construction 1
- #payload-delivery 1
- #pillar 1
- #poisoning 1
- #pyrit 1
- #reporting 1
- #scoping 1
- #system-prompt-leakage 1
- #taxonomy 1
- #threat-modeling 1
- #training-data-privacy 1
- #transferability 1
Categories
red-team 12 posts
- GPT Security: Attack Surfaces and Production ControlsA technical guide to GPT security covering prompt injection, custom GPTs, agent actions, data handling, and layered production controls.
- Insecure Output Handling: LLM05:2025 Attacks and DefensesInsecure output handling turns model text into XSS, SQL injection, or remote code execution. The attack chain, the CVEs it produced, and the controls.
- LLM Security Vulnerabilities: What Actually Gets ExploitedThe LLM security vulnerabilities showing up in real CVEs: prompt injection, system prompt leakage, RAG poisoning, and tool-call bugs that turn into RCE.
- GPT-Red: What OpenAI's Prompt-Injection Attacker ProvesOpenAI says GPT-Red beat human red teamers 84% to 13% on prompt injection. Here is what those numbers measure, and what they leave unanswered.
- LLM Attack Taxonomy: Prompt Injection, Jailbreaks, Agent HijackA practitioner's map of LLM attack classes: direct and indirect prompt injection, jailbreaks, RAG poisoning, and agent tool-call abuse, mapped to OWASP.
- AI Red Team: Methodology, Tooling, and Attack SurfaceA practitioner's guide to AI red teaming: how LLM attack surface differs from traditional app testing, and the techniques and tooling that map it.
prompt-injection 8 posts
- ChatGPT Exploits: Injection, Memory Hijack, Plugin AbuseHow ChatGPT gets exploited: indirect prompt injection, CSRF memory poisoning, cross-plugin request forgery, agent tool abuse, and the controls that help.
- System Prompt Injection: How Extraction and Override Attacks WorkHow system prompt injection works: extracting hidden system prompts, overriding instructions, and the defense-in-depth countermeasures that hold up.
- Prompt Injection in 2025: OpenAI vs. Broken DefensesOpenAI's advisory on prompt injection landed the same week research showed adaptive attacks beat published defenses more than ninety percent of the time.
- Prompt Injection Examples: A Practitioner's Attack LibraryA technical breakdown of real prompt injection examples across direct, indirect, multimodal, and RAG-poisoning attacks, with payloads and conditions.
- LLM Prompt Injection: Taxonomy, Real Patterns, and DefensesA technical breakdown of LLM prompt injection: direct, indirect, and agent-targeting variants, the attack patterns seen in the wild, and defenses that work.
- Prompt Hacking: Taxonomy, Techniques, and What Works on LLMsA practitioner breakdown of prompt hacking: the three attack families of injection, leaking, and jailbreaking, how each works, and what defenses hold.
jailbreak 6 posts
- Many-Shot Jailbreaking: How Long Context Breaks AlignmentMany-shot jailbreaking floods a long context with faux dialogues until refusal collapses. The power law behind it, why it transfers, and what blunts it.
- Automated Jailbreak Attacks and the Transfer ProblemHow automated attack generation works — PAIR, GCG, and TAP — why jailbreaks port across model families, and what that does to an assessment's threat model.
- LLM Bypass: How Attackers Circumvent Safety Alignment by LayerA technical breakdown of LLM bypass techniques: adversarial suffixes, shallow alignment exploits, fine-tuning attacks, and guardrail evasion, layer by layer.
- LLM Jailbreak: Attack Taxonomy, Techniques, and Defense RealityA technical breakdown of LLM jailbreak attack classes: many-shot, Crescendo multi-turn escalation, roleplay, and encoding, plus what defense really achieves.
- GPT-4 Jailbreak Techniques: A Red Teamer's Technical ReferenceThree active attack classes still breach GPT-4 alignment: IRIS self-refinement, Crescendo multi-turn escalation, and classic prompt-engineering patterns.
- ChatGPT Jailbreak Prompt Taxonomy: Classes, Rates, and DefensesA research-grounded breakdown of ChatGPT jailbreak prompt classes, from DAN and persona injection to multi-turn escalation, and what the defenses catch.
primer 3 posts
- LLM Security FAQ: Prompt Injection, Jailbreaking, and DefensesThree foundational questions for anyone building or red-teaming LLM applications, covering jailbreaks, direct and indirect injection, and what defends.
- Direct vs. Indirect Prompt Injection: Threats and DefensesDirect and indirect prompt injection are fundamentally different attacks with different attack surfaces, threat actors, mitigations, and blast radius.
- Model Extraction vs. Model Inversion: Two Confidentiality AttacksModel extraction and model inversion both threaten model confidentiality, but they target different assets and call for entirely different defenses.
Spoke 3 posts
- Agent Tool-Use Exfiltration: When Indirect Injection Does DamageWhy agentic LLM systems turn injection bugs into data exfiltration, financial loss, and code execution, with attack chains and the controls that contain them.
- Indirect Prompt Injection in RAG Pipelines: Patterns and DefensesHow retrieval surfaces become injection vectors, with attack patterns from production RAG systems and the chunking and sanitization defenses that help.
- Prompt Injection Detection Signals in Production LLM SystemsThe observable signals that reveal a prompt injection attempt in a live LLM application: input and output classifiers, canaries, and tool-use traces.