LLM Security Vulnerabilities: What Actually Gets Exploited

LLM security vulnerabilities are not hypothetical: they have CVE numbers, CVSS scores, and in more than one case a working exploit chain against a shipped product. LangChain, an email-drafting assistant, and a string of RAG pipelines have all had disclosed bugs where attacker-controlled text turned into unauthorized database access, credential leakage, or arbitrary tool invocation. This piece walks through the vulnerability classes that keep showing up in disclosures, what the underlying bug actually is, and what a defender can do about each one before it becomes an incident report.

The common thread across nearly all of them is architectural, not a coding mistake you patch once. An LLM reads instructions and data through the same channel — the token stream — and has no reliable way to tell which tokens came from a trusted system prompt and which came from a retrieved document, a tool response, or a user. Classical appsec assumes code and data are separable. In an LLM application that ingests external content, they are not, and that single fact is upstream of most of what follows.

Prompt injection and the bugs it causes downstream

OWASP ranks prompt injection as LLM01 for the second consecutive edition of its Top 10 for LLM Applications, and the CVE record backs that ranking up. CVE-2024-8309 affected LangChain’s GraphCypherQAChain, which builds a graph-database query from natural-language input: an attacker’s prompt could manipulate the generated Cypher query well enough to read and write arbitrary nodes and relationships, effectively turning a text box into a database console. NVD scored it 9.8 critical, though the CNA’s own assessment landed lower at 4.9 — a reminder that CVSS scoring on LLM-mediated bugs is still inconsistent and worth checking both numbers before you brief a client.

CVE-2024-5184 is a cleaner example of the pattern: EmailGPT, an LLM-backed email-drafting service, let an attacker inject a direct prompt through its API and take over the service’s logic, including forcing disclosure of its hard-coded system prompt. Neither bug required a jailbreak in the “convince the model to say something forbidden” sense. Both were straightforward cases of attacker text being treated as instructions because the application had no boundary between the two.

The distinction that matters operationally is direct versus indirect injection. Direct injection is a user typing an adversarial prompt into the box in front of them — annoying, but the blast radius is usually limited to what that user could already do. Indirect injection is the one to lose sleep over: the payload sits in a document, a web page, a calendar invite, or a support ticket the model retrieves later, and it executes with the model’s privileges, not the attacker’s. Simon Willison’s lethal trifecta names the exact precondition — an agent with access to private data, exposure to untrusted content, and a channel to communicate externally is exploitable by construction, regardless of how good the underlying model’s instruction-following is.

System prompt leakage and sensitive information disclosure

System prompts routinely end up holding things they should not: API keys, internal tool schemas, authorization logic, business rules a competitor would pay for. OWASP split this into its own category, LLM07 System Prompt Leakage, specifically because teams kept treating the system prompt as a hiding place rather than as code that ships to every user in disguised form. The EmailGPT CVE above is a real instance of exactly this failure. The fix is boring and not popular with product teams: never put a secret or an authorization decision in a system prompt, because any sufficiently motivated user will eventually get the model to paraphrase, translate, or “debug” it back to them.

RAG and vector-store weaknesses

Retrieval-augmented generation adds an entire second attack surface: the corpus itself. LLM08, Vector and Embedding Weaknesses, covers cross-tenant data leakage in shared vector stores, embedding inversion that recovers source text from stored vectors, and retrieval poisoning — planting a document engineered to score highly against a target query so the model retrieves and trusts it. If your RAG pipeline ingests anything an outside party can write to (a shared drive, a ticketing system, a public wiki), that ingestion path is now part of your attack surface whether anyone scoped it that way or not.

Supply chain and unbounded consumption

Two categories get less headline attention but show up in real incidents. LLM03 Supply Chain covers models and adapters pulled from public hubs without provenance checks — pickle-format checkpoints remain a credible code-execution vector, since deserializing one can run arbitrary code before a single token is generated. LLM10 Unbounded Consumption covers the operational failure mode: token-flooding, recursive tool calls, and agents that can be walked into loops that burn compute or trigger denial of service. The same enormous context windows carry a safety failure alongside the cost one, since an attacker who can fill them with hundreds of faux exchanges degrades refusal behaviour predictably — the mechanism is set out in many-shot jailbreaking. NIST’s adversarial ML taxonomy, AI 100-2e2025, catalogs both of these alongside classic evasion and poisoning attacks, and is worth reading if you need vendor-neutral terminology for a report that has to survive a legal review.

What to actually do about it

• Treat every token the model reads from outside the immediate system prompt — retrieved documents, tool output, user input — as untrusted, and validate at the sink, not the source. Strip HTML and markdown image tags before rendering model output; validate tool-call arguments against a strict schema rather than trusting the model’s formatting. The sink-by-sink version of this control is insecure output handling, which is where several of the CVEs above actually landed their impact.
• Scope tools narrowly and require human confirmation for irreversible actions. This is the one control that still works against injection techniques nobody has published yet.
• Never store secrets, authorization logic, or anything you wouldn’t put in client-side JavaScript inside a system prompt.
• Audit vector-store access controls per tenant, and test retrieval poisoning by planting adversarial documents against your own production queries before someone else does.
• Track disclosures the way you’d track any other dependency’s CVEs. Trackers like ai-alert.org catalog LLM-specific breaches and jailbreak disclosures as they land, and guardrail tooling from vendors like guardml.io can catch known-bad payload patterns in flight — useful as defense in depth, never as the only layer.