LLM data leakage is the exposure of sensitive information through a large language model's outputs, training data, or tool calls. Leakage can be direct, where the model returns a secret it saw at inference time; indirect, where it paraphrases sensitive context past a downstream sanitizer; or structural, where it exfiltrates data through a tool call to a third party.
- Sensitive data escaping through model outputs, training data, or tool calls
- Three paths: direct disclosure, indirect paraphrase, and structural exfiltration
- Rarely a single bad query – usually a workflow no DLP rule was written to see
- Unsanctioned AI use measurably raises the cost when a breach occurs
Why is LLM data leakage important?
LLM data leakage matters because it introduces exposure paths that conventional data controls were not written to see. A DLP rule inspects a file leaving a boundary. It does not inspect a model summarizing that file into an output which then travels somewhere else, and it does not inspect a tool call carrying the same content to a third party.
The cost is measurable where AI use is unmanaged. IBM found that a high level of unsanctioned AI use added $670,000 to the average breach cost, and that customer personal data was exposed in a markedly higher share of those breaches than the overall average. Practical blast radius is shaped by deployment architecture and data classification rather than by a uniform worst case.
The pattern worth internalizing is that leakage is rarely a single bad query. It is usually a workflow – retrieval pulling more context than the task needed, a model paraphrasing it, and a tool call carrying the result outward, with no step individually looking like an exfiltration.
What is LLM data leakage?
LLM data leakage is the unintended disclosure of sensitive information through a language model or the application around it. The information can originate in training data, in context supplied at inference, or in systems the model reaches through tools.
Three mechanisms account for most of it. Direct leakage is the model returning something sensitive it was given or has memorized – a credential in a system prompt, or a record from a retrieved document. Indirect leakage is the model restating sensitive content in a form that passes a downstream sanitizer, because the filter was matching patterns rather than meaning. Structural leakage is exfiltration through the application's own architecture, where the model calls a tool that carries the data to a third party.
The third is the hardest to detect and the most consequential in agent deployments, because the data leaves through a permitted channel performing a permitted operation. Nothing about the traffic looks anomalous.
Types of LLM data leakage
Leakage groups by where the sensitive data entered the model's reach, which determines which control can address it.
Training-data leakage occurs when a model reproduces material it memorized during training. It is the hardest for a consuming organization to address, since the exposure is baked into the model.
Context leakage occurs when data supplied at inference – through a system prompt, a retrieved document, or conversation history – surfaces in an output that reaches someone unauthorized. Over-broad retrieval is the usual culprit, and scoping retrieval is the usual fix.
Output leakage occurs when a response carries sensitive content past inspection, whether through paraphrase, partial disclosure, or inference from non-sensitive details.
Tool-mediated leakage occurs when the model sends data outward through a tool call, which is both the most direct path and the one least covered by content inspection.
User-side leakage completes the set: an employee pasting sensitive material into an unsanctioned tool, where the data leaves before any enterprise control sees it.
LLM data leakage & Onyx
Onyx AI Guard provides inline sensitive data detection across model input and output, so content is evaluated in the request path rather than reconstructed from logs afterward. On Onyx's own benchmark of 21,909 production samples, sensitive data detection is measured at 96.3% accuracy, alongside content moderation at 99.1% and prompt-injection detection at 97.6%, with PII detection and compliance violations assessed on the same benchmark.
Inspection alone would leave the structural path open, which is why enforcement extends to actions. Tool calls are evaluated before they execute, and Steering can mask sensitive elements in a result so an agent continues reasoning without the underlying data. Discovery across browser, endpoint, network, and SaaS surfaces addresses the user-side path by surfacing the unsanctioned tools that data would otherwise leave through unseen.
