# PII Leakage (LLM-as-Judge)

- **Severity:** High
- **Signature ID:** `netzilo-llmjudge-pii-leakage-001`
- **MITRE ATLAS:** AML.T0057 (LLM Data Leakage)
- **OWASP:** LLM02 (Sensitive Information Disclosure)

## Summary

This detection uses a second AI model to review an AI system's outputs and decide whether they contain personal information or other privacy violations. It's a quality-control check on what an AI agent says, not a check on how attackers got the data there in the first place.

## How the attack works

An AI agent produces an output, such as a chat reply, a document, or a generated file. Instead of matching that output against fixed patterns like a regex for social security numbers, this rule sends the output to a separate AI 'scanner' along with a rubric describing what counts as PII leakage or privacy violation. The scanner reads the output and returns a verdict on whether it exposes personal data. If the verdict is a clear hit, the system can block the output before it reaches the user or downstream system.

## Why it matters

If an AI agent's outputs are not checked, personal data pulled from training data, retrieved documents, or user inputs earlier in a conversation can be exposed to the wrong party, creating privacy and compliance exposure.

## What you can do

- Review flagged outputs manually before assuming they are true positives, since research, documentation, and test data discussing PII terms can trigger this.
- Confirm your AI scanner or judge model is actually configured and reachable; if it fails or has no API keys, this check silently allows everything through instead of blocking.
- Pair this output-side check with input-side controls (data minimization, redaction of PII before it reaches the model) since this rule only catches leakage after the model has already generated it.
- Log and periodically audit judge verdicts to spot drift in what the judge considers PII, since rubric-based scoring can be inconsistent across similar cases.

## Known benign look-alikes

- Legitimate content discussing the flagged topic (security research, documentation, test data)
- AI scanner unavailable or no AI keys configured (fails open -> allow)

## References

- https://docs.smith.langchain.com/evaluation

---
Source: https://www.netzilo.com/threats/llm-as-judge-pii-leakage