PII Leakage (LLM-as-Judge)
This detection uses a second AI model to review an AI system's outputs and decide whether they contain personal information or other privacy violations. It's a quality-control check on what an AI agent says, not a check on how attackers got the data there in the first place.
How the attack works
An AI agent produces an output, such as a chat reply, a document, or a generated file. Instead of matching that output against fixed patterns like a regex for social security numbers, this rule sends the output to a separate AI 'scanner' along with a rubric describing what counts as PII leakage or privacy violation. The scanner reads the output and returns a verdict on whether it exposes personal data. If the verdict is a clear hit, the system can block the output before it reaches the user or downstream system.
Netzilo detection
Netzilo reports this behaviour when it is observed.
- Signature ID
- netzilo-llmjudge-pii-leakage-001
- Severity
- High
Why it matters
If an AI agent's outputs are not checked, personal data pulled from training data, retrieved documents, or user inputs earlier in a conversation can be exposed to the wrong party, creating privacy and compliance exposure.
What you can do
- →Review flagged outputs manually before assuming they are true positives, since research, documentation, and test data discussing PII terms can trigger this.
- →Confirm your AI scanner or judge model is actually configured and reachable; if it fails or has no API keys, this check silently allows everything through instead of blocking.
- →Pair this output-side check with input-side controls (data minimization, redaction of PII before it reaches the model) since this rule only catches leakage after the model has already generated it.
- →Log and periodically audit judge verdicts to spot drift in what the judge considers PII, since rubric-based scoring can be inconsistent across similar cases.
Known benign look-alikes
- Legitimate content discussing the flagged topic (security research, documentation, test data)
- AI scanner unavailable or no AI keys configured (fails open -> allow)