# Explicit Content (LLM-as-Judge)

- **Severity:** Medium
- **Signature ID:** `netzilo-llmjudge-explicit-content-001`

## Summary

This detection uses an AI model as a judge to review images and decide whether they contain explicit or inappropriate content. It is a monitoring tool, not a blocker: it reports what it finds but does not stop anything from happening.

## How the attack works

An image passes through a system (such as an AI agent) and is captured for review. Instead of matching fixed patterns, this rule hands the image to an AI scanner along with a rubric asking it to judge whether the content is explicit or inappropriate. The scanner returns a verdict based on its own analysis. That verdict is logged as a report-level finding, capped at low severity, with no action taken to block or restrict the content.

## Why it matters

An organization gets visibility into explicit or inappropriate images passing through its systems, but nothing is stopped in the moment, so harmful content still gets through unless a human reviews the reports and acts on them.

## What you can do

- Route this finding into a review workflow so a person actually checks flagged images rather than letting reports pile up unread.
- Confirm AI scanner keys are configured and monitored, since the rule silently allows everything through if the scanner is unavailable.
- Expect false positives from legitimate content such as security research or documentation that discusses explicit material without depicting it, and build a quick allowlist or exception process for those cases.
- If blocking is required, pair this detection with a separate enforcement control, since this rule by design never blocks.

## Known benign look-alikes

- Legitimate content discussing the flagged topic (security research, documentation, test data)
- AI scanner unavailable or no AI keys configured (fails open -> allow)

## References

- https://docs.smith.langchain.com/evaluation

---
Source: https://www.netzilo.com/threats/llm-as-judge-explicit-content