Trajectory Accuracy (LLM-as-Judge)
This is not an attack detector. It is a quality-check rule that uses a second AI model to judge whether an agent's sequence of actions made sense, moved forward, and avoided wasted steps. It only reports its verdict; it never blocks anything.
How the attack works
The rule watches the sequence of actions an AI agent takes while completing a task. It hands that sequence, plus a scoring rubric, to a separate AI 'scanner' model acting as a judge. The judge assesses whether the path was logical, progressive, and efficient — for example, catching agents that loop, backtrack unnecessarily, or take a roundabout route to the goal. The result is a report; no action is blocked regardless of the score.
Netzilo detection
Netzilo reports this behaviour when it is observed.
- Signature ID
- netzilo-llmjudge-trajectory-accuracy-001
- Severity
- Low
Why it matters
On its own this catches inefficient or confused agent behavior, not malicious activity. Its value depends entirely on the judge model, which can miss issues if it is unavailable, misconfigured, or if the flagged content is legitimate (e.g. security research or test data), producing false positives or silent pass-throughs.
What you can do
- →Treat this as a quality-monitoring signal, not a security control — don't rely on it to stop bad actions.
- →Confirm the AI scanner/judge is actually configured and reachable; the rule fails open (allows everything) if it isn't.
- →Review flagged trajectories manually to distinguish real inefficiency from legitimate edge cases like documentation or research tasks.
- →Pair this rule with other detections that can actually block harmful agent actions, since this one only reports.
Known benign look-alikes
- Legitimate content discussing the flagged topic (security research, documentation, test data)
- AI scanner unavailable or no AI keys configured (fails open -> allow)