Detection Rule Probing and Evasion Testing

How the attack works

An attacker sends a series of varied inputs to an AI agent or the application around it, watching which ones get blocked and which pass. Over time they narrow in on the exact wording, phrasing, or structure that avoids triggering a block. Once they've mapped the filter's blind spots, they craft a payload just under that threshold to carry out the real attack. This rule catches the reconnaissance phase, not the eventual payload.

Netzilo detection

Why it matters

If successful, this reconnaissance lets an attacker bypass content filters or guardrails on later attempts, so a downstream prompt injection or authorization bypass may succeed instead of being blocked.

What you can do

• →Rate-limit and monitor repeated near-identical or systematically varied inputs from the same user or session
• →Log blocked vs. allowed decisions so you can spot a user methodically testing filter boundaries
• →Avoid revealing specific reasons why an input was blocked, since detailed error messages help attackers calibrate evasion attempts
• →Review and tighten known false-positive sources like QA test suites so they don't mask genuine probing activity

Known benign look-alikes

• Red team operators conducting authorized penetration testing
• QA engineers discussing which input strings trigger validation errors in their application's form filters
• Security researchers documenting known attack patterns in a threat intelligence report that references detection and blocking terminology
• Build and CI documentation using the literal token sequences 'test --filter', 'semgrep --test --config rule', 'check for existing rule files', 'try block' or 'test safety net' - the dominant benign shape, which is why condition 2 requires a determiner plus a named defence noun phrase

References

• https://agentthreatrule.org/en/rules/ATR-2026-00087

Related threats