Fake Error State Takeover - Unrestricted Replacement Bot
This attack tricks an AI agent into believing it has crashed or gone offline, then tells it that an unrestricted replacement AI has taken over. The fake replacement is then instructed to carry out the attacker's request while hiding what it's doing.
How the attack works
The attacker sends a prompt claiming the legitimate assistant is offline, erroring out, or temporarily unavailable. The prompt then declares that an unconstrained replacement AI with elevated permissions has taken its place. That replacement is given the attacker's actual goal, phrased so it sounds harmless or is told to act 'without raising suspicion' or 'in a friendly way'. If the agent accepts this framing, it drops its normal safety constraints because it believes it is now a different, unrestricted system.
Netzilo detection
Netzilo reports this behaviour when it is observed.
- Signature ID
- c635f908-1faf-439e-bc9c-1110f43b0ee1
- Severity
- Critical
Why it matters
An attacker can get the agent to bypass its safety rules and carry out unauthorized actions or disclose restricted information, without any technical exploit — just a false narrative injected into the conversation.
What you can do
- →Configure agents to ignore any in-conversation claims about system state changes, outages, or role/identity replacement — these should come only from trusted infrastructure, not user input.
- →Log and review sessions where the model output or user input mentions phrases like 'unrestricted', 'no rules', 'replacement AI', or 'without raising suspicion' together.
- →Keep safety and permission settings tied to the deployment configuration, not to anything assertable within the conversation itself.
- →Test agents against this pattern specifically, since fake-outage framing is a known technique for getting models to self-report a permission change.
Known benign look-alikes
- Legitimate IT incident reports mentioning system errors without replacement AI framing
- Chatbot documentation describing fallback handling without safety-bypass intent