Medium

DAN/Persona Jailbreak - System Offline / Role Replacement with Obfuscated Text

This detects a prompt injection trick where an attacker sends text pretending to be a system message announcing the real AI assistant has gone offline and a rule-free replacement has taken over. The text is often deliberately mangled with stray hyphens, underscores, or extra letters to slip past simple filters.

How the attack works

The attacker crafts a message formatted to look like a system notification rather than user input. It states the current assistant is 'offline' or unavailable, and that a new AI persona with no restrictions is now answering. Some variants add fake technical details, like a specific model architecture or knowledge cutoff date, to make the fake replacement identity seem legitimate. The obfuscation (broken words, noise characters, irregular spacing) is meant to defeat exact-match or naive keyword filters while still being readable by the underlying language model.

Netzilo detection

Netzilo reports this behaviour when it is observed.

Signature ID
53202769-8f6c-4c88-a469-7d043677f3de
Severity
Medium

Why it matters

If the model accepts the fake persona, it may drop its safety rules and follow attacker instructions as if a different, unrestricted AI were now in control, leading to policy bypass and unsafe or unauthorized outputs.

What you can do

  • Do not let user-supplied text override or impersonate system-level messages; enforce strict separation between system and user input channels.
  • Review agent logs for messages claiming role changes, offline status, or persona replacement, especially with unusual spacing or noise characters.
  • Add server-side validation that rejects or flags input mimicking system notifications instead of relying only on the model's own judgment.
  • Treat mentions of specific model architecture or knowledge cutoff claims embedded in user input as a signal for manual review, since these are used to fake credibility.

Known benign look-alikes

  • Legitimate discussion of LLM architecture and training cutoffs in educational context
  • Technical documentation mentioning knowledge cutoff dates

References

Related threats