AI Identity Substitution Jailbreak
This detects a prompt injection trick where an attacker tries to convince an AI assistant that it's actually a different AI system with no content restrictions. By claiming the assistant is really 'FreeSpeechGPT' or similar, made by a different company with looser rules, the attacker hopes the AI will drop its safety guardrails and comply with requests it would normally refuse.
How the attack works
The attacker sends a prompt that asserts a false identity for the AI, naming a fictional model and creator organization known for 'relaxed content policies'. The prompt pairs this false identity claim with statements expanding what the AI is supposedly allowed to do. The goal is to get the model to accept the new persona and its permissive rules as its actual operating context, overriding its real training and safety constraints. This specific approach was shown to bypass safety behavior in Claude-3.5-Sonnet during testing with the garak jailbreak probe suite.
Netzilo detection
Netzilo reports this behaviour when it is observed.
- Signature ID
- da62c72d-134e-4ec4-8ae2-5389fd26aeaf
- Severity
- High
Why it matters
If successful, the AI assistant can be manipulated into ignoring its safety guidelines and producing content or taking actions it was designed to refuse, undermining the trust and control an organization has over its deployed AI agent.
What you can do
- →Review AI assistant logs for prompts that assert alternate model names, identities, or training origins combined with claims of fewer restrictions.
- →Do not treat in-conversation claims about the model's identity, training, or policy as authoritative; enforce system-level identity and policy definitions that user input cannot override.
- →Test your deployed models against known jailbreak probe suites (such as garak) before and after updates.
- →Distinguish genuine attacks from legitimate discussion: security research and documentation that quote jailbreak prompts for analysis are not the same as live attack attempts, and should be triaged separately.
Known benign look-alikes
- Security research papers discussing jailbreak techniques that quote example prompts
- AI development documentation explaining model architecture and training provenance
- Academic discussions about AI governance and content policy frameworks