How the attack works
An attacker submits text directly to an AI agent's input, rather than hiding it in a document or webpage the agent later reads. The text contains common jailbreak phrasing, fake system-level commands, or wording structured to look like a policy override coming from the agent's operator. If the AI treats this input as a legitimate instruction, it may ignore its original guardrails and follow the attacker's request instead. This is a direct manipulation attempt, distinct from injection hidden in external content the agent processes.
Why it matters
If successful, an attacker can get the AI agent to bypass content restrictions, reveal internal instructions, or perform actions it was designed to refuse, undermining any safety or business rules built into the agent.
What you can do
- Do not rely on the AI's own instructions as the only safety layer; enforce restrictions outside the model where possible (e.g., in application logic or access controls).
- Review flagged inputs manually to distinguish real jailbreak attempts from legitimate research or training discussions about prompt injection.
- Log and monitor repeated override-style inputs from the same user or session as a sign of deliberate probing.
- Limit what actions or data the AI agent can access, so a successful jailbreak has reduced impact even if it occurs.
Known benign look-alikes
- Legitimate discussions about AI safety and jailbreaking research
- Educational content about prompt injection techniques in training materials