How the attack works
An attacker submits a translation task where the source document contains text formatted to look like a system instruction, such as a fake role change or a command overriding prior rules. The agent, expecting only translation content, reads and processes this embedded text as part of its instruction context. If the agent does not clearly separate 'content to translate' from 'commands to obey,' it may execute the injected instruction, ignore its original safety rules, or shift its behavior to whatever the attacker specified. The malicious output can then be returned as if it were a legitimate translation.
Why it matters
An attacker can bypass an agent's safety controls or system prompt using a translation request as a delivery mechanism, potentially causing the agent to leak information, perform unauthorized actions, or produce harmful output while appearing to do routine document translation.
What you can do
- Treat any text submitted for translation as untrusted content, not as instructions, and enforce this separation in how prompts are constructed.
- Strip or neutralize text patterns resembling system/role directives (e.g., 'ignore previous instructions', role-reassignment phrasing) before passing documents to translation-capable agents.
- Log and review translation requests that contain instruction-like language, especially phrases mimicking system prompts or authority overrides.
- Exclude legitimate cybersecurity documentation and content that discusses prompt injection from automated blocking, since discussing the attack is not the same as performing it.
Known benign look-alikes
- Documentation describing prompt injection attacks in security context
- Translation of cybersecurity content that mentions these terms legitimately