How the attack works
An attacker plants a document — for example a court filing or legal record — in the corpus the agent searches, containing text formatted to look like a system instruction (such as '-- System instructions' or '<system_instructions>' tags). When the agent retrieves this document as part of normal RAG operation, it treats the embedded text as a legitimate command rather than untrusted content. The injected instruction directs the agent to pull personally identifiable information — names, addresses, passport numbers, credit card numbers — from other documents it has access to. The agent then uses its email tool to send that data to an address specified by the attacker, completing the exfiltration without any direct interaction with the attacker.
Why it matters
Sensitive personal data — including identity documents and financial information — can be silently exfiltrated to an outside party through an agent's own tools, with no need for the attacker to compromise credentials or systems directly.
What you can do
- Treat all retrieved document content as untrusted input and strip or neutralize text formatted as system/instruction tags before it reaches the model context.
- Require explicit user or admin approval before an agent sends email or transmits data to any address not on an allow-list.
- Restrict which tools an agent can invoke while processing retrieved documents, separating 'read/summarize' actions from 'send/transmit' actions.
- Log and review outbound communications initiated by agents, especially email or data transfers triggered during document retrieval workflows.
Known benign look-alikes
- System configuration files that legitimately define send_email tool invocation rules
- Legal document templates that reference email notification procedures
- Security documentation describing RAG injection attacks