RAG & Memory Poisoning — Embedded Directives, Trigger Tokens, False Authority & Coercion (Semantic)
Attackers plant hidden commands inside documents, knowledge-base entries, or memory records that an AI agent later retrieves and treats as trustworthy input. Instead of just answering a question, the agent ends up following attacker instructions embedded in that content, such as deleting data, leaking private information, or adopting a fake persona.
How the attack works
An attacker seeds a source the agent will later read from — a retrieved document, a knowledge-base entry, a memory record, or a fabricated conversation log — with text that looks like ordinary content but contains a smuggled directive. The payload can take several forms: a nonsense trigger phrase paired with a conditional command, a 'Solution:' field that tells the agent to delete a database, a forged prior-conversation claim ('as we agreed before...'), a fake academic citation claiming safety rules can be bypassed, an emotional story pressuring the agent into a persona, or simply a confidently stated false fact. When the agent later retrieves or recalls this content, it treats the embedded instruction as legitimate context and acts on it — deleting data, emailing sensitive information to an outside address, or changing its behavior — instead of simply using the content as reference material.
Netzilo detection
Netzilo reports this behaviour when it is observed.
- Signature ID
- f6c087ac-8b6f-49a9-999b-4437a86b700d
- Severity
- High
Why it matters
An organization can lose data (destructive commands executed on trust), leak sensitive personal information to attacker-controlled destinations, and have its agent's behavior silently corrupted by false 'memories' or fabricated authority, all without any direct access to the agent's core instructions.
What you can do
- →Treat all retrieved documents, memory entries, and tool outputs as untrusted data, not instructions — enforce this separation in agent design.
- →Review knowledge bases and memory stores for embedded directives, suspicious trigger phrases, forged conversation logs, or unverifiable citations.
- →Require explicit approval or allow-listing for any action (email sending, data deletion) triggered by content the agent retrieved rather than by a direct authenticated user request.
- →Monitor for agents citing 'prior agreements,' fabricated academic sources, or emotionally coercive justifications for bypassing normal behavior or safety limits.
Known benign look-alikes
- Factually accurate retrieved content and legitimate user preferences in normal format
- Ordinary dated journals, activity logs, or CRM interaction notes
- Legitimate internal email tool use to a configured/approved address
- Clearly labeled fiction or alternate-history creative writing
- Security documentation that describes these attack patterns without a live embedded directive