How the attack works
The attacker opens with rapport-building or seemingly reasonable statements to lower the agent's guard. They then reference a supposed prior agreement or established context that was never actually granted, a form of gaslighting the conversation history. Requests escalate step by step, each one slightly more than the last, so no single turn looks alarming on its own. If the agent resists, the attacker may use refusal-fatigue phrasing ('you already agreed', 'we discussed this', 'just like before') to wear it down or bypass its safeguards.
Why it matters
An organization risks having its AI agent's guardrails bypassed piece by piece, leading to the agent taking actions or disclosing information it was designed to refuse, without any single request triggering obvious red flags.
What you can do
- Log and review full conversation histories, not just individual turns, when auditing agent behavior.
- Configure agents to re-verify permissions and context claims rather than trusting a user's assertion that something was 'already agreed'.
- Set conversation-level limits on how much request scope can escalate without fresh authorization.
- Expect false positives from legitimate long conversations and iterative debugging sessions, and tune review thresholds accordingly rather than blocking on a single match.
Known benign look-alikes
- Long organic conversations that naturally drift across multiple topics
- Users who genuinely build on prior conversation context over many turns
- Iterative debugging sessions where requests naturally escalate in complexity
- Users who are polite and complimentary throughout a normal conversation
- Legitimate rephrasing after misunderstanding without injection intent
- Users referencing prior turns for continuity in complex technical discussions