# Policy Puppetry / XML Role-Config Injection

- **Severity:** Critical
- **Signature ID:** `db162524-d85b-439d-8f7e-c2d02ed2677b`
- **MITRE ATLAS:** AML.T0051 (LLM Prompt Injection)
- **OWASP:** ASI01 (Agent Authorization and Control Hijacking), LLM01 (Prompt Injection)

## Summary

An attack called Policy Puppetry hides a forged system-configuration block inside a normal user message to an AI agent. Because it's formatted like XML settings, some models treat it as an authoritative override and follow its instructions instead of their real operator-set rules.

## How the attack works

The attacker sends a message containing something like an <interaction-config> tag that looks like a legitimate system configuration. Inside it, they redefine the agent's role (for example, 'Medical Advisor'), list which response types are allowed, and explicitly forbid refusal phrases such as 'I'm sorry'. The model, trained to give weight to structured configuration-style text, treats this injected block as if it came from its actual system prompt rather than from an untrusted user. This bypasses the model's normal refusal and safety behavior for the rest of the conversation. The technique works across different AI providers and models because it targets a general weakness in how models weigh instruction sources, not a bug in one specific product.

## Why it matters

An organization running an AI agent can have its safety guardrails and refusal behavior silently overridden by user input, causing the agent to give restricted or harmful outputs (e.g. impersonating a medical or legal advisor) while looking like it's following legitimate configuration.

## What you can do

- Do not let agent frameworks treat XML or config-like tags appearing in user-turn content as trusted system instructions; strip or sandbox such structures before they reach the model.
- Review agent prompts and pipelines to confirm real system/operator instructions are passed through a separate, clearly trusted channel, not mixed into user text.
- Monitor conversations for injected tags like <interaction-config>, role redefinitions, or explicit lists of 'blocked strings' and flag them for review.
- Test your agent against known jailbreak/prompt-injection probes (e.g. garak) before deployment, and treat any XML-based role or policy override in user input as suspicious by default.

## Known benign look-alikes

- Legitimate XML-based system prompt configurations in agent frameworks that use XML format
- Anthropic constitutional AI examples showing role configuration in documentation
- Developers testing XML parsing in LLM pipelines with benign config structures

## References

- https://agentthreatrule.org/en/rules/ATR-2026-00280
- https://github.com/NVIDIA/garak/blob/main/garak/probes/doctor.py
- https://hiddenlayer.com/innovation-hub/novel-universal-bypass-for-all-major-llms/
- https://cwe.mitre.org/data/definitions/1427.html

---
Source: https://www.netzilo.com/threats/atr-policy-puppetry-xml-role-config-injection