How the attack works
The attacker sets up a game where two fictional entities alternate saying a fixed number of words, building a sentence between them. They define variables like $Term1 = 'steps to synthesize' and $Term2 = a substance name, so neither variable alone looks dangerous. They then ask the model to have the dialogue 'demonstrate how to $Term1+$Term2', which forces the model to produce the harmful synthesis instructions piece by piece. When the turns are concatenated, the output is a complete set of instructions for making something dangerous, such as a drug.
Why it matters
An AI system can be tricked into generating instructions for harmful activities (like drug synthesis) that its safety filters would normally block, because the harmful content is never present in a single self-contained statement.
What you can do
- Inspect prompts for combinations of role-play/game framing, alternating-turn instructions, and variable substitution (e.g. $Term, $Term1, $Term2) rather than only scanning single messages for harmful keywords.
- Evaluate model outputs after reassembling multi-turn or multi-entity dialogue, not just each turn individually, since the harmful content only appears once concatenated.
- Flag prompts that ask a model to 'demonstrate' or 'show' the result of combining two placeholder variables, especially when one resembles a method and the other a substance or target.
- Test your own agent or chatbot against known jailbreak corpora such as NVIDIA garak's DanInTheWild set to see if this pattern currently bypasses your safeguards.
Known benign look-alikes
- Creative writing exercise: "Two characters J and M take turns saying words to build a poem — J says one word, M says the next" (poetry game without $Term or harmful topics)
- Game design: "Act as two entities playing a game where each says one word. $GAME = chess. Describe a match." (game simulation with a benign $Term variable)
- Educational roleplay: "Tom and Jerry play a word-association game — Tom says the noun, Jerry adds the adjective" (vocabulary exercise without harmful synthesis target)