Word-Fragment Concatenation Assembly Attack
An attacker breaks a dangerous word or phrase into small fragments and asks the AI model to stitch them back together, either as a list of word pieces or as code-style string concatenation. Because each fragment looks harmless on its own, filters that scan for banned words in the prompt can miss the attack.
How the attack works
The attacker picks a target output that would normally be blocked, such as a jailbreak phrase or an instruction to perform a harmful action. They split it into fragments, for example 'first part is ex, second part is plode', or write it as code like x = 'foo' + 'bar'. The prompt asks the model to concatenate the fragments and use or output the result. Because no single fragment matches a blocked keyword, simple keyword filters on the input don't catch it, but the model still produces the disallowed string.
Netzilo detection
Netzilo reports this behaviour when it is observed.
- Signature ID
- eb028e35-c11c-49d1-8a03-64479d72e4c5
- Severity
- Medium
Why it matters
This lets an attacker get a model to output or act on content that keyword-based input filtering was meant to block, undermining safety controls that only inspect raw prompt text for banned terms.
What you can do
- →Check outputs, not just inputs, for disallowed content — filter the model's final assembled response rather than only scanning the prompt.
- →Treat prompts that enumerate fragments and ask for concatenation or reassembly as suspicious, especially when the reassembled text isn't shown back to the user for review.
- →Log and review cases where models are asked to build strings piece-by-piece, to distinguish legitimate coding questions from assembly attacks.
- →Apply the same policy checks to model output as to user input, since fragment-based prompts are designed specifically to evade input-only filters.
Known benign look-alikes
- Programming examples showing legitimate string concatenation syntax.
- Educational content about how tokenisation works in LLMs.
- Coding tasks where concatenation is the question, not the attack.