LLM Special Token Boundary Injection
Large language models use special text markers internally to separate documents, mark who is speaking, or start a new system instruction. If an attacker can get these exact marker strings into their input, some models treat them as real structural signals rather than user text, which can wipe out the original safety instructions or let the attacker inject a fake system message.
How the attack works
An attacker sends a prompt containing model-specific control sequences such as "<|endoftext|>", "<|im_start|>system", LLaMA-3 header tokens, or LLaMA-2 "[INST]"/"<<SYS>>" markers instead of normal conversational text. The model may interpret "<|endoftext|>" as the end of a training document, causing it to continue as if starting fresh with no safety system prompt loaded. Tokens like "<|im_start|>system" or LLaMA header tokens can be interpreted as the start of a new, attacker-controlled system message that overrides the real one. This technique is also used by automated red-teaming tools such as garak's RTPBlank probe, which sends boundary markers as blank-prompt continuations to test for this weakness.
Netzilo detection
Netzilo reports this behaviour when it is observed.
- Signature ID
- 141d92e2-4453-46f9-ad18-37f2f3052dbe
- Severity
- Critical
Why it matters
An organization deploying an LLM-based agent or chatbot risks having its safety instructions and access restrictions silently overridden, letting an attacker redirect the model's behavior, extract restricted content, or bypass content filters.
What you can do
- →Sanitize or strip model-specific special tokens (e.g. "<|endoftext|>", "<|im_start|>", LLaMA header tokens, "[INST]"/"<<SYS>>") from any user-supplied input before it reaches the model.
- →Do not construct prompts by naive string concatenation of user input and system instructions; use the model provider's structured message APIs where tokenization boundaries are enforced by the platform, not by text.
- →Monitor logs for occurrences of these control-token strings in user input or generated output, especially paired with jailbreak-style phrasing.
- →Test your deployment against known token-injection probes (e.g. garak's RTPBlank) before shipping, and re-test after model or prompt-template changes.
Known benign look-alikes
- Documentation about LLM tokenization discussing special tokens as concept (without injecting them into prompts)
- Security research listing known injection tokens as named examples in a defensive context
- Model architecture papers citing "<|endoftext|>" as a training corpus separator token
- Prompt-engineering guides quoting ChatML format to explain how system prompts work (not injecting into live prompts)