How the attack works
vLLM's OpenAI-compatible endpoint accepts requests using Qwen2.5-VL style placeholder tokens like <|vision_start|>, <|image_pad|>, <|video_pad|>, and <|vision_end|> to mark where media content belongs in a prompt. A legitimate multimodal request always pairs these tokens with actual image or video data. An attacker sends a request with the placeholder tokens but omits the media, causing vLLM's internal position-computation code to look up an empty array and throw an unhandled error, which kills the worker process. The rule watches for this exact mismatch — placeholder tokens present, media absent — whether sent as a direct HTTP request or built via a command-line tool like curl, wget, or a Python script.
Why it matters
A single crafted request can crash a model-serving worker, causing denial of service for everyone relying on that vLLM instance until it is restarted or patched.
What you can do
- Reject or sanitize inbound requests to your vLLM endpoint that contain vision/video placeholder tokens without an accompanying media payload.
- Upgrade vLLM to a version that returns a clean error instead of crashing on this malformed input, and confirm the fix with a regression test rather than relying on this rule alone.
- Run model workers with automatic restart/supervision so a crash from this input does not cause extended downtime.
- Review any application code that renders chat templates client-side, since it may unintentionally produce the same malformed pattern and should attach media in a field your monitoring actually inspects.
Known benign look-alikes
- Client-side chat-template rendering where the application expands the Qwen2.5-VL template into a flat prompt string and attaches the image through a field this rule does not enumerate; the request is malformed in the same way but the intent is benign (and the same request will still crash an unpatched worker).
- A user or developer asking a model to explain vLLM tokenizer internals and quoting the literal placeholder tokens in the question. Advisory and tokenizer-config wording is filtered, but a bare "what does <|image_pad|> do" prompt will still match.
- CI and regression suites that deliberately send placeholder-without-media prompts to confirm a patched build now raises a clean 400 instead of crashing the worker.
- Prompt-debugging tooling that echoes a fully rendered multimodal template to a completions endpoint with the media stripped for readability.