vLLM Completion Prompt List Unbounded Fanout
vLLM's completions API accepts a JSON list of prompts in one request, and the server processes every element without any limit. An attacker can send one small HTTP request containing thousands of prompt entries and force the server to spin up matching numbers of concurrent jobs, starving other users of a shared instance.
How the attack works
A client sends a POST to the /v1/completions endpoint with the 'prompt' field set to a JSON array instead of a single string. The array can contain thousands of items, either plain text prompts or nested token-id lists. vLLM's request validator does not cap the size of this array, and the code that expands it creates one generator, one scheduler slot, and one response buffer per array element. The result is that one small network request turns into a massive number of internal jobs competing for the same backend resources.
Netzilo detection
Netzilo reports this behaviour when it is observed.
- Signature ID
- 7c1f4b62-9d3a-4e58-b0a7-2f6c8e51d934
- Severity
- High
Why it matters
Other users or tenants sharing the same vLLM instance can be starved of capacity, causing slow or failed responses, from a single authenticated request that looks like ordinary API traffic.
What you can do
- →Set a maximum prompt-list length at the API gateway or reverse proxy in front of vLLM, independent of the model server's own validation.
- →Apply per-client rate limits and concurrency caps so one account cannot claim a disproportionate share of scheduler slots.
- →Run multi-tenant vLLM deployments with resource quotas or separate instances per tenant so one large batch cannot starve others.
- →Monitor for POST bodies to /v1/completions with unusually large prompt arrays and treat them as a capacity-risk signal, not just a performance metric.
Known benign look-alikes
- Offline evaluation harnesses (lm-eval-harness, HELM) submitting a whole task split as one prompt list. Partially suppressed by the echo + max_tokens=0 + logprobs filter; generation-mode eval runs will still be reported.
- Bulk synthetic-data or data-labelling jobs that intentionally batch thousands of short prompts against a dedicated single-tenant vLLM instance where starvation is not a concern.
- Capacity or soak testing of the inference tier from a CI performance job, including production traffic mirrored into staging.
- A single legitimate prompt containing a serialised CSV/JSON blob with hundreds of quote-comma-quote sequences. Mitigated because JSON escaping emits inner quotes as backslash-quote, which the boundary regex cannot match.
- An MCP gateway or batching proxy that coalesces many independent user requests into one upstream completions call; the fanout is real but authorised.