vLLM Structured-Decoding Grammar Cache Flood
An attacker sends many structured-output requests to a vLLM inference server, each with a slightly different JSON schema or grammar. vLLM compiles and saves a new artifact to disk for every distinct schema, so a stream of unique schemas fills up disk space or resource capacity over time.
How the attack works
The attacker calls the guided-decoding endpoint (guided_json, json_schema, guided_grammar, guided_regex, or guided_choice) repeatedly, making sure each request body is different so it can't reuse a cached schema. vLLM's outlines library compiles each new schema into an FSM/grammar artifact and writes it to its on-disk cache. Because none of the schemas repeat, none of the cached artifacts get reused, and the cache grows without bound. Over a sustained burst this consumes disk space and compilation resources on the inference host.
Netzilo detection
Netzilo reports this behaviour when it is observed.
- Signature ID
- 7b3c1f9e-4d2a-4f6b-9c81-2e5a6d0f38b4
- Severity
- High
Why it matters
The inference server's disk fills up or its compilation resources get exhausted, degrading or stopping service for legitimate users. This is a resource-exhaustion / availability issue, not a data breach or code execution.
What you can do
- →Cap or quota the number of distinct guided-decoding schemas a single client can submit per time window.
- →Set a size limit and eviction policy on the outlines/grammar FSM cache directory so it cannot grow unbounded.
- →Monitor disk usage on inference hosts and alert on rapid growth in the grammar/FSM cache directory.
- →Rate-limit structured-output requests per client/API key, separate from general request rate limits.
Known benign look-alikes
- Model-evaluation or benchmark harnesses that generate a per-item JSON schema (e.g. one schema per test row) and fire hundreds of guided-decoding requests in a burst. Mitigated by the 10 minute window, the 25-unique-schema floor, the 15 minute per-host report cooldown and report-only enforcement.
- Agent frameworks that inline a dynamically built tool/response schema on every call, so each request body differs. These usually reuse the same small set of tool schemas, which collapses to few fingerprints; only genuinely novel schemas per request accumulate.
- Fuzzing or conformance test suites that intentionally mutate guided_json / guided_grammar payloads against a staging inference server.
- Load or soak testing of a structured-output endpoint with randomised schemas.
- A shared multi-tenant inference host serving many unrelated applications, where the aggregate distinct-schema count is high without any single abuser.