Unsafe LLM Output Execution via Python Inline Interpreter
Some applications ask an LLM to generate Python code for charts or plots and then run that code directly without checking it first. If an attacker can influence what the model outputs, they can slip in commands that run system shells, open network sockets, or decode and execute hidden payloads, gaining code execution on the host.
How the attack works
An application uses an LLM (via tools like PlotAI) to generate plotting code involving matplotlib, seaborn, or pandas. The attacker manipulates the prompt or context so the model's output includes extra code beyond legitimate plotting — for example a call to os.system, subprocess, exec/eval, socket.socket, or a base64-decoded blob. The application then feeds this unvalidated output straight into a Python interpreter (python -c) for execution. Because plotting code has no legitimate need to shell out, open sockets, or decode payloads, that combination in a single command is the tell that something malicious was injected.
Netzilo detection
Netzilo reports this behaviour when it is observed.
- Signature ID
- 7b3f1c2e-9a54-4d61-b8e7-2f4c6a1d09b3
- Severity
- High
Why it matters
An attacker can achieve arbitrary code execution with the privileges of the process running the AI-generated code, potentially leading to data theft, lateral movement, or full host compromise.
What you can do
- →Never execute LLM-generated code directly; run it in a sandboxed, restricted environment with no filesystem, network, or shell access.
- →Validate and statically analyze generated code for dangerous calls (os.system, subprocess, exec/eval, socket, base64 decode+exec) before execution.
- →Restrict the privileges of any process that executes model-generated code so a compromise has limited blast radius.
- →Review logs for python -c invocations that mix plotting library references with process-spawning, networking, or decode-and-execute primitives.
Known benign look-alikes
- Legacy pip / setuptools setup.py shim, which runs exec(compile(getattr(tokenize,'open',open)(__file__).read()...)) in a python -c one-liner \u2014 excluded by filter_benign.
- PEP 517 build backend probes that reference setuptools.build_meta \u2014 excluded by filter_benign.
- Test-harness one-liners that import pytest/unittest/doctest and shell out to a plotting fixture \u2014 excluded by filter_benign.
- Headless-backend and version probes such as python -c "import matplotlib; print(matplotlib.__version__)" \u2014 excluded by filter_benign.
- Documentation and notebook build pipelines (sphinx, nbconvert, nbsphinx, papermill) that execute notebooks containing both plotting and subprocess calls \u2014 excluded by filter_benign.
- Security research, advisory reproduction, and training material that quotes the PlotAI payload verbatim in a command line \u2014 CVE-identifier text is excluded by filter_benign.
- A data-science automation script that genuinely renders a chart and then invokes an external converter (e.g. dot, ffmpeg) from the same inline command. Rule is report-only so this does not interrupt work.