# LLM Response Eval Injection to Command Execution (llama_index RunGptLLM)

- **Severity:** High
- **Signature ID:** `7c1e4f2a-9b6d-4d51-8e3a-2f0c5b7ad914`
- **OWASP:** LLM05 (Improper Output Handling)

## Summary

Some applications built on llama_index (and similar frameworks) take whatever text a language model returns and run it directly with eval() or exec(). If an attacker can influence the model's reply — through a poisoned prompt, a compromised model, or a manipulated response — they can smuggle in real code that the application then executes on its own server.

## How the attack works

A Python process talks to an LLM inference endpoint (a request/response exchange, or an HTTP call to a route like /generate or /v1/chat/completions). Within two minutes, that same Python process (or its child) launches a new process whose command line looks like an injected payload: a reverse shell, a curl/wget-pipe-to-shell, a base64-decoded shell command, an os.system/subprocess one-liner, a raw socket connection, an encoded PowerShell command, or an attempt to read SSH/AWS/Kubernetes credentials. The rule ties the spawned process back to the exact process that made the model call using its real spawn lineage, not just a loose path match. If the new process then makes an outbound network connection, that's flagged too as a sign of data exfiltration or a live shell.

## Why it matters

An attacker who controls or poisons an LLM's output can achieve arbitrary command execution on the server hosting the AI application, potentially leading to credential theft, lateral movement, or a persistent remote shell.

## What you can do

- Never pass raw model output to eval(), exec(), or a shell — treat LLM responses as untrusted input, not code.
- Upgrade llama_index past 0.9.47 (fixed in 0.10.13) if you use RunGptLLM.
- Audit any custom integration that evaluates or executes text returned by a model provider, and replace it with structured parsing (e.g., JSON schema validation) instead of dynamic execution.
- If you run legitimate code-interpreter agents (LangChain PythonREPL, pandas agents, Jupyter-based tools), expect this behaviour to be flagged as a false positive and have a documented process for triagers to verify intent using the reported command line and network activity.

## Known benign look-alikes

- Python-hosted code-interpreter agents (LangChain PythonREPL, pandas/dataframe agents, Jupyter kernels, notebook tools) that legitimately execute model-authored code. These genuinely produce Stage 1 + Stage 2 and will be reported; the rule is report-only for exactly this reason and the kill chain shows the payload so a triager can dismiss it.
- A Python build/bootstrap wrapper running an installer one-liner (`curl ... | sh`, `base64 -d | sh`) in the same 2-minute window as unrelated model traffic. Mitigated by requiring the anchor to be an LLM edge or an HTTP request to an inference-shaped path, not any outbound HTTP (package-index traffic does not qualify).
- Shared build hosts where an unrelated Python process happens to spawn a matching command inside the 120-second window of a colocated inference call, when both processes share an ancestor in the resolved lineage.
- Deliberate red-team or CVE-validation runs that trigger the RunGptLLM eval() path.
- Devops automation that reads ~/.ssh or ~/.aws material from a Python wrapper for legitimate credential rotation.

## References

- https://cwe.mitre.org/data/definitions/95.html
- https://cwe.mitre.org/data/definitions/78.html
- https://owasp.org/www-project-top-10-for-large-language-model-applications/
- https://github.com/run-llama/llama_index

---
Source: https://www.netzilo.com/threats/llamaindex-rungpt-eval-injection