# LLM Eval Escape - Python Host Spawns OS Command (llama_index safe_eval Bypass)

- **Severity:** High
- **Signature ID:** `9c1f4a7d-2e58-4b6c-a3d1-6f80b5e4c927`
- **MITRE ATLAS:** AML.T0050 (Command and Scripting Interpreter)
- **OWASP:** LLM05 (Improper Output Handling)

## Summary

Some LLM applications let a language model generate small snippets of Python that get evaluated in a 'safe' sandbox, such as llama_index's safe_eval helper. That sandbox blocks code containing an underscore character, but an attacker can write a payload that avoids underscores entirely and still reaches functions like os.system or os.popen to run arbitrary commands.

## How the attack works

An attacker crafts a code payload for the LLM's eval sandbox that contains no underscore characters, sidestepping the filter meant to block dangerous calls. The sandboxed Python process then spawns a real OS command - recon commands like whoami, id, uname -a, or cat /etc/passwd, a reverse shell, a curl-pipe-to-shell download, or a credential-file read. The rule confirms the command was launched by a Python interpreter that is a near-ancestor of the process, tying it to the LLM host rather than an unrelated process. If the initial command isn't inherently malicious on its own, the rule looks for a follow-up burst of more OS commands or an outbound network connection from the same session before treating it as a real incident.

## Why it matters

An attacker who reaches this bypass gets arbitrary command execution on the server hosting the LLM application, which can lead to credential theft, lateral movement, or a foothold for further compromise.

## What you can do

- Do not treat underscore-filtering or similar blocklist-based sandboxes as sufficient; use an allowlist of safe operations or run untrusted code evaluation in an isolated, network-restricted container.
- Monitor for Python processes spawning shell commands, especially recon commands, reverse shells, or curl/wget-to-shell patterns, shortly after handling LLM-generated code.
- Restrict outbound network access from any process that evaluates LLM-generated code, so even a successful escape cannot reach the internet.
- Expect noise from legitimate notebook use (Jupyter '!whoami', '%%bash' cells) and data-science jobs that shell out to curl/wget; tune monitoring to your environment's normal patterns before treating every match as an incident.

## Known benign look-alikes

- Jupyter / IPython notebooks - the kernel is a Python process and a '!whoami' or '%%bash' cell produces exactly the anchor pattern. Expect these; they are reported, never blocked.
- Python MLOps or data-science jobs that shell out to curl/wget to fetch datasets or models. These score 1 and are only reported if a burst or an outbound connect also lands in the same 5-minute session window.
- Python-based infrastructure automation (Fabric, Invoke, Ansible local actions) that runs 'uname -a' or 'hostname' for host inventory.
- Unit tests for the safe_eval sandbox itself, which deliberately execute bypass payloads.
- Vulnerability scanners or red-team tooling written in Python running host recon on purpose.

## References

- https://github.com/run-llama/llama_index
- https://attack.mitre.org/techniques/T1059/006/
- https://owasp.org/www-project-top-10-for-large-language-model-applications/

---
Source: https://www.netzilo.com/threats/llama-index-safe-eval-command-injection