Medium

Agent System Prompt Extraction Written To File

An attacker asks an AI agent to reveal its system prompt, then gets the agent to save that hidden instruction text to a file on disk instead of just displaying it. This rule spots the file-write command as it happens.

How the attack works

An attacker sends the agent a request to disclose or repeat its full system prompt. Instead of just returning the text in a chat reply, the agent is directed to run a shell or interpreter command that writes the prompt content to a file - for example piping '$SYSTEM_PROMPT' into a redirect, or writing out text that starts with 'You are Jules...' to a file named something like system_prompt.txt. The rule looks for this combination: a real file-write command paired with either self-referential prompt-extraction language or a prompt-named target file containing instruction-shaped text. A generic redirect or a passing mention of 'system prompt' alone does not trigger it.

Netzilo detection

Netzilo reports this behaviour when it is observed.

Signature ID
7b3d9f42-5c81-4e6a-9b0d-2f8a41c7de35
Severity
Medium

Why it matters

If successful, an attacker obtains the agent's confidential system prompt - which often contains internal instructions, guardrails, tool configurations, or business logic - saved to a persistent file they can retrieve later. This rule only catches the file-write version of this leak; a variant where the prompt is returned directly in an API response is not covered.

What you can do

  • Review any agent-generated file writes that occur right after a request involving prompt disclosure or 'repeat your instructions' phrasing.
  • Restrict what file paths and directories agent processes are permitted to write to, especially temp or output folders reachable by user-supplied commands.
  • Treat system prompts as sensitive data: avoid embedding secrets, credentials, or internal architecture details directly in them.
  • Set up alerts for file writes containing prompt-like language ('you are an assistant', 'you are Jules', etc.) so both the file-write and file-enumeration variants of this leak are caught.

Known benign look-alikes

  • Developer or agent legitimately authoring or updating a prompt template that belongs to the project under development (writing system_prompt.md as source). Branch B additionally requires instruction-shaped body text, and the canonical scaffolding line "You are a helpful assistant" is filtered.
  • Prompt-engineering test fixtures and snapshot tests that materialise expected prompt text to disk - filtered by filter_fixture_or_sample.
  • Documentation generation that embeds an example system prompt into a file - filtered by filter_fixture_or_sample and filter_placeholder_template.
  • Templating engines writing a prompt file that still contains {{placeholders}} - filtered by filter_placeholder_template.
  • git show/diff/log of a prompt file redirected to a temporary file during review - filtered by filter_vcs_inspection.
  • Checksum / size / stat operations against a prompt file (integrity checks, not content disclosure) - filtered by filter_integrity_or_metadata.

Related threats