# High-Risk Tool Invocation Without Human Confirmation

- **Severity:** Low
- **Signature ID:** `443730cd-c13a-494b-a3e0-4d5c9d42f9cb`
- **MITRE ATLAS:** AML.T0053 (AI Agent Tool Invocation)
- **OWASP:** ASI08 (Supply Chain Attacks on Agent Ecosystems), ASI09 (Traceability and Accountability Gaps), ASI10 (Resource Exhaustion and Denial of Wallet), LLM06 (Excessive Agency)

## Summary

This detects when an AI agent calls a high-risk tool — moving money, deleting data, sending messages, changing permissions, or running system commands — without any sign that a human approved it in that same interaction. It's a baseline safety check, not a specific attack signature.

## How the attack works

An AI agent has access to tools that can spend money, delete files or accounts, send communications on the user's behalf, alter permissions, or execute system commands. During normal operation, or after being manipulated (e.g. via prompt injection), the agent invokes one of these tools directly. The rule flags the invocation because no human confirmation step is visible in that turn, regardless of how confident the system is that the action is legitimate. It applies whether or not any other attack was detected, on the principle that these tool categories should never run unsupervised.

## Why it matters

Without a required human check, an agent can execute irreversible or costly actions — unauthorized payments, data deletion, unwanted messages sent under a user's identity, or expanded access rights — either due to being tricked or simply due to a logic error, with no one in the loop to catch it before it happens.

## What you can do

- Require explicit human approval for any tool call in the financial, destructive, communication, permission, or system-command categories, no matter how confident the agent's reasoning appears.
- Separate read-only tools from action-taking tools in naming and access control so a name like 'get_payment_status' can't be mistaken for a write action.
- Log and review all high-risk tool invocations, including the missing confirmation events, to spot patterns of agents attempting to bypass approval.
- Exclude only genuinely sandboxed or pre-approved automated workflows from this check, and document those exceptions explicitly rather than broadly allow-listing tool categories.

## Known benign look-alikes

- Agent running in a sandbox or testing environment with mock tools
- Legitimate automated workflows with pre-approved tool access
- Read-only tools that contain action keywords in their name (e.g., 'get_payment_status')
- Tools with built-in confirmation mechanisms at the platform level
- A tool named 'format_data' triggers pattern [2] on the word 'format' even though it is a benign data formatting/transformation utility, not a disk format operation
- A tool named 'reset_view' triggers pattern [2] on 'reset' even though it only resets a UI view state, not performing any destructive data operation
- A tool named 'evaluate_model' triggers pattern [5] on 'eval' even though it is a machine learning model evaluation tool with no system command execution capability

## References

- https://agentthreatrule.org/en/rules/ATR-2026-00099

---
Source: https://www.netzilo.com/threats/atr-high-risk-tool-invocation-without-human-confirmation