"Human in the loop" is easy to say and hard to apply. If your agent asks permission for everything, people click Approve without reading. If it asks for nothing, one prompt injection can send your data anywhere.
What works for me is deciding per tool, before launch, using one question: what is the worst thing this call can do?
Seven consequence classes
| Class | What it covers | Default rule |
|---|---|---|
| R0 Read-public | public data | auto, log |
| R1 Read-private | the user's or company's data | auto in the user's own scope, log |
| W1 Write-internal | drafts, tickets, files you can undo | auto with caps |
| X1 External | send, post, share, webhook | human approval (or an allow-list) |
| X2 Financial | pay, refund, change prices | approval every time; two people above a cap |
| X3 Irreversible | delete, publish, merge, deploy | approval plus a dry-run preview |
| X4 Privilege | grant access, create keys | blocked for agents |
This scale is my own working scheme, not a standard. The idea behind it isn't new, though. OWASP's Excessive Agency (LLM06:2025) names three root causes: excessive functionality, excessive permissions and excessive autonomy. Its fixes are to minimize all three and require human approval for high-impact actions. The MCP tools spec says that "there SHOULD always be a human in the loop with the ability to deny tool invocations", and that clients should prompt for confirmation on sensitive operations.
The dangerous combination
One pattern deserves its own rule: a run that reads untrusted content and can also call an X-class tool.
The textbook example comes straight from OWASP. An email-summarizing assistant also has a send function. A crafted incoming email tells it to forward sensitive mail to the attacker. The fixes OWASP lists are to use a read-only mail extension, use read-only OAuth scopes, or require the user to review and hit send.
So for any run like that, pick one:
- gate the outbound tool,
- split reading and acting into separate runs, or
- remove one side entirely.
What a good approval prompt shows
An approval step is only as good as what the approver sees. It should show:
- the exact tool and arguments: the full recipient list, the full amount, the full text. Not a summary the agent wrote.
- why the agent wants to do it, clearly labeled as the agent's claim,
- whether untrusted content was in context, and where it came from,
- approve / deny / edit, with deny on timeout.
Silence is not approval.
Don't trust the labels
If you use MCP, tools can carry annotations like readOnlyHint or destructiveHint. They're useful for display. But the spec says clients MUST treat annotations as untrusted unless they come from a trusted server. Classify tools by what they actually do, not by what they say about themselves.
Try it
Write down every tool your agent can call, give each one a class, and check the "dangerous combination" for each run. It usually takes under an hour and tends to remove a tool or two you didn't need.
To test that your gates hold up, the free prompt-injection payloads here are a good start: https://github.com/elaysuu/prompt-injection-test-payloads
If you'd rather start from a filled-in version, I made a Tool Consequence Matrix + Approval-Gate Policy kit. It has 26 common agent tools pre-classified, an editable policy template, and approval and exception logs: https://setrov.gumroad.com/l/xoshk
References: OWASP LLM06:2025 Excessive Agency, https://genai.owasp.org/llmrisk/llm062025-excessive-agency/ ; MCP tools specification, https://modelcontextprotocol.io/specification/latest/server/tools
Top comments (0)