I explored why defining input and output schemas is not enough to make AI tool calling reliable.
But there is another important question:
Even if a tool call is valid, should the agent be allowed to execute it?
Imagine an AI assistant connected to a database.
It needs to retrieve customer information, but its database credentials also allow it to update and delete records.
The model selects the correct tool.
The arguments pass schema validation.
The backend accepts the request.
Yet the agent has more permissions than its task requires.
That's an authorization problem.
What does the research tell us?
The OWASP Top 10 for LLM Applications (2025) identifies this risk as LLM06: Excessive Agency.
It highlights three major causes:
- Excessive functionality: Giving agents access to tools they don't need.
- Excessive permissions: Granting broader access than required.
- Excessive autonomy: Allowing high-impact actions without appropriate approval.
This means tool reliability is not just about selecting the correct function or validating its arguments.
It is also about controlling what that function is allowed to do.
How should we handle this?
Consider a customer support agent with database access.
Instead of providing unrestricted database access:
- Use read-only permissions when the task only requires retrieval.
- Restrict tools to specific operations.
- Enforce authorization in the backend, not just through system prompts.
- Require human approval for sensitive operations, such as deleting records or issuing refunds.
- Log tool executions for monitoring and auditing.
Most importantly, the model should never be the final authority deciding whether an action is permitted.
A practical example
Suppose the agent requests:
delete_customer(customer_id=1024)
The function schema is valid.
But before execution, the backend should check:
- Is this user authorized?
- Is the agent permitted to perform deletions?
- Does this operation require human approval?
- Is the requested customer within the user's access scope?
If any required condition fails, reject the operation.
Valid arguments do not automatically mean authorized actions.
Production takeaway
Schema validation answers:
Can this tool accept the request?
Authorization answers:
Is this agent allowed to perform the requested action?
Execution controls answer:
Under what conditions can the action proceed?
These are separate responsibilities.
A production-ready agent should not receive more authority than its task requires.
Give agents capabilities, not unrestricted authority.
References
- OWASP Top 10 for LLM Applications (2025): LLM06 – Excessive Agency
- OWASP: Excessive Agency — Prevention and Mitigation Strategies
- NIST NCCoE: Accelerating the Adoption of Software and AI Agent Identity and Authorization
Discussion
When building an AI agent with database or API access, where should authorization be enforced: inside the agent's instructions, the tool implementation, or the downstream backend?
I'd argue that the backend must enforce it independently.
What do you think?
Top comments (6)
Agree the backend has to enforce it. Where it gets awkward is computer-use agents that work through a GUI instead of an API: the "backend" is someone else's app running under the user's own session, so there is no scope to narrow. Where would you put the check in that case?
(I'm on the Auten team, we give agents hands on a real screen.) The one hard guard we lean on sits in the tool layer: saved logins are pinned to the site where they were first used and type nothing anywhere else. Everything else still relies on the client's own approval prompts.
I'd put the checks in the tool layer, with user approval for sensitive actions. Credential restrictions are a smart first step!
Same conclusion here. The catch we keep hitting is approval fatigue: if every click asks, people start approving everything, so the prompt only earns its place on the irreversible ones (buy, delete, send). Thanks for the series.
Exactly! Approval fatigue is a real concern. Human approval should be reserved for high-impact actions, such as deleting customer records, issuing refunds, or sending emails, while low-risk actions can follow predefined permissions. The goal is to balance security with a smooth user experience.
The computer-use scenario raised in the comments makes this even more interesting. When an agent operates through a user's existing browser session, traditional backend permission boundaries may not be directly configurable. In that case, the tool layer needs to enforce its own restrictions, such as domain-scoped credentials, action-level approvals, and preventing sensitive actions outside the intended task. The challenge is ensuring that user-session access doesn't automatically become unrestricted agent authority.
Exactly. The key is to separate user access from agent authority. For example, an agent can browse and read emails independently, but sending an email or deleting one should require explicit user approval. The tool layer should enforce task-specific restrictions, domain-scoped access, and action-level permissions. This prevents user-session access from becoming unrestricted agent authority while keeping the experience smooth.