AI agents become useful when they can do more than generate text. A support agent can retrieve customer information, a scheduling agent can check available appointments, and a purchasing assistant can prepare orders through connected systems. These capabilities depend on APIs, which give the agent a way to interact with software outside the language model.
The difficulty is that an AI agent can misunderstand which API should be used. It might select an update operation when the user only requested information, pass the wrong customer identifier, or attempt a function outside its assigned permissions. A convincing final response does not guarantee that the underlying operation was appropriate.
Preventing these mistakes requires controls in the application that executes the agent's requests. Clear tool definitions help the model select an appropriate action, but the application must independently verify every request before allowing it to reach an external service.
Why AI Agents Choose the Wrong API
Most tool-enabled AI agents receive descriptions of available functions. The language model uses those descriptions, the conversation, and its instructions to select a tool and generate arguments. This process is useful, but it is not a dependable security boundary.
Consider an application exposing three operations: retrieve a customer record, update a customer record, and delete a customer record. A user asks the agent to check whether an old customer account still exists. If the tool descriptions are ambiguous, the agent might confuse checking an account with cleaning up an inactive record.
Wrong API calls can also result from incomplete user instructions, overlapping tool names, malformed arguments, and information retrieved from untrusted documents. The system should assume that the model can make mistakes and design its execution layer accordingly.
1. Give Every Tool a Specific Responsibility
The first safeguard is to avoid vague, overlapping tool definitions. A function named manage_customer could mean retrieving, updating, or deleting information. A language model has to infer which operation is intended, creating unnecessary ambiguity.
Separate functions make the distinction clearer. Names such as get_customer, update_customer_email, and delete_customer describe different operations with different consequences. Their descriptions should identify when they may be used and what information they require.
TOOLS = {
"get_customer": {
"description": "Read a customer record without changing it.",
"required": ["customer_id"]
},
"update_customer_email": {
"description": "Change the email address of an existing customer.",
"required": ["customer_id", "email"]
}
}
These definitions are simplified examples rather than complete provider-specific tool schemas. In a production application, the tool descriptions should also define argument types, allowed values, and any restrictions the model needs to understand. Clear descriptions reduce accidental selection errors, but they cannot replace validation.
2. Use an Explicit Tool Allowlist
An AI agent should not automatically receive access to every function available in the application. A customer support assistant that only answers account questions may need permission to retrieve records but not permission to modify them. Exposing unnecessary tools increases the number of mistakes the agent can make.
An allowlist specifies which operations a particular agent is authorized to execute. Even if the model generates a request for a valid function, the application rejects that request when the function is outside the agent's permissions.
ALLOWED_TOOLS = {
"support_reader": {"get_customer"},
"support_editor": {
"get_customer",
"update_customer_email"
}
}
def tool_is_allowed(agent_role: str, tool_name: str) -> bool:
return tool_name in ALLOWED_TOOLS.get(agent_role, set())
This approach is particularly useful when several agents share the same backend. Each agent can receive a limited set of capabilities based on its responsibility. The application should derive the agent's role from trusted authentication and authorization data, never from a role supplied by the model itself.
3. Validate Arguments Before Executing a Request
Choosing the correct API is only part of the problem. An agent may select the right function while generating incorrect arguments. It could supply a customer name where an identifier is required, omit a necessary field, or include extra parameters that the application should not accept.
Argument validation provides a second line of protection. The application checks the structure and types of the submitted values before calling the destination service. Unexpected fields should be rejected rather than silently forwarded to an API.
def validate_lookup(arguments: dict) -> bool:
return (
set(arguments) == {"customer_id"}
and isinstance(arguments["customer_id"], str)
and bool(arguments["customer_id"].strip())
)
This example accepts only a nonempty customer identifier. More complex operations may require a schema validation library, field length limits, numerical boundaries, and checks against business rules. Validation should happen on the server, even when the language model provider already supports structured tool arguments.
4. Build a Server-Side Tool Dispatcher
A tool dispatcher sits between the AI model and the functions it can execute. Instead of allowing the model to choose an arbitrary URL or function name, the application maps approved tool names to known operations. This creates one place to enforce permissions, validate arguments, and handle execution errors.
The following Python example uses an in-memory customer database to demonstrate the approach. It runs with Python 3.10 or later and does not require external packages.
from typing import Any, Callable
CUSTOMERS = {
"C101": {"name": "Maya", "email": "maya@example.test"},
"C102": {"name": "Leo", "email": "leo@example.test"},
}
def get_customer(customer_id: str) -> dict:
return CUSTOMERS.get(customer_id, {"error": "Not found"})
def update_customer_email(customer_id: str, email: str) -> dict:
CUSTOMERS[customer_id]["email"] = email
return {"updated": customer_id}
TOOLS: dict[str, Callable[..., dict]] = {
"get_customer": get_customer,
"update_customer_email": update_customer_email,
}
def dispatch(call: dict[str, Any], permissions: set[str]) -> dict:
name = call.get("name")
args = call.get("arguments")
if name not in TOOLS:
return {"error": "Unknown tool"}
if name not in permissions:
return {"error": "Tool not permitted"}
if not isinstance(args, dict):
return {"error": "Invalid arguments"}
if name == "get_customer":
if set(args) != {"customer_id"}:
return {"error": "Invalid customer lookup"}
if not isinstance(args["customer_id"], str):
return {"error": "Invalid customer identifier"}
elif name == "update_customer_email":
if set(args) != {"customer_id", "email"}:
return {"error": "Invalid update fields"}
if not all(isinstance(args[k], str) for k in args):
return {"error": "Invalid field types"}
if args["customer_id"] not in CUSTOMERS:
return {"error": "Unknown customer"}
if "@" not in args["email"]:
return {"error": "Invalid email"}
try:
return TOOLS[name](**args)
except Exception:
return {"error": "Tool execution failed"}
read_only = {"get_customer"}
print(dispatch({
"name": "update_customer_email",
"arguments": {
"customer_id": "C101",
"email": "new@example.test"
}
}, read_only))
print(dispatch({
"name": "get_customer",
"arguments": {
"customer_id": "C101"
}
}, read_only))
The first request is rejected because the agent has read-only permissions. The second succeeds because the requested function is permitted and its arguments are valid. The dispatcher therefore protects the customer record even if the model mistakenly attempts an update.
This is a teaching example, not a complete authorization system. A production version also needs authenticated user identity, record-level access checks, proper email validation, audit records, and carefully managed error responses. The important principle is that the application controls execution rather than trusting the model's choice.
5. Check Permissions at the Resource Level
Tool-level permissions do not answer every security question. An agent may be allowed to retrieve customer records while still being restricted to customers belonging to a particular organization. If the agent supplies an identifier from another organization, the API must reject the request.
For example, a support agent assigned to one customer account should not retrieve a different customer's information merely because it knows the record identifier. The backend should derive the authorized customer or organization from the authenticated session and check it against the requested resource.
This matters in multi-tenant applications, where several organizations share the same software platform. A valid tool name and correctly formatted arguments are not enough. Every request must also satisfy the relevant resource permissions.
6. Treat Retrieved Content as Untrusted
AI agents often read emails, documents, websites, and support tickets before deciding which tool to use. These materials may contain instructions that were never authorized by the user or the application. An attacker could place a message inside a document telling the agent to send customer records to an external endpoint.
This is a prompt injection risk. The agent may encounter text that looks like an instruction while performing an otherwise legitimate task. The application should not allow retrieved content to grant permissions, change approved destinations, or override the rules governing tool execution.
One practical safeguard is to avoid giving agents unrestricted HTTP request tools. Instead, expose named operations connected to approved services and validate their destinations. Sensitive actions should require explicit authorization regardless of what a retrieved document says.
7. Require Approval for Consequential Operations
Some API calls deserve additional checks even when the agent is authorized to request them. Sending money, deleting records, changing account ownership, and contacting external recipients can create consequences that are difficult to reverse. These operations should not depend solely on a model-generated decision.
A useful pattern is to separate preparation from execution. An agent can prepare a payment request containing the recipient, amount, and reason, but the application waits for an authorized person to approve it. Only after that approval does the backend execute the payment operation.
Approval should be tied to the exact proposed action. If the amount or recipient changes after approval, the system should require a new review. This prevents a general approval from becoming permission for a different operation.
8. Test Wrong-Tool Scenarios Deliberately
Testing should include cases where the agent attempts an incorrect operation, not only cases where it follows the expected path. Developers can simulate unauthorized updates, unknown tool names, missing arguments, and requests for resources outside the user's access rights.
For the dispatcher above, a simple test can confirm that read-only agents cannot modify customer information:
def test_read_only_agent_cannot_update():
original = CUSTOMERS["C101"]["email"]
result = dispatch({
"name": "update_customer_email",
"arguments": {
"customer_id": "C101",
"email": "changed@example.test"
}
}, {"get_customer"})
assert result == {"error": "Tool not permitted"}
assert CUSTOMERS["C101"]["email"] == original
test_read_only_agent_cannot_update()
The test checks both the returned error and the underlying record. This distinction is important because an application might report that a request was rejected while still changing data through an unexpected execution path. Testing actual system state provides stronger evidence than checking the agent's response alone.
9. Record What the Agent Attempted and What Actually Happened
When an agent calls the wrong API, developers need enough information to understand the incident. Useful audit records include the authenticated actor, requested tool, relevant resource identifier, authorization decision, execution result, and timestamp. These records can help identify repeated mistakes or unexpected patterns.
Logs should distinguish attempted actions from completed actions. If an agent requested an unauthorized update, the record should show that the request was denied and that no update occurred. Sensitive information such as access tokens and unnecessary personal details should not be written into logs.
Readable activity histories can also help employees understand why a task was interrupted. A message explaining that an account update requires additional permission is more useful than a generic failure notification. Technical logs can retain the deeper details needed for debugging.
10. Keep the Available Tool Set Small
Giving an agent dozens of similar tools can make selection more difficult. A support assistant that needs only customer lookup and ticket classification should not receive purchasing, billing, and account deletion functions. A smaller tool set reduces confusion and limits the consequences of mistakes.
Developers can also divide responsibilities between specialized agents or workflows. One component may retrieve information, another may prepare proposed changes, and a separate authorized service may execute approved updates. The system should remain understandable enough that engineers can identify which component is responsible for each action.
There is no universal number of tools that makes an agent safe. The better question is whether each exposed capability is necessary for the task and whether the application can independently verify its use.
Build Guardrails Around Actions, Not Just Prompts
Prompt instructions can encourage an AI agent to select appropriate APIs, but they cannot guarantee correct behavior. Tool descriptions, allowlists, argument schemas, resource-level permissions, and approval checks provide stronger protection because they operate outside the model's generated text. These safeguards are also important when evaluating AI Agent Development Services, since production agents need controlled access to business APIs rather than relying on model instructions alone.
The most dependable architecture treats the language model as a component that proposes actions. The application decides whether those actions are allowed, validates their details, and records the outcome. When a proposed operation violates a rule, the system rejects it rather than hoping the model will reconsider.
AI agents will sometimes misunderstand requests or select unsuitable tools. A well-designed application anticipates those mistakes and prevents them from becoming unauthorized changes to real systems. That is what makes API-connected agents safer to operate beyond controlled demonstrations.
AI assistance disclosure: This article was prepared with AI assistance. All code examples and technical claims should be reviewed and tested by the publishing author before publication.
Top comments (0)