Your AI agent needs more than one safety check.
AI agents are no longer just chatbots that generate text. They can retrieve documents, query databases, call APIs, execute business tasks, and make decisions with minimal human intervention.
That creates a new challenge: How do you ensure an AI agent does what it is supposed to do without exposing sensitive information or performing unauthorized actions?
The answer is to implement guardrails at different stages of the agent's workflow.
Here are the five critical places to put them.
1. Input Guardrails: Before the Agent Processes a Request
Every interaction starts with an input. If that input contains a malicious instruction, sensitive information, or an attempt to manipulate the agent, you want to identify the risk before it influences the agent's behavior.
Input guardrails help screen requests before they enter the main reasoning process.
What to implement:
- Detect prompt injection and jailbreak attempts.
- Identify and redact sensitive information, such as passwords and personal identifiers.
- Block malicious or disallowed requests.
- Validate input formats and enforce request size limits.
Tools to consider: Llama Guard, Microsoft Presidio.
Example: A customer asks an AI agent to process a payment and includes instructions to ignore its existing security rules. Input guardrails can help identify the suspicious request before processing continues.
However, detecting a suspicious prompt is not enough. The agent's tools must still enforce their own permissions.
2. Retrieval Guardrails: Before Retrieved Data Reaches the Agent
Many AI agents use Retrieval-Augmented Generation (RAG) to access company documents, knowledge bases, and databases.
But what happens when an agent retrieves a document the current user is not authorized to see?
Or when a document contains malicious instructions designed to manipulate the agent?
Retrieval guardrails help ensure the agent accesses appropriate information and treats retrieved content as data rather than automatically trusting it as instructions.
What to implement:
- Enforce document-level and user-level access permissions.
- Filter irrelevant or untrusted sources.
- Detect suspicious instructions embedded in retrieved documents.
- Apply access controls before passing retrieved content to the model.
- Track the sources used to generate an answer.
Tools to consider: NVIDIA NeMo Guardrails, access control lists (ACLs), and permission-aware retrieval filters.
Example: An employee asks an internal AI assistant for information about company finances. The agent should retrieve only the financial documents that employee is authorized to access, not every document available to the system.
Important: Do not rely on the model to decide whether a user deserves access to a document. Enforce authorization in the retrieval system itself.
3. Tool-Call Guardrails: Before the Agent Executes an Action
This is one of the most critical places to implement guardrails.
An AI agent may have access to tools that send emails, update customer records, initiate payments, delete files, or call external APIs.
If the agent makes a wrong decision, the consequences can extend beyond an incorrect answer.
Tool-call guardrails control which actions the agent can execute, under what conditions, and with what permissions.
What to implement:
- Allowlist approved tools and restrict access to unnecessary capabilities.
- Validate API arguments and enforce strict input schemas.
- Apply role-based access control and least-privilege permissions.
- Set transaction limits, spending limits, and rate limits.
- Require authorization before sensitive or irreversible operations.
- Log tool calls and their outcomes.
Tools to consider: Open Policy Agent (OPA), Pydantic, and Docker-based sandboxing where appropriate.
Example: An AI finance agent identifies a supplier invoice and prepares a payment. The agent should not be able to transfer any amount to any account simply because its reasoning process concludes that the payment is necessary.
The payment service should independently validate the recipient, amount, account permissions, and approval requirements before executing the transaction.
The principle: Let the AI propose an action. Let deterministic application logic and authorization policies decide whether that action is permitted.
4. Output Guardrails: Before the Response Reaches the User
Even when an agent retrieves the right information and uses its tools correctly, its final response can still be inaccurate, misleading, or unsafe.
Output guardrails validate what the agent produces before delivering it to the user or another system.
What to implement:
- Validate structured outputs against predefined schemas.
- Check responses for sensitive information and accidental data exposure.
- Verify factual claims against trusted sources where possible.
- Detect disallowed content and enforce business-specific response rules.
- Ensure generated outputs meet the requirements of downstream systems.
Tools to consider: Guardrails AI, Pydantic, and custom validation logic.
Example: An AI customer support agent generates a refund confirmation. Before the message is sent, the application should verify that the refund actually succeeded.
The agent must not tell a customer that their money has been refunded simply because it generated a convincing confirmation message.
Remember: A well-written response is not necessarily a correct response. Validate important claims against authoritative system records.
5. Human Approval Gates: Before High-Risk Actions Are Finalized
Not every decision should be left entirely to an AI agent.
Some actions carry financial, legal, operational, or reputational consequences that justify human review.
Human approval gates introduce a checkpoint where an agent can prepare an action but cannot finalize it until an authorized person approves it.
What to implement:
- Require approval for transactions above defined thresholds.
- Review sensitive data deletion and permission changes.
- Approve high-impact customer communications.
- Escalate ambiguous or unusual cases to a human.
- Record who approved an action, when it was approved, and what was authorized.
Tools to consider: LangGraph's interrupt() mechanism for pausing workflows, alongside application-level authorization and approval systems.
Example: An AI agent identifies a customer who may qualify for a KES 100,000 refund. It gathers the relevant records, checks the refund policy, and prepares the transaction for review.
An authorized employee reviews the evidence and approves or rejects the refund. Only then can the application proceed with the permitted action.
Human approval should not be the only security control. The application must still validate the approved action, enforce permissions, and prevent unauthorized changes between approval and execution.
The Bigger Picture: Guardrails Must Work Together
These five checkpoints address different risks, and none can replace the others.
| Checkpoint | Main Question | Primary Risk |
|---|---|---|
| Input | Is this request safe to process? | Prompt injection and malicious inputs |
| Retrieval | Is the agent allowed to access this information? | Unauthorized data exposure |
| Tool calls | Is this action permitted? | Unauthorized or destructive operations |
| Output | Is this response safe and valid to deliver? | False claims and sensitive data leaks |
| Human approval | Does this action require authorized human review? | High-impact mistakes |
A secure agent also needs controls that operate across its entire lifecycle, including authentication, secrets management, audit logging, monitoring, rate limiting, and incident response.
For example, a malicious instruction could bypass an input filter. That is precisely why tool permissions must be enforced independently. If an agent generates an incorrect response, output validation can catch some errors. If an action is particularly risky, human approval can provide another layer of protection.
The goal is not to build one perfect filter. It is to prevent a single failure from becoming a serious security incident.
Final Takeaway
Guardrails are not a single filter. They are checkpoints throughout an AI agent's workflow.
As developers, we should not ask only whether an AI agent can complete a task. We should also ask:
- What happens when it receives a malicious instruction?
- What prevents it from accessing unauthorized data?
- What stops it from executing an incorrect or unauthorized action?
- How do we verify that its response reflects what actually happened?
- Which actions require human approval?
The more access and autonomy an agent has, the more important these controls become.
Build the agent to be useful. Build the guardrails to keep its actions within authorized boundaries.
Top comments (1)
Your line about preventing changes between approval and execution is the one most setups miss. What's worked for me: the approval is bound to the exact request, so the human approves a hash of the action (tool, arguments, amount, recipient), and the executor refuses anything that doesn't match it, and refuses it if it's already been used once. Otherwise the agent can get a yes for a KES 10,000 refund and send KES 100,000 a second later.