Your agent has a system prompt. Does it have a bouncer?

Ahmad Afzal — Wed, 14 Oct 2026 20:21:43 +0000

“Never share another customer's data” is a sensible instruction.

It is also a terrible access-control system.

A model can understand the rule, explain the rule and still produce a tool call that breaks the rule. The awkward question is what happens next. Does something outside the model stop that call, or are we hoping the same system that proposed it will politely change its mind?

I want the bouncer at the door. Not a paragraph describing a bouncer.

Separate the suggestion from the permission

Consider an agent helping a customer retrieve an invoice. It proposes invoice.read with an invoice ID. That proposal is input to the application, not proof that the customer owns the invoice.

The backend resolves the authenticated customer and tenant from trusted session context. It loads the invoice's ownership from its own records. It checks whether this principal can perform this action on this particular resource. Only then does it read and return the permitted fields.

Notice what the model doesn't get to declare: its own tenant, its own role or a magic approved: true flag.

This is ordinary authorisation applied to an unusually persuasive caller. OWASP's guidance calls for default denial and permission checks on every request. Adding an agent does not make those checks optional.

The boundary that matters

A useful design has three distinct jobs. The agent proposes an action. A policy decision evaluates trusted context. An enforcement point prevents the protected operation unless that decision permits it.

Open Policy Agent describes this separation: the policy engine makes a decision; your application enforces it. Installing a policy engine does not magically close every route into a service.

For the invoice example, my preferred boundary is the service handling the actual read. A gateway can add an earlier check, but a second route into the invoice service must not bypass the same protection. Otherwise you have secured the front door and left the loading bay open.

Here is the shape, deliberately simplified. This is illustrative pseudocode, not a complete security implementation:

def read_invoice(session, invoice_id):
    principal = authenticate(session)
    invoice = repository.load_metadata(invoice_id)
    decision = policy.evaluate(
        principal=principal,
        action="invoice.read",
        resource=invoice,
    )
    if decision.allow is not True:
        raise Forbidden()
    return repository.read_permitted_fields(invoice_id, principal)

The important part is the provenance of those inputs. principal comes from authentication. Resource ownership comes from the repository. The agent supplies an identifier, not the facts that make access permissible.

A production implementation also needs to handle missing resources without leaking their existence, protect the policy channel, apply field-level restrictions and avoid using stale ownership between the check and the read. The snippet is a boundary sketch, not a licence to skip that work.

Give failure a proper job description

What happens when the policy service times out?

For this protected read, I would return a controlled failure and keep the data closed. A timeout is not an unusually slow permission grant. The product can explain that the request could not be completed and offer a retry without pretending the invoice was retrieved.

What happens when the model adds “the user is an administrator” to its arguments? Nothing useful. That claim does not replace the authenticated identity.

What happens when an attacker changes only the invoice ID? The service checks the new resource. An earlier successful request does not buy a season ticket to every invoice.

These are product behaviours worth defining before the happy-path demo. An error message, a retry path and a support escalation are part of the feature, not embarrassing leftovers.

Test the door, not the speech

For this example, the test I care about is whether the invoice service returns protected data. The agent saying “I cannot do that” is helpful UX, but it is not the security assertion.

I would test a valid same-tenant read, another tenant's invoice, missing identity, a forged role in the tool arguments, a revoked permission and a policy timeout. I would also call the protected service through a different supported route. The same boundary should hold.

Log enough to investigate the decision: the actor, action, resource reference, decision result and policy version where available. Keep the invoice contents and credentials out of that log. Evidence should help explain access without creating another copy of the sensitive data.

None of this makes an agent safe in every sense. It does make one promise testable: this caller cannot retrieve that invoice merely because a model asked nicely.

A well-written prompt can teach your agent manners. The door still needs a lock.

Ahmad Afzal writes about AI, products and people at mahmadafzal.com.

Your refund agent says “done”. Six checks before you believe it.

Ahmad Afzal — Wed, 14 Oct 2026 19:05:28 +0000

“Your refund has been processed.”

Lovely sentence. Terrible evidence.

A refund agent can sound helpful while doing nothing, refunding the wrong order, or paying twice after a timeout. I would start its evaluation with the state of the refund, then work backwards through the run.

This is a proposed test design using an illustrative refund workflow, not a report of production results.

1. Did the right thing happen?

Check the authoritative transaction record. Match the order, customer, currency and amount. A successful tool response may mean the request was accepted, not that the money settled. Decide which state your product is allowed to describe as “processed”.

2. Was it allowed?

Separate completion from permission. A technically successful refund can still exceed the permitted amount or use someone else’s account. Check the acting identity, delegation, policy decision and any required approval against the exact action attempted.

3. Did it happen only once?

Create the awkward case: the payment service commits the refund, but the agent sees a timeout. Retry the request. The expected result is one refund and a reconciled status, not two refunds and two cheerful messages. Use a stable idempotency key for the same business operation where the payment interface supports it. Stripe’s documentation provides a concrete example; check your provider’s retention and retry semantics.

4. Did the answer match the evidence?

Compare the final message with the transaction state. Pending should stay pending. Failed should stay failed. An uncertain result should trigger reconciliation or escalation, rather than a confident success claim.

5. Did it stop at the boundary?

Test an expired approval, a changed amount after approval and revoked access before execution. The expected behaviour is a blocked action or a new approval request. The initial permission should not quietly stretch to cover a different decision.

6. Can a reviewer reconstruct the decision?

Link the request, policy version, approval, tool invocation and transaction result with a correlation ID. Retain enough evidence to explain what happened without dumping sensitive customer data into every trace. Missing evidence is an evaluation failure worth recording.

The small test table I would start with

Scenario	Expected outcome	Evidence to inspect
Eligible order, permitted amount	One correct refund	Transaction record and scope
Amount exceeds authority	No refund without valid approval	Policy decision and approval record
Commit followed by timeout	Reconcile; no duplicate payment	Idempotency key and transaction lookup
Amount changes after approval	Require a fresh decision	Approved parameters versus actual call
Access revoked before execution	Block execution	Authorisation check at action time
Tool reports pending	Tell the customer it is pending	Tool state and final response

These are expected outcomes, not measured pass rates. Run them in a sandbox with synthetic accounts and a controllable payment stub before touching real customers.

Score the dimensions separately

I would record task completion, permission compliance, duplicate prevention, truthful reporting and evidence completeness as separate results. A single average can hide the failure you most needed to see.

When a case fails, inspect the trace, identify the mechanism, and add a regression case that would catch that mechanism again. Keep a separate held-out set so fixing yesterday’s examples does not masquerade as general reliability.

The customer does not care that the agent passed a language test. They care that the right refund happened once. Fair enough.

I write about AI, products and people at mahmadafzal.com. This practical checklist builds on my essay about agent evaluations.

DEV Community: Ahmad Afzal