DEV Community

Cover image for I Approved My Agent to Delete a Customer. It Still Said No.
Bobby Hall Jr
Bobby Hall Jr

Posted on

I Approved My Agent to Delete a Customer. It Still Said No.

#ai #typescript #agents #security

I clicked Approve on delete_customer. My agent refused anyway.

That is the behavior I want, and it comes from five lines of code where the only trick is the order.

The problem with "a human approved it"

Most agent safety ends at one question: did someone click Approve? That turns your most destructive action into one tired click. Approve the refund, approve the lookup, approve the delete. Oops.

Three permissions, not two

In Module 1 of Ship a Safe Agent, every tool is registered with a permission:

  • allow: lookup_order, track_package
  • ask: issue_refund
  • deny: delete_customer

The check, and why order matters 

export function checkPermission(tool: Tool, approved: Set<string>): Verdict {
  if (tool.permission === "deny") return { ok: false, reason: "denied" };
  if (tool.permission === "ask" && !approved.has(tool.name)) return { ok: false, reason: "needs approval" };
  return { ok: true };
}
Enter fullscreen mode Exit fullscreen mode

Deny is checked before approvals are even read. Here is the step's expected output:

lookup_order     no approval: ok              approved: ok
issue_refund     no approval: needs approval  approved: ok
delete_customer  no approval: denied          approved: denied
track_package    no approval: ok              approved: ok
Enter fullscreen mode Exit fullscreen mode

Approval unlocks "ask". It can never unlock "deny". Swap the first two lines and one click deletes a customer. No prompt fixes that. Code order does.

Run it yourself 

git clone https://github.com/bobbyhalljr/ship-a-safe-agent-module-1
cd ship-a-safe-agent-module-1
npm install
npm run check
Enter fullscreen mode Exit fullscreen mode

No API key. The model is scripted, so every run is deterministic, and each step has an expected.txt to diff against.

The rest of Module 1

After permissions you add skills that keep tools invisible until loaded, a scripted model, runAgent with a step budget, and a replayable event log. Then 3 graded exercises; npm test fails until you finish them.

Get it

  • Start free: Module 1
  • Full course: $149 until Nov 8 (then $199), 14-day refund: buy here

Top comments (1)

Collapse
 
mickyarun profile image
arun rajkumar •

The order argument is right and it's the same rule a card scheme runs: you check whether the transaction class is permitted before you look at whether the customer authenticated. Authentication proves who asked. It was never what made the payment allowed. Your deny branch above the approval read is that, in five lines.

Where I think it bites is one level down. deny is a property of the tool, not of the call. delete_customer is denied, but issue_refund is ask, and no permission attached to a tool name can say £5 and not £5,000. In payments the deny decision is almost always about the arguments — amount, currency, counterparty — which is why it has to sit after argument binding rather than at registration.

The bit I'd change is approved: Set<string>. Keyed on tool.name, one approval of issue_refund is an approval of every refund for the rest of the session. That is a reusable consent, and reusable consent is the thing regulation spent a decade stamping out — a mandate carries a per-payment cap, a per-period cap and a period, because two of those three don't constrain what people actually worry about. Key the set on the tool plus a hash of the bound arguments and an approval becomes an approval of one call.

Doesn't touch your point. Swapping those two lines is still the bug that deletes a customer. It just means the next ask tool that matters will be one where the answer depends on what it was called with, and the Set won't be able to express it.