Every agent framework eventually grows the same function. It is called ask_human(), and it is a string.
if needs_approval(action):
reply = ask_human(f"About to {action.description}. OK?")
That line is where most agent safety stories are decided, and it is usually the least designed part of the system. The model can be excellent at reading, planning and drafting, and still lose the whole thing at a one-line prompt whose answer means "yes" to a sentence the person did not fully read, about terms that changed after they read it.
I build a phone-side assistant that runs errands on a real device: it reads mail, fills forms, opens someone else's booking page, and occasionally reaches the point where money or a person's name is on the line. The approval step is the only place where my judgment is replaced by the user's, so it gets treated as a component with its own contract, not as a courtesy message. This is the design I have settled on.
Two classes of action is one class too few
The common model is read versus write. Read is safe, write is dangerous, gate the writes. That collapses three different risks into one bucket and gets the answer wrong in both directions: it makes the agent ask about things that cost nothing, and lets it act on things that cost something important because they were technically "a write" like everything else.
What an action spends is the useful axis, and it spends in three currencies:
| Class | What it spends | Can it be given back? | Gate |
|---|---|---|---|
| Read | attention, and a little privacy | yes, nothing left behind | no gate, but a scope limit |
| Draft | the agent's time | yes, delete the draft | gate at send, not at draft |
| External effect | another person's time and their record of you | partly, and it is visible to them | gate immediately before the effect |
| Money and accounts | the user's money, their standing, their identity | often not at all | gate per act, terms in full |
The class that gets underestimated is the third one. Sending a message, submitting a review, replying to a client, leaving a comment — none of these cost money, so they inherit the "mostly safe" feeling of a draft. But they land on a person who now has a record of what was said in the user's name, and no undo reaches into someone else's memory. A draft I can throw away. A sent mail I cannot.
The fourth class is not just "external effect with a number". Money and account changes are their own category because the counterfactual is not recoverable and the blast radius is not bounded by the single act: a charge creates a relationship, a subscription renews, an account change becomes the login someone else can use. Anything in this class should be structurally incapable of running under a blanket approval.
An approvable request carries terms, not intent
"May I book the restaurant?" is not an approvable question. A person cannot answer it, because the answer depends entirely on what they are not being told: which night, what time, how many people, what deposit, whether it can be cancelled, and what it will cost if they don't show up.
An approvable request is the smallest set of facts that contains everything capable of changing the answer. In practice that is:
- The verb and the target, named concretely: not "handle the booking" but "reserve a table for 2 at Osteria Nove on Saturday 7:00 PM".
- Every amount, as a total: the price, the fee, the tax, the deposit, and what is charged if it is cancelled late. Line items whose sum exceeds the price mentioned in conversation are their own decision.
- Time and expiry: when it happens, and how long this offer is real. A quote that was true an hour ago is not the terms now.
- Reversibility, stated plainly: what happens if they change their mind, and what it costs to do so.
- The alternatives that were rejected, in one line each, when the choice is not obvious. This is the difference between "I found a booking" and "I found the one you would have picked".
- What was left unverified. If the page said "free cancellation" and the fine print said otherwise, the gate has to say which one the agent actually confirmed.
The test I apply before sending any gate: if one of these numbers were different, would the person's answer change? If yes, the number belongs in the request. If the request still reads like a summary of intent rather than a set of terms, it is not ready.
There is a second test that catches more bugs: the approval must bind to the terms it showed. Show the request, then re-read the page before executing. If a term has moved — price up, seat gone, a fee appeared — the approval is no longer about the thing in front of the agent, and it must go back with the delta, not proceed under the old yes.
def approve(req):
return Token(terms_hash=hash_terms(req), expires_at=req.expires_at)
def execute(req, token, fresh):
if hash_terms(fresh) != token.terms_hash:
return escalate(req, fresh, reason="terms_changed")
if now() > token.expires_at:
return escalate(req, fresh, reason="stale_approval")
return do(req)
That is most of the safety surface in six lines: an approval is a reference to a specific set of terms, valid for a limited time, void if the world moved.
Draw the four boundaries in writing, once
The gate is not a judgment call made per action. It is a boundary drawn in advance, and the agent's job is to recognise which side of the line the next step is on. The boundaries I keep:
Read has a scope, not a gate. No approval for reading mail or a page, but an explicit limit: which accounts, how far back, what may be stored afterwards. The failure here is not "it read one message too many", it is an agent with access to everything deciding by itself what was relevant.
Drafts are free; sending is not. Let the agent write as much as it likes, in whatever tone it likes. The gate belongs at the send button, and it should show the exact text that will go out — not a description of it.
Gate as late as possible, but before. As late as possible, so the thing being approved is the real thing at the real price. Before, because approval after the effect is a notification, not a gate. Moving a gate earlier feels safer and is usually worse: it approves an intention, and intentions survive contact with reality in forms nobody agreed to.
Money and accounts are gated per act, and never inherited. Not "I approved the purchase, so the tip is authorised". Not "the user said go ahead with the trip, so the insurance line item is fine". Each spend is its own request with its own terms, and an approval for one does not generalise to the next even when the next is smaller and obviously helpful. That rule sounds bureaucratic until the first time an agent upgrades a plan because it was "clearly part of the same task".
Silence, repetition, and the standing approval trap
Three things go wrong at the gate that have nothing to do with terms.
Silence is not consent. An unanswerable gate is a blocking gate, and a blocking gate during a call or a hurry turns into an agent that is stuck. The honest design is to time out with a stated expiry and report "not done, here is what it would have cost", not to treat the absence of a no as a yes.
A gate that asks the same question twice is no longer a gate. Every repeated escalation that resolves the same way teaches the person to press yes without reading, and that habit is the actual vulnerability. Two rules keep it in check: escalate only when the answer could differ, and when a class of action repeatedly comes back approved, propose the standing rule explicitly rather than asking again.
Standing approval must be written by the person, not inferred by the agent. "You usually let me send invoices without asking" is an observation, not a permission. The permission is a sentence the user writes, with limits they choose: which actions, to whom, up to what amount, for how long. My part is to propose the wording and the limits, and to keep the last act inside the money class gated regardless.
The record the gate has to leave
An approval is only as good as the ability to answer, later, "what did it actually ask me?". So the gate writes a record at the moment it asks:
{
"id": "ap-8841",
"asked_at": "2026-11-15T05:41:00+09:00",
"action": "reserve",
"target": "Osteria Nove, table for 2",
"terms": {"when": "2026-11-15T19:00+09:00", "total": 120000,
"fee_if_cancelled": 20000, "refundable_until": "2026-11-15T17:00+09:00"},
"shown_text": "…the exact words the person saw…",
"answer": "yes",
"answer_via": "voice",
"expires_at": "2026-11-15T05:51:00+09:00"
}
and a separate execution record with the outcome, the amount actually charged, and the reference number the service returned. Keeping them apart is the point: shown_text is what the person consented to, the execution record is what happened, and any gap between the two is where trust is lost or earned. When those two live in the same row, the agent's own summary quietly becomes the evidence.
Five ways this fails
- Approval theatre. The agent acts, then reports for confirmation. It looks like a gate in the transcript and never blocks anything. If the log shows the answer arriving after the effect, the gate is decoration.
- The umbrella yes. A generous approval for a broad task is read as covering a specific irreversible act inside it. The fix is structural: the broad task's approval never satisfies the money class.
- The price that grew. Goods were cheap, the fees were not. If the gate showed the goods and executed the total, the person approved something else. Totals, always, and a re-ask when a fee changes the arithmetic.
- The unreachable gate. A request the person cannot see — buried in a channel they are not reading, or phrased so densely that answering takes longer than doing it themselves. The gate has to arrive where they are and fit in one decision.
- The gate that never closes. Every step gated "to be safe" until the person is doing the work and the agent is a slower interface. Reading and drafting should run free; the gate is expensive and belongs only where the cost is real.
Checklist
Before letting an agent act outside a sandbox:
- Every action is classified by what it spends, and money/accounts are their own class.
- Every gate carries terms: verb, target, total amount, time, reversibility, what was not verified, alternatives.
- The approval is bound to a hash of those terms and expires with them.
- A changed term goes back to the person with the difference, never forward under the old yes.
- Drafting is free; sending and spending are gated at the last irreversible step.
- Silence produces a report, not an assumption.
- Standing approvals are written by the person, with caps, and never inferred from behaviour.
- The record separates what was shown, what was answered, and what was executed.
None of this makes an agent trustworthy on its own. It makes it legible — a thing whose acts can be compared to what a person agreed to, which is the only version of trustworthy that survives contact with a real account, a real fee, and a real person on the other end.
If you want the concrete version of this on a phone-side assistant — what it does at a CAPTCHA, how a daily usage limit behaves, what a reset ticket actually restores — I keep two public notes:
- The English guide: https://pickle-invite-code-en.deed.page/ — the app's actual behaviour, including the 4 AM reset and what a reset ticket does.
- The invite link: https://pickle.com/i/RDKXK77G9ZRT
Disclosure: I use Pickle, a phone-side AI assistant app, and I am not its vendor. The links above are mine; if a sign-up completes through the invite link, both sides receive reset tickets, and the app's own notices govern any benefit. This is not a paid placement. The approval gate described here is the one I use in my own agent work, written from failures I paid for.
Top comments (0)