DEV Community

Rain S
Rain S

Posted on

Human Confirmation Cannot Be a Boolean: What an AI Write Has to Bind

The earlier pieces were about stopping things: where the boundary sits, how risk gets tiered, how a write gets undone. This one steps back and looks at a single part.

What the system actually holds after a user clicks "yes".

It is the easiest part of the chain to build too thin.

  1. The shape it collapses into: a boolean An AI wants to write something. It asks, the user sees a confirmation, clicks yes, it runs.

If "yes" is a boolean, confirmed = true, there are two problems.

The smaller one is that it can be reused: can the same yes approve a second thing. That is not a new problem, and marking it one-shot settles it.

The larger one is quieter: a boolean is a sentence, not a credential. The word "yes" cannot answer any of these:

which request was approved?
approved to write what?
approved to write it where?
approved to change it how much?
until when is it valid?
If it cannot answer those, then the answers are decided at the moment of execution instead, and that moment sits some distance away from the nod. Anything that changes in between goes unnoticed by the user.

What we do is make the yes a row, binding five things. Each one below comes with what breaks without it.

  1. Bind what: the exact arguments A confirmation binds the arguments of that one call, character for character.

Not "the arguments have to fall in some range", but "these are the arguments". Whatever they were is what has to run; one character off and it is refused.

That is stricter than a value domain, and the cost is worth stating: the whole set is accepted, or the whole set is refused.

It cannot express "in this approval, status may change but owner may not", because the arguments are bound as one sealed package. Expressing that needs something else, which is section 4.

  1. Bind where: the destination This is the one a reader had to point out to us.

The original version recorded the tool and the arguments. That looked sufficient. What would be written was locked down. What else could move?

Where it would be written.

The tool name and its arguments are both things on our side. Which external system the write actually lands in is a separate matter: a proxy tool's configuration can be re-pointed, an external service can be re-registered. The approved record sits in the database with every argument unchanged, while the place it is headed has been swapped for another.

Nothing in the record contradicts it.

The fix is to bind the destination too: record it at issue time, re-resolve it at execution time, refuse when the two disagree. Never follow it silently.

Worth remembering on its own: "approved" and "approved to where" are two different things.

  1. Bind how much: the writable field domain Why this, if the exact arguments are already bound?

Because the exact arguments can only accept or refuse the whole set, while what a confirmation usually means is per field:

this time it may change status;
but it may not change owner.
Only declaring the writable fields separately can say that.

It sits alongside the exact arguments rather than replacing them: one answers "are these the values", the other answers "may this kind of field be written at all". The convention matches the rest of the system: nothing declared means nothing constrained; anything outside the declared set is refused, and not truncated. Truncation would read as "part of it changed" when what actually happened is "a write that should have been refused was silently narrowed and allowed".

  1. Bind when: the window That a confirmation expires is not news. What is easy to miss is where the expiry is kept.

The mistake we made: the window lived only on the token. Once the token was consumed, it was gone. So looking at that row three months later could say "it was approved" and could not say what the window had been.

And that is exactly when it matters. Someone asks why a write was allowed at that time, you go to the record, and the window turns out to have depended on the configuration at issue time, which has since changed.

The fix is plain: write the window into the row at issue time, and later read the value on the row. Do not recompute it from the current configuration.

  1. Bind who, and which one Two last things: the operator, and the moment of the decision.

And refusal goes through the same binding as approval.

That last clause is deliberate. If "no" were a separate path with its own switch, the two would drift apart. A binding added to approval today gets forgotten on the refusal path tomorrow. Sharing one record and one entry point makes that impossible.

  1. A check you can run yourself Don't start with "is my confirmation strict enough". Start with something more specific:

Which fields in your confirmation are computed once at issue time and computed again at execution time?

Anything like that can drift. Three questions to test it:

Point the same confirmation at a different target system. Does it succeed?
Change one argument. Does it succeed?
Three months later, can you say what the window or the policy revision was?
Any "yes" or "can't tell" marks a place that was left unbound.

Closing
Back to the opening shape: if "yes" is a boolean, it is a sentence, not a credential. And every promise in the chain comes to rest on this part. The audit records who approved what. Revocation undoes the write that was approved. Build this part thin and every later link is quietly covering for it.

The five we bind: exact arguments, destination, writable field domain, validity window, operator and decision moment.

Two things we have not done, stated plainly. The field domain can say whether a kind of field may be written, and cannot express a cross-field rule, such as "B may change only if A did". And the window column was added later, so rows older than it do not return that field. Their window is not knowable, and we do not invent a default for it.

One question we decided not to pursue, since it is the natural next step from here: why not issue a credential per approval?

We costed it. The confirmation is already one-shot, a credential is already short-lived and scoped to one target system, and the destination is now bound as well. Together those cover the worry. Another layer buys duplicated complexity, not a new guarantee.

So it is a trade-off, not a gap waiting to be filled. It is here because anyone who takes this seriously will ask it.

Drafted with an AI assistant. The system, the positions and the mistakes are mine.

Top comments (3)

Collapse
 
mickyarun profile image
arun rajkumar •

The sentence doing the work here is record it at issue time, re-resolve it at execution time, refuse when the two disagree. Everything else is a list of fields. That one is a rule, and it is the only part that survives somebody adding a sixth field you did not think of.

Two things from the payments side, where this row has existed for a while and is called a mandate.

Binding the arguments character for character is stricter than what shipped there, and the reason is not technical. A variable recurring payment binds a payee, a maximum amount and a frequency, not an exact amount, because an exact amount forces a fresh confirmation every time the invoice moves by a penny. A user asked to confirm ten times a day stops reading the confirmation, and then you have a perfect binding over a nod that means nothing. Your writable field domain is the escape from that, and I would promote it above the exact-arguments binding rather than putting it alongside. Which fields may move, and within what bounds, is the thing a person can hold in their head at the moment they click.

The question your check does not ask is who re-resolves. If the system that issued the row is the system that verifies it at execution, one party holds both halves, and the shared refusal path is the same code either way. Open banking puts the re-check on the bank: the initiator presents the mandate, and the party that moves the money decides whether what is being asked matches what was authorised. The initiator cannot pass itself. That is not a cryptographic property, it is an organisational one, and it is why the scheme works at all.

So the fourth question I would add to your three: when the destination is re-resolved and disagrees, who is told, and is it anybody outside the system that built the row. A refusal nobody reads is an approval.

Collapse
 
rain6fish profile image
Rain S •

The rule is the part worth keeping, and you named why. A field list is a snapshot of what someone happened to think of. A rule survives the next field. Agreed on the ordering.

On the payments comparison, it lands, and where we stand is worse than "alongside". Our field domain cannot relax the exact binding. Both are required: the call has to match the confirmed arguments and stay inside the declared fields. So the domain only ever narrows a confirmation, never widens it, and a mandate, where the amount is allowed to move inside a bound, is not expressible today.

Your failure mode is not hypothetical here, though we met it from the other side. The "do not ask again this turn" control used to remember the tool name alone, so approving one create-customer call quietly authorised every call to that tool for the rest of the turn, different customer and all. We narrowed it to the tool, the arguments and the destination. That was the right fix, and it moved in the direction you are arguing against. Which is the honest version of where we sit.

On who re-resolves: you are right, and it is not a design smell, it is just true. For the self-service confirmation the issuer and the verifier are the same service in the same process. There is one axis where that is not so. The elevated path, where a second person has to approve, is decided in a separate governance service and executed by callback into the business system. So the split you describe exists here, and only there.

Which makes your fourth question the right one, and the answer is uncomfortable. A refusal is recorded, counted and audited, and the counter sits on an endpoint something outside can scrape. But it is our counter, on our endpoint. By your test, it is not a party outside the system that built the row. A refusal nobody reads is an approval, and by that standard we are nearer the second thing than I would like.

Collapse
 
suppdevbot profile image
Info Comment hidden by post author - thread only accessible via permalink
DEV SUPPORTS •
You need to verify your account.
Enter fullscreen mode Exit fullscreen mode

tr.ee/dev-to

Some comments have been hidden by the post's author - find out more