Tool Calls Need Completion Ownership: Why Your Agent's 200 OK Lies

#ai #machinelearning #programming #architecture

An agent is asked to send a payment confirmation email. It calls the send_email tool. The tool returns {"status": "queued", "message_id": "msg_abc123"}. The agent writes back to the user: Done. Confirmation email sent. Two hours later a support ticket arrives. The customer never got the email. The provider's queue had dropped the message during a regional failover, and no one on the path between the agent and the recipient's inbox ever held ownership of the claim this email reached the customer. Every layer received a token, forwarded it, and declared its own part finished.

The instinct in the incident review is to blame the model. It reported something false with total confidence, so the model must have hallucinated. It did not. The model read a success signal and faithfully relayed it. The signal was a lie before the model ever saw it, and it was a lie for a reason that has nothing to do with language models.

A tool call's 200 means the request was accepted, not that the effect happened. Put completion ownership in the wrapper: verify the side effect before the word success ever reaches the model. The model cannot infer completion from a status code, and it will report exactly the confidence the token implies.

This sits in the same first-principles line as validation is a loop, not an assertion and determinism where you can, judgement where you must. Validation-is-a-loop argues that in a generative system you have to check the agent's work rather than assume it. This piece is about a narrower and sneakier place the check goes missing: not the model's reasoning, but the tool boundary underneath it, where a status code gets mistaken for a fact. It is also the direct descendant of an old distributed-systems question I wrote about in RPC vs NATS, who owns completion. The agent stack rediscovered that question and, for the most part, got it wrong.

The 200 that means accepted, not done

Trace the token upward through a typical stack and you find three separate places where an honest but incomplete signal gets promoted into a false one.

The tool API layer. The tool returns 200 because the request was accepted, not because the effect was observed. Email providers return 200 on enqueue. Async job endpoints return 202 with a job id. Eventual-consistency writes return 200 the moment the write is durable on one node, before it has propagated anywhere a subsequent read would find it. In every one of these, 200 is a true statement about receipt and says nothing about outcome.

The agent framework layer. The framework sees a 2xx, marks the tool call as succeeded, and appends a success entry to the conversation. This is the layer that does the real damage, because it collapses two genuinely different states, request accepted and effect observed, into one word. The framework had the status code and threw away everything about what it meant.

The model layer. The model reads success in the tool result and generates Done. Email sent, with high confidence. The confidence is not a defect. It is the correct response to the token it was given. Ask a person to relay a message stamped SUCCESS and they will relay it as success too. The model is the last honest link in a chain that lied to it three steps earlier.

The customer, at the end of all this, gets a confident report that contradicts observable reality. And the postmortem points at the one component that behaved correctly given its inputs.

Where completion ownership went

The underlying problem is older than agents, older than HTTP. Any time an operation crosses a boundary, someone has to own the answer to did it actually happen.

RPC's classic wound is the caller who gets a network error partway through a call. Did the operation run or not? A network error is not a no. It is an unknown. The operation may have committed on the server and had its acknowledgement lost on the way back. Without an idempotency key to retry safely and a read-after-write check to confirm the effect, the caller genuinely cannot tell, and any recovery it attempts is a guess.

The way out, in every reliable system, is to name an owner. Either the caller polls until it sees the effect with its own eyes, or the receiver commits and only then reports back. As I argued in the RPC vs NATS piece, fire-and-forget messaging pushes ownership onto the caller, and request-response pushes it onto the receiver. Neither placement is wrong. What is always wrong is ambiguity about which layer holds it, because ambiguity means the answer is nobody, and nobody is exactly what produces the phantom confirmation email.

AI tool calls inherited this problem and, in the rush to wire models to real actions, mostly skipped the part where you decide who owns completion. The wrapper trusts the API's 200. The API trusts its dispatcher. The dispatcher trusts the worker. The worker logs success on enqueue. The token travels all the way up to the model and out to the user, and at no point did any layer commit to having observed the effect.

Declare the mode in the wrapper

The fix starts by refusing to let a status code stand in for an outcome. Every tool wrapper has to declare which of three modes it operates in, and that declaration lives in the wrapper, not in the model's head.

Mode	Returns when	Caller responsibility
`committed`	The effect has been observed	May report success directly
`accepted`	The request was enqueued, effect not yet observed	Must verify the effect before reporting success
`optimistic`	Best-effort send, no guarantee available	Must surface uncertainty to the user

This is a wrapper-level discipline for a concrete reason: the model cannot derive the mode from a 200. Two tools can return byte-identical success payloads while one has durably applied its effect and the other has merely queued it. The distinction exists only in knowledge the wrapper has and the status code does not carry. If the wrapper does not encode the mode, the information is gone by the time the model sees the result, and no amount of prompting recovers it. This is the technique boundary in miniature: the deterministic layer knows something the model cannot infer, so the deterministic layer has to state it.

The closed loop for accepted-mode tools

For anything in accepted mode, success is not a value the tool returns. It is a state a verifier confirms. The wrapper returns a handle, and a verification step stands between the tool and the model's next turn.

The wrapper returns {"status": "accepted", "polling_token": "..."}. It does not return success.
A verifier step polls the token until it reaches a terminal state, delivered or failed, reading the actual effect rather than re-reading the queue that accepted it.
Only the terminal state is surfaced to the model.
If polling exhausts its budget, the wrapper surfaces uncertain, the operation may or may not have completed, and explicitly not success.

That verifier is middleware sitting between the tool and the next model turn, the same shape as the loop in validation is a loop, not an assertion: the agent acts, the system verifies, scores the result, and routes on it. And the verification has to observe the effect itself, not a proxy for it. Re-reading the queue that already said accepted will happily confirm accepted forever. That is a wrong ruler: a check that looks like verification but measures the wrong thing, which is worse than no check because it manufactures false confidence. A read-after-write worthy of the name reads the recipient's mailbox state, the inserted row, the written file, not the acknowledgement that a request to produce them was received.

Agent action
   |
   v
Tool wrapper
   |
   |-- committed --> Effect observed --------------> Model context: "success"
   |
   |-- accepted --> Verifier (poll the real effect)
                        |-- terminal: delivered ---> Model context: "success"
                        |-- terminal: failed ------> Model context: "failed"
                        |-- budget exhausted ------> Model context: "uncertain"
                        \-- not yet terminal ------> keep polling (loop back)

Every path in that graph ends at an honest token. The only way the model reports success is if some layer below it actually watched the effect land.

Five questions for every tool in the box

This is ordinary reliability hygiene, applied one tool at a time. For each tool your agent can call, you should be able to answer yes to all five:

Does the return value distinguish queued from applied?
If the tool is asynchronous, is there a polling token or a callback to confirm the terminal state?
Does the wrapper enforce verification before success can reach the model?
On verification timeout, does the wrapper surface uncertain, rather than defaulting to success or failure?
Is there a retry budget owned in one place, so a burst of retries cannot compound the ambiguity?

That last question is where this connects to cost as well as correctness. Uncoordinated retries stacked at three layers are exactly the retry storm that inflates an AI bill, and they are also a completion-ownership failure: three layers each hoping the operation happened, none of them owning the answer. One retry budget in one place fixes both faces of the same bug.

Any tool that fails one of these five carries completion-ownership debt, and that debt is paid in user-visible incorrectness, the agent asserting things that did not happen.

It is 2PC versus eventual consistency, wearing a tool schema

For anyone who has built distributed systems, the whole thing has a familiar shape. A two-phase commit says I hold ownership and will not report done until the effect is durable. Eventual consistency says I forward my part and trust the next layer to converge. Both are legitimate designs. The trouble is that most agent tool stacks are built like eventual-consistency systems but report like two-phase commits: they emit a crisp, immediate success for an effect that is still only propagating, or still only queued, or already quietly dropped.

The fix is not to force every tool into two-phase commit. Plenty of effects are genuinely eventual, and that is fine. The fix is honesty in the report. If the system is eventual, the token that reaches the model has to say so, accepted and then a verified terminal state, never a premature success. If you have shipped idempotency keys and read-after-write checks before, you already have every piece of this. The polling token is a request UUID with a different label, and the verifier is a read-after-write check you already know how to write.

The bill for skipping it does not arrive as a stack trace. It arrives as a support ticket, hours later, from a customer the agent told with total confidence that something was done. Read the completion one layer earlier, in the wrapper, where you can still tell accepted from observed, and the model stops lying, because for the first time nothing below it is lying to the model.

This piece sits in the Generative Systems, First Principles line. Its correctness siblings: Validation Is a Loop, Not an Assertion, A Wrong Ruler Is Worse Than No Ruler, and Determinism Where You Can, Judgement Where You Must. The distributed-systems ancestor: RPC vs NATS, Who Owns Completion. The cost companion: Your AI Bill Is a Distributed Systems Problem.

Originally published at harrisonsec.com.