What should an AI agent do when a tool may have succeeded before a crash?

wang vince — Mon, 21 Sep 2026 15:52:22 +0000

The familiar agent loop is model → tool → result. It is a useful sketch, but it hides a failure window: the tool may change the outside world, then the agent process may die before it records the result.

After a restart, “I did not receive a result” does not mean “nothing happened.” Repeating the call might duplicate a payment, an API request, or a file operation. Treating it as successful might skip work that never happened. A language model cannot recover this fact by reasoning over a transcript that does not contain it.

I ran into this while building Kiso, an open-source TypeScript runtime for AI agents. I initially wanted to make a smaller, more token-efficient agent than Pi. Reading Pi closely made me realize that shaving a few tools or prompt tokens was less interesting than defining what an agent is allowed to treat as a fact after a crash.

Record the boundaries, not just the conversation

Kiso writes an append-only event log. For a tool call that may have an effect, the relevant order is:

complete model response
    ↓
durable turn commit
    ↓
approval decision
    ↓
durable execution start
    ↓
tool runs and may change the outside world
    ↓
durable success or failure receipt
    ↓
result is shown to the model

The turn commit matters. A streamed model response can contain the beginning of a tool call and then fail or end incompatibly. Until the whole response has completed cleanly, that tool call is only a draft; it has no authority to change anything.

The execution start matters for a different reason. It is written before the tool handler runs, so recovery can distinguish a call that never started from one whose outcome is now unknown.

What recovery can actually know

Last durable fact	What recovery knows
No committed turn	The incomplete model output cannot authorize a tool call. It may be generated again.
Committed call, no execution start	The tool handler has not run.
Execution start, no receipt	The tool may or may not have changed the outside world. Its outcome is uncertain.
Success or failure receipt	The recorded result can be reused; a confirmed success is not run again.

The third row is the hard one. Kiso stops there and asks a person to decide whether to rerun or abandon that execution. For a remote API, that person might first inspect the remote system. If the API offers idempotency keys or a way to query the result, those are better sources of evidence than a guess from the model.

This does not claim exactly-once effects in an external system. A local event log cannot prove what happened inside a bank or another service after the connection was lost. The useful guarantee is narrower: Kiso will not silently turn an unknown outcome into a success or automatically repeat an effect it cannot prove failed.

Why this belongs in the runtime

A normal “continue” prompt works when someone deliberately stops an agent and the host can tell the model what happened. It is weaker when the whole process disappears in the middle of an action. The new process needs durable execution evidence, not a plausible story assembled from the surviving conversation.

Kiso includes a TUI coding agent, kiso-code, to exercise this design in a real tool loop. The repository has an end-to-end test that kills the agent process group during a shell call and resumes the same trajectory in a fresh process. The test checks that the interrupted execution is marked uncertain and requires a decision before the run continues.

The design has costs. An unfinished model generation may need to be requested again, and an uncertain external action requires investigation. I think making those costs visible is preferable to hiding them behind an automatic retry.

The Kiso repository includes the runnable coding agent, durability contract, and the real kill -9 test. I would be interested in how other agent runtimes handle this window, especially for tools without idempotency support.

Disclosure: This English article was drafted with AI assistance from my original Chinese writing. The technical claims were checked against the Kiso repository.

DEV Community: wang vince

What should an AI agent do when a tool may have succeeded before a crash?

Record the boundaries, not just the conversation

What recovery can actually know

Why this belongs in the runtime