- Book: AI That Acts
- The series: AI in TypeScript — 5 books, from your first LLM call to agents in production — all five here
- My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
- Me: xgabriel.com | GitHub
An agent turn fails after the tool ran. The network dropped between
your charge_card call succeeding and the response reaching the
model. Your retry logic does what retry logic does: it replays the
turn.
The card is charged twice.
Nothing in that sequence is a bug in the ordinary sense. The tool
worked. The retry worked. What was missing is the property that makes
replaying safe — and in an agent loop there are three separate places
a replay can originate, which is more than most retry code accounts
for.
Where the duplicate comes from
Your own retry. A 500 or a timeout from the model provider, and
you re-send the same messages. If the previous attempt's tool ran
before the failure, it runs again.
The model calling twice. It gets a tool result it finds
unconvincing — empty, ambiguous, an error it reads as transient — and
issues the same call again in the next turn. This one is not a bug
anywhere; it is the model doing what a person would do.
Resume after a crash. The process dies mid-turn. A durable agent
restarts from its last checkpoint, which is before the tool call, and
replays it.
All three produce the same shape: the same logical operation
attempted more than once. The fix is not to prevent duplicates but to
make them harmless.
The type-level split
Start by making the two kinds of tool distinguishable to the
compiler, because most of them do not need any of this machinery.
type ReadTool<S extends z.ZodType, R> = {
kind: "read";
name: string;
schema: S;
run: (a: z.infer<S>, ctx: Ctx) => Promise<R>;
};
type WriteTool<S extends z.ZodType, R> = {
kind: "write";
name: string;
schema: S;
run: (a: z.infer<S>, ctx: Ctx & { idem: string }) => Promise<R>;
};
export type AnyTool = ReadTool<any, any> | WriteTool<any, any>;
The asymmetry is deliberate: a write handler receives idem in its
context, a read handler does not. That single difference means a
write tool cannot be written without the author noticing there is an
idempotency key to use, and a read tool cannot accidentally depend on
one.
const search = { kind: "read", /* ... */ } satisfies ReadTool<...>;
const charge = { kind: "write", /* ... */ } satisfies WriteTool<...>;
Retrying a read is free. Retrying a write is the whole problem. Now
tool.kind tells you which you are holding.
The key must be the operation, not the attempt
The obvious key is the tool_use_id from the model's block. It is
convenient and it is wrong for the second failure mode.
tool_use_id identifies this attempt. When the model issues the
same call again in a later turn, it is a new block with a new id — so
an id-keyed dedupe sees two different operations and lets both
through.
Key on what the operation is:
import { createHash } from "node:crypto";
export function idemKey(
runId: string,
toolName: string,
args: unknown,
): string {
const canonical = JSON.stringify(args, Object.keys(args as object).sort());
const h = createHash("sha256")
.update(`${runId}\0${toolName}\0${canonical}`)
.digest("hex")
.slice(0, 32);
return `idem:${h}`;
}
runId scopes it to one agent run, so the same user charging the
same amount tomorrow is a different operation. Sorted keys make the
hash stable regardless of property order. And the null separators
stop ("ab", "c") and ("a", "bc") colliding.
Scoping to the run is a decision worth making consciously. Scope too
narrowly and legitimate repeats within a run get blocked. Scope too
broadly and a user genuinely buying the same thing twice is refused.
Run-scoped is the right default for agents; adjust where your domain
says otherwise.
The store
Three states, not two. "In flight" is the one that gets forgotten,
and it is what protects you against concurrent duplicates rather than
sequential ones.
export type Entry =
| { status: "running"; startedAt: number }
| { status: "done"; result: unknown }
| { status: "failed"; error: string };
export interface IdemStore {
claim(key: string, ttlMs: number): Promise<Entry | null>;
complete(key: string, result: unknown, ttlMs: number): Promise<void>;
fail(key: string, error: string): Promise<void>;
}
claim returns null if it won the claim and you should execute, or
the existing entry if someone got there first. On Redis that is one
atomic operation:
export class RedisIdem implements IdemStore {
constructor(private redis: Redis) {}
async claim(key: string, ttlMs: number) {
const running: Entry = { status: "running", startedAt: Date.now() };
const ok = await this.redis.set(
key, JSON.stringify(running), "PX", ttlMs, "NX",
);
if (ok === "OK") return null;
const raw = await this.redis.get(key);
return raw ? (JSON.parse(raw) as Entry) : null;
}
async complete(key: string, result: unknown, ttlMs: number) {
const done: Entry = { status: "done", result };
await this.redis.set(key, JSON.stringify(done), "PX", ttlMs);
}
async fail(key: string, error: string) {
await this.redis.del(key);
}
}
SET NX PX is the atomic claim — set only if absent, with an
expiry. Read-then-write would leave a window where two workers both
see nothing and both proceed.
fail deletes rather than recording the failure, so a genuinely
transient error can be retried. If you want failures to stick,
record them and decide by error class; deleting is the safer default.
The wrapper
export function idempotent<S extends z.ZodType, R>(
t: WriteTool<S, R>,
store: IdemStore,
ttlMs = 24 * 60 * 60 * 1000,
): WriteTool<S, R> {
return {
...t,
async run(args, ctx) {
const key = idemKey(ctx.runId, t.name, args);
const existing = await store.claim(key, ttlMs);
if (existing?.status === "done") {
return existing.result as R;
}
if (existing?.status === "running") {
throw new OperationInFlight(t.name);
}
try {
const result = await t.run(args, { ...ctx, idem: key });
await store.complete(key, result, ttlMs);
return result;
} catch (err) {
await store.fail(key, String(err));
throw err;
}
},
};
}
Returning the cached result on a duplicate — rather than an error —
is what makes this invisible to the model. It asked for the charge,
it gets the charge result. It never learns there was a retry, which
is correct: the operation happened exactly once and the model's view
of the world is accurate.
Applying it is a line in the registry:
const TOOLS = [
search,
idempotent(charge, store),
idempotent(sendEmail, store),
] as const;
Pass the key downstream
The wrapper stops your duplicate execution. It does not stop a
duplicate at the payment provider if your process dies between their
API succeeding and your complete call.
That is why idem is in the write handler's context — hand it to the
downstream system that has its own idempotency support:
const charge: WriteTool<typeof ChargeArgs, Receipt> = {
kind: "write",
name: "charge_card",
schema: ChargeArgs,
async run({ amount, currency, customerId }, ctx) {
const res = await stripe.paymentIntents.create(
{ amount, currency, customer: customerId },
{ idempotencyKey: ctx.idem },
);
return Receipt.parse(res);
},
};
Now the guarantee holds across a crash in the gap. Most payment and
messaging APIs accept a key like this; using yours rather than
generating a fresh one is what chains the two layers together.
Testing it
The test is more valuable than the implementation, because the
implementation looks obviously correct and the bug is in the
sequencing.
it("executes once when the same call arrives twice", async () => {
let calls = 0;
const t = idempotent(
{ kind: "write", name: "t", schema: Args,
run: async () => { calls++; return { id: "r1" }; } },
store,
);
const a = await t.run({ amount: 100 }, ctx);
const b = await t.run({ amount: 100 }, ctx);
expect(calls).toBe(1);
expect(b).toEqual(a);
});
it("rejects a concurrent duplicate", async () => {
const [x, y] = await Promise.allSettled([
t.run({ amount: 100 }, ctx),
t.run({ amount: 100 }, ctx),
]);
const fulfilled = [x, y].filter((r) => r.status === "fulfilled");
expect(fulfilled).toHaveLength(1);
});
Sequential and concurrent are different paths through the store. A
naive read-then-write implementation passes the first test and fails
the second, which is exactly the failure that only appears under
load.
The rule
Mark every tool that changes something. Give the operation an
identity derived from what it does, not from which attempt this is.
Claim before executing, cache the result, and hand the key onward.
Then a retry is a retry rather than a second charge.
If this was useful
AI That Acts covers tools
that touch the real world — the read/write distinction, idempotency,
error results, and the guards that make an agent's side effects
something you can reason about.
Crash-resume, where this matters most, is book four. The full series
is at xgabriel.com/ai-in-typescript.
Top comments (0)