- Book: AI That Acts
- The series: AI in TypeScript — 5 books, from your first LLM call to agents in production — all five here
- My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
- Me: xgabriel.com | GitHub
An agent turn is only as fast as its slowest tool. One upstream service takes
forty seconds to fail, and the whole run sits there — the user watching a
spinner, your request holding a connection, the turn budget ticking.
Node makes this easy to get wrong in a specific way: await has no deadline,
and fetch without a signal will wait as long as the socket stays open.
A deadline is not a timeout
Most implementations put a timeout on each tool call. That is necessary and
insufficient, because five tools with a ten-second timeout each is a
fifty-second turn.
What you want is a run deadline that individual calls inherit:
export type Deadline = { at: number };
export const remaining = (d: Deadline) => Math.max(0, d.at - Date.now());
export function budgetFor(d: Deadline, toolMax: number) {
return Math.min(toolMax, remaining(d));
}
Now a tool that would normally get ten seconds gets three if only three
remain, and the run stays inside its overall promise to the caller.
const deadline: Deadline = { at: Date.now() + 60_000 };
Per-tool budgets, because tools differ
const TIMEOUTS: Record<string, number> = {
search_docs: 5_000,
get_order: 3_000,
generate_report: 30_000,
send_email: 8_000,
};
const DEFAULT_TIMEOUT = 10_000;
A uniform timeout is either too short for the report generator or too long for
a primary-key lookup. Neither is acceptable, and the table costs nothing.
Cancellation that actually cancels
The part people get wrong: racing a promise against a timer stops your
waiting but not the work.
// wrong — the fetch keeps going, the socket stays open
const out = await Promise.race([
runTool(block),
new Promise((_, rej) => setTimeout(() => rej(new Timeout()), ms)),
]);
The signal has to reach the I/O:
export async function executeWithDeadline(
block: ToolUseBlock,
ctx: Ctx,
deadline: Deadline,
): Promise<ToolResultBlockParam> {
const ms = budgetFor(deadline, TIMEOUTS[block.name] ?? DEFAULT_TIMEOUT);
if (ms <= 0) {
return errorResult(block.id,
"Skipped: the overall time budget for this task is exhausted.");
}
const ac = new AbortController();
const timer = setTimeout(() => ac.abort(), ms);
try {
const out = await runTool(block, { ...ctx, signal: ac.signal });
return okResult(block.id, out);
} catch (err) {
if (ac.signal.aborted) {
metrics.increment(`tool.timeout.${block.name}`);
return errorResult(block.id, timeoutMessage(block.name, ms));
}
throw err;
} finally {
clearTimeout(timer);
}
}
Three details that matter in Node.
clearTimeout in finally — an uncleared timer keeps the event loop alive,
and a process that will not exit after its work finishes is genuinely
unpleasant to diagnose.
Checking ac.signal.aborted rather than the error type — an aborted fetch
throws an AbortError, but a downstream library may wrap it into something
else. The signal is the reliable witness.
Passing signal down to every fetch and database call inside the tool. A
signal that stops at the tool boundary has cancelled nothing.
async run({ query }, ctx) {
const r = await fetch(url, { signal: ctx.signal });
return r.json();
}
What the model should be told
This is the part that decides whether the run recovers. A timeout is not a
generic failure — it carries information the model can act on.
function timeoutMessage(tool: string, ms: number): string {
return [
`${tool} did not respond within ${Math.round(ms / 1000)}s.`,
"This is a timeout, not an empty result — the data may exist.",
"Do not retry with the same arguments.",
"Either narrow the request, use a different tool, or continue without it",
"and tell the user which part is missing.",
].join(" ");
}
Every line is doing work. Distinguishing timeout from empty result stops the
model concluding the record does not exist. Forbidding an identical retry
prevents the loop this otherwise causes. And offering three explicit options
is what turns a dead end into a next step.
Compare it to what most implementations send:
return errorResult(block.id, "Error: ETIMEDOUT");
The model has no idea whether to retry, give up, or apologise, so it usually
retries.
Parallel calls share the deadline
When a turn emits several tool calls, they run concurrently and all inherit
the same run deadline:
const outs = await Promise.all(
blocks.map((b) => executeWithDeadline(b, ctx, deadline)),
);
Promise.all is safe here precisely because executeWithDeadline never
rejects — every path returns a result block. That is the property that makes
partial failure work: three tools succeed, one times out, and the model
receives all four outcomes.
Degrade instead of failing
For read-only tools, a timeout does not have to mean nothing.
async run({ query }, ctx) {
try {
return await liveSearch(query, ctx.signal);
} catch (err) {
if (!ctx.signal.aborted) throw err;
const cached = await cache.get(query);
if (cached) {
return { ...cached, stale: true, note: "Cached result; live search timed out." };
}
throw err;
}
}
The stale flag and the note are both for the model. A stale answer presented
as current is worse than a timeout; a stale answer labelled stale is usually
better than nothing, and the model will caveat it appropriately.
Watch the p99, not the mean
metrics.histogram("tool.duration_ms", ms, { tool: block.name });
metrics.increment(`tool.outcome.${block.name}.${outcome}`);
Timeout budgets should be set from the p99 of a healthy day, not from a round
number someone liked. A tool whose p99 is 4.2 seconds does not want a
three-second timeout, and you will only find that out from the histogram.
The ratio worth alerting on is timeouts as a share of calls per tool. A tool
that times out on one call in twenty is degrading your agent invisibly — the
runs still complete, just with a quarter of the information missing.
The checklist
Every tool gets a budget derived from a run-level deadline. Every abort signal
reaches the actual I/O. Every timer is cleared in finally. Every timeout
returns a result block, never a throw. Every timeout message says it was a
timeout, forbids the identical retry, and offers a next step.
Get those five right and a hanging upstream service costs you one degraded
answer instead of a hung request and a burnt budget.
If this was useful
AI That Acts covers the execution
layer of an agent — deadlines, cancellation, partial failure, and error
results written so the model can recover from them.
The full series is at
xgabriel.com/ai-in-typescript.
Top comments (0)