AI Agent Reconciliation: What to Do When a Tool Times Out After It May Have Succeeded

A timeout after an agent sends a refund, deploy, booking, or customer message is not a failure. It is an unknown outcome: the remote system may have completed the work after your connection stopped waiting. If the agent retries because it sees the word “timeout,” one intended action can become two real-world actions.

The fix is not a better prompt that says “be careful.” Put a reconciliation protocol in the tool layer: assign one durable operation key, record the intended effect before dispatch, query for evidence after an ambiguous result, and only retry when the evidence says the effect did not happen. This guide shows the state model, API contracts, and tests that make that rule practical.

Why a timeout is not a failed action

There are two timelines in every state-changing tool call:

1. The caller sends a request and waits for an acknowledgement.
2. The downstream service receives the request and applies an effect.

A timeout only tells you that timeline one ended without a useful answer. It says nothing conclusive about timeline two. A worker can crash after sending the request. A load balancer can drop the response. A payment service can commit just after the client deadline. In each case, immediately sending the same logical request again can duplicate the effect.

This matters more for agents than ordinary forms. Agents are designed to recover and continue. They can interpret an error as a reason to retry, call a fallback tool, or resume a durable workflow later. A natural-language instruction cannot be the final authority over a write to an external system.

Treat these outcomes as separate states:

Tool result	Meaning	Next move
confirmed	The target returned a receipt or authoritative result.	Continue.
rejected	The target proved it did not accept the operation.	Correct input or stop.
unknown	A timeout, connection reset, process crash, or malformed acknowledgement occurred.	Reconcile; do not retry.
not_found	Reconciliation can prove the effect did not land.	Retry with the same operation key if safe.
conflict	Evidence is incomplete or contradictory.	Escalate; do not guess.

The important design choice is exposing unknown as a first-class outcome. Do not translate it into false, throw a generic exception, or let the model decide what it means.

The narrow problem this solves

Idempotency is necessary, but it is not the whole recovery story. It prevents many duplicate writes when a downstream API accepts an idempotency key. Reconciliation answers a different question: what actually happened to this intended business operation?

That distinction matters when an integration does not support client keys, when a write spans several systems, or when a successful response was lost. It also prevents the opposite bug: an agent tells a customer an action failed even though it succeeded.

Use this pattern for consequential writes such as:

• creating a charge, refund, booking, order, ticket, or deployment;
• sending an external email, Slack message, or webhook;
• changing a permission, subscription, or account setting;
• deleting or publishing a customer-visible record.

Pure reads can normally retry. A deterministic update such as “set plan to Pro” is safer than “append a note,” but it still needs an operation record if the user needs a truthful outcome.

Make the application own effect state

The model should propose an action. Your application should own its identity, authorization, execution, and recovery. A minimal operation ledger looks like this:

create table agent_operations (
  operation_id uuid primary key,
  tenant_id uuid not null,
  run_id uuid not null,
  tool_name text not null,
  request_hash text not null,
  state text not null check (state in (
    'prepared', 'dispatched', 'confirmed', 'unknown', 'reconciling',
    'not_found', 'escalated', 'cancelled'
  )),
  provider_reference text,
  receipt jsonb,
  reconcile_after timestamptz,
  created_at timestamptz not null default now(),
  updated_at timestamptz not null default now(),
  unique (tenant_id, tool_name, operation_id)
);

operation_id identifies one intended business effect, not one HTTP attempt. Store a hash of the normalized request too. If a resumed agent presents the same operation ID with different arguments, reject it. That is a serious programming or orchestration error, not a request to “try again.”

Create the row before making the external call. Otherwise, a crash in the gap between dispatch and persistence recreates the very ambiguity you are trying to control.

Design tool contracts for honest recovery

An agent-facing write tool should not return a raw network exception. Return a small structured result that forces the workflow into a safe branch.

type EffectResult =
  | { state: "confirmed"; operationId: string; receipt: Record<string, unknown> }
  | { state: "rejected"; operationId: string; reason: string }
  | { state: "unknown"; operationId: string; retryAllowed: false; nextCheckAt: string }
  | { state: "escalated"; operationId: string; reason: string };

The wrapper receives a stable operationId, writes it to the ledger, and passes it downstream as an idempotency header or external reference whenever possible.

async function sendRefund(input: RefundInput, operationId: string): Promise<EffectResult> {
  await operations.prepare(operationId, input);

  try {
    const response = await billing.refund({
      ...input,
      idempotencyKey: operationId,
      metadata: { agent_operation_id: operationId },
    });
    await operations.confirm(operationId, response.id, response);
    return { state: "confirmed", operationId, receipt: { refundId: response.id } };
  } catch (error) {
    if (isDefiniteRejection(error)) {
      await operations.reject(operationId, safeReason(error));
      return { state: "rejected", operationId, reason: safeReason(error) };
    }
    await operations.markUnknown(operationId, error);
    return {
      state: "unknown", operationId, retryAllowed: false,
      nextCheckAt: new Date(Date.now() + 30_000).toISOString(),
    };
  }
}

Notice what is missing: there is no catch { retry() }. The tool owns transport uncertainty so the agent never treats an unconfirmed write as a fresh task.

Reconcile before any retry

Reconciliation is a targeted read of an authoritative system of record. The best API pair is a write that accepts your client-generated key and a read such as GET /refunds?operation_id=.... That read should return the effect’s lifecycle state, not merely recent activity that happens to look similar.

Use this decision flow:

prepare operation → dispatch write → acknowledgement?
                                 ├─ yes → confirm receipt → continue
                                 └─ no  → mark unknown → query by operation ID
                                                     ├─ found → confirm
                                                     ├─ absent after safe window → retry same key
                                                     └─ partial/ambiguous → escalate

The “safe window” is integration-specific. A provider with eventual consistency may need several read attempts with exponential backoff before absence means anything. Record the policy per tool: maximum reconciliation delay, available lookup key, whether a retry is safe, and who owns escalation.

For systems that cannot query by a client key, look for a compound business identity: a draft ID, customer ID plus immutable request nonce, or a provider correlation field. If you cannot reliably identify the effect, the correct automated outcome is escalated, not a duplicate write disguised as resilience.

Do not confuse a similar record with proof

A tempting shortcut is to search for “a recent refund for this customer” or “a message with roughly this text.” That can turn reconciliation into a new source of bad decisions. A customer may have made two valid requests, an operator may have sent the message manually, or a background job may have changed the same record.

Define the minimum evidence needed to confirm every integration. Prefer a provider receipt that carries the operation key. If that is unavailable, require a combination of immutable business ID, normalized request hash, target identity, and a time bound that is documented by the provider. Mark the result as conflict when evidence is merely plausible rather than conclusive.

This is also where tenant boundaries matter. An operation lookup must be scoped by the requesting tenant before its evidence reaches an agent or reviewer. A global search for a matching customer email can accidentally confirm another tenant’s action. Keep the target reference, hashes, and receipts in application-owned storage, and authorize reconciliation reads exactly as you authorize the original write.

Give each operation a small state machine

Do not let every caller update the ledger with arbitrary strings. Define allowed transitions such as prepared → dispatched → confirmed, dispatched → unknown → reconciling → confirmed, and unknown → reconciling → escalated. Disallow confirmed → dispatched entirely. The guard belongs in the database transaction or service method, not in the prompt or UI.

This protects against races between a late provider webhook, a queue retry, and a human reviewer. When a webhook supplies a valid receipt after the reconciliation job has started, it can safely claim confirmed; the job then sees the terminal state and exits. When a reviewer declines an uncertain deletion, that decision becomes a terminal cancelled record rather than an instruction an agent can reinterpret later.

Include an optimistic version or row lock on transitions, and append an immutable event for each transition. The current state makes the workflow fast to query; the event history explains why it reached that state. This is especially valuable during incident review, where you need to distinguish “the provider acted late” from “two workers independently dispatched the same effect.”

Use a reconciliation queue, not a blocked agent turn

Avoid holding a model session open while waiting for a slow target. When a wrapper returns unknown, enqueue a durable reconciliation job keyed by operation_id. That job should lock the operation, perform a bounded read-back sequence, and write one terminal state.

async function reconcile(operationId: string) {
  const op = await operations.lockForReconciliation(operationId);
  if (["confirmed", "rejected", "escalated"].includes(op.state)) return;

  const evidence = await provider.findByOperationId(op.operation_id);
  if (evidence.kind === "found") return operations.confirm(op.operation_id, evidence.id, evidence);
  if (evidence.kind === "not_found" && canRetry(op)) {
    await operations.markNotFound(op.operation_id);
    return queue.enqueue("replay_operation", { operationId: op.operation_id });
  }
  return operations.escalate(op.operation_id, evidence.reason);
}

The replay uses the same operation ID and the same normalized request. Do not restart the whole agent run; that can repeat unrelated successful steps. If a workflow includes five writes, reconcile each effect independently and allow downstream steps only after their prerequisites are confirmed.

Add evidence, boundaries, and human decisions

An operations table without useful evidence becomes a second mystery. Store redacted dispatch time, provider request ID, response status, error class, attempt count, reconciliation reads, and final receipt. Never store secrets or unredacted customer content merely to debug retries.

Classify each tool in a policy registry:

Action class	Example	On unknown outcome
Safe replay	Set a named record to a known value	Reconcile, then replay same key.
Dedupe-supported	Refund API with idempotency key	Reconcile; same key may be replayed.
Compensatable	Create ticket then update local state	Reconcile; compensate only with evidence.
Irreversible	Send a legal notice or delete a record	Reconcile, then human review.

Require an approval or incident handoff for conflicts involving money, irreversible deletion, permissions, or customer communications. The reviewer needs the intended request, operation ID, timeline, evidence, and choices—not hidden chain-of-thought or a vague “agent failed” message.

Test the failures that happy paths hide

Your integration tests should simulate a target that commits the effect and drops the response. That is the bug that ordinary mock tests miss.

At minimum, test these invariants:

• the same operation ID with changed input is rejected;
• a timeout produces unknown, never automatic replay;
• a reconciliation read that finds a receipt reaches confirmed without a second dispatch;
• an absent result only enables retry after the configured consistency window;
• two workers reconciling the same operation produce one terminal state;
• a resumed workflow uses existing operations rather than creating new ones;
• escalation contains redacted evidence and no model reasoning trace.

For a simple test double, let POST /refunds persist a refund and then throw a timeout. Run the wrapper twice. The expected result is one refund record, one operation ID, and a second call that returns the original receipt or waits for reconciliation.

Track unknown-outcome rate by tool, time-to-reconciliation, escalations, duplicate effects detected after the fact, and disagreements between an agent’s message and the reconciled state. Those metrics reveal fragile integrations before a rare timeout becomes a customer incident.

A rollout plan that does not freeze delivery

Start with one high-impact tool rather than wrapping every integration at once.

1. Inventory state-changing tools and rank them by irreversibility and customer impact.
2. Add operation IDs and ledger rows in observe-only mode; do not change retry behavior yet.
3. Capture whether each provider supports idempotency keys, lookup by key, and reliable receipts.
4. Switch the selected tool so ambiguous outcomes become unknown and enter reconciliation.
5. Add alerts for operations stuck past the tool’s expected window.
6. Run timeout, crash, duplicate-delivery, and delayed-consistency drills before expanding coverage.

This sequence produces evidence before policy changes. It also makes conversations with API owners concrete: ask for a client-generated key, a lookup endpoint, and a receipt—not a promise that timeouts are rare.

FAQ

Is an AI agent timeout the same as a failed tool call?

No. A timeout only means the caller did not receive a conclusive response. The target may have applied the effect, so a state-changing call should enter an unknown-outcome reconciliation path.

What is an operation key for an AI agent?

It is a durable ID for one intended business effect. Reuse it across attempts, send it to the target when possible, and use it to find the effect during reconciliation.

Are idempotency keys enough for agent retries?

They are the best protection when every target honors them, but you still need a ledger and read-back path to report the outcome honestly and handle APIs without key support.

When can an agent retry a timed-out action?

Only after a provider receipt or authoritative lookup proves the effect did not happen, or when the action is intrinsically safe to replay. Use the same operation key, not a new one.

What should happen when reconciliation cannot determine the result?

Stop automatic execution and escalate with a redacted operation timeline. Guessing can create a duplicate charge, message, deletion, or permission change.

Should the language model decide whether an action succeeded?

No. Let the model propose the action and explain the user-facing result, but derive success from deterministic receipts and reconciled system state.

How is reconciliation different from a workflow retry?

A workflow retry repeats a task. Reconciliation checks whether the original business effect happened and chooses confirm, safe replay, compensation, or escalation for that one operation.