The agent had one job: update a customer record, issue a refund, and send a confirmation email.
It called all three tools.
Then it replied:
Done — the customer record was updated, the refund was issued, and the confirmation email was sent.
There was just one problem: the refund tool had failed.
The HTTP request timed out, the tool returned an error object, and the agent continued as if nothing had happened. From the user's perspective, the workflow succeeded. From the payment system's perspective, no refund existed.
That was the moment I stopped treating a tool call as proof that an action happened.
A tool call is only an attempt
A lot of agent workflows follow roughly this pattern:
- The model decides which tool to call.
- The application executes the tool.
- The tool returns something.
- The result goes back into the conversation.
- The model writes a final answer.
The dangerous assumption is between steps three and four:
If the tool returned, the operation must have succeeded.
That assumption breaks surprisingly easily.
A tool can return:
- an HTTP 500 response;
- a timeout;
- malformed JSON;
-
{ success: false }; - a partial result;
- a valid-looking response for an operation that was never committed;
- or an error that gets converted into an ordinary string.
If the application passes that result back without preserving its failure state, the model may interpret “the tool ran” as “the task completed.”
The model is not necessarily ignoring the error. Sometimes the surrounding code has already erased the distinction.
The version that failed silently
Here is a simplified version of the pattern that caused the problem:
async function updateCustomer() {
return { customerId: "cus_123", updated: true };
}
async function issueRefund() {
throw new Error("Payment provider timed out");
}
async function sendEmail() {
return { messageId: "msg_456", sent: true };
}
async function runTool(name, fn) {
try {
const result = await fn();
return {
tool: name,
result,
};
} catch (error) {
return {
tool: name,
result: error.message,
};
}
}
const results = [];
results.push(await runTool("update_customer", updateCustomer));
results.push(await runTool("issue_refund", issueRefund));
results.push(await runTool("send_email", sendEmail));
console.log(results);
Run it with:
node agent.js
The output looks reasonable at first glance:
[
{
"tool": "update_customer",
"result": {
"customerId": "cus_123",
"updated": true
}
},
{
"tool": "issue_refund",
"result": "Payment provider timed out"
},
{
"tool": "send_email",
"result": {
"messageId": "msg_456",
"sent": true
}
}
]
But the failed tool has the same outer shape as the successful tools. They all produced a result.
If this array is turned into a prompt such as:
The tools have finished. Summarize the completed actions for the user.
the workflow is already encouraging the wrong conclusion.
The error became content instead of state.
I now make failure impossible to miss
The first fix is not a smarter prompt. It is a stricter result contract.
Every tool execution should return an explicit status that the orchestration layer can evaluate without asking the model to interpret arbitrary text.
async function runTool(name, fn) {
const startedAt = Date.now();
try {
const data = await fn();
return {
tool: name,
status: "succeeded",
data,
error: null,
durationMs: Date.now() - startedAt,
};
} catch (error) {
return {
tool: name,
status: "failed",
data: null,
error: {
message: error.message,
retryable: error.message.includes("timed out"),
},
durationMs: Date.now() - startedAt,
};
}
}
Now success and failure cannot accidentally share the same shape.
More importantly, the application can make a deterministic decision before the model writes anything:
function summarizeExecution(results) {
const succeeded = results.filter(
(result) => result.status === "succeeded"
);
const failed = results.filter(
(result) => result.status === "failed"
);
return {
status:
failed.length === 0
? "completed"
: succeeded.length === 0
? "failed"
: "partially_completed",
succeeded,
failed,
};
}
Using the same three tools:
async function main() {
const results = [];
results.push(
await runTool("update_customer", updateCustomer)
);
results.push(
await runTool("issue_refund", issueRefund)
);
results.push(
await runTool("send_email", sendEmail)
);
console.dir(summarizeExecution(results), {
depth: null,
});
}
main().catch(console.error);
The workflow now has three possible outcomes:
completedpartially_completedfailed
“Partially completed” matters. Collapsing every workflow into success or failure hides the most uncomfortable state: some real-world actions happened and others did not.
The application should decide whether "done" is allowed
I used to let the model infer the final workflow status from tool messages.
I do not do that anymore.
The orchestration layer determines whether the completion claim is permitted:
function buildFinalResponse(execution) {
if (execution.status === "completed") {
return {
completionAllowed: true,
message:
"Done. All requested actions completed successfully.",
};
}
if (execution.status === "partially_completed") {
const completedTools = execution.succeeded
.map((item) => item.tool)
.join(", ");
const failedTools = execution.failed
.map((item) => item.tool)
.join(", ");
return {
completionAllowed: false,
message:
`I couldn't complete the full request. ` +
`Completed: ${completedTools}. ` +
`Failed: ${failedTools}.`,
};
}
return {
completionAllowed: false,
message:
"I couldn't complete the requested actions.",
};
}
The model can still make the response clearer and more conversational, but it cannot upgrade partially_completed to completed.
That boundary belongs in code.
Tool planning and tool execution are different states
Another mistake I made was treating a planned action as if it were already underway.
An agent might produce a plan like this:
[
{
"tool": "update_customer",
"status": "planned"
},
{
"tool": "issue_refund",
"status": "planned"
},
{
"tool": "send_email",
"status": "planned"
}
]
Those entries should move through explicit states:
planned
-> executing
-> succeeded
planned
-> executing
-> failed
For operations with side effects, I also track an operation ID:
const operation = {
operationId: crypto.randomUUID(),
tool: "issue_refund",
status: "planned",
attempts: 0,
externalReference: null,
};
This helps answer questions that become critical after a timeout:
- Did the request reach the provider?
- Did the provider complete it before the connection failed?
- Is retrying safe?
- Do we have an external transaction ID?
- Could a retry issue the refund twice?
A timeout does not always mean “nothing happened.” Sometimes it means “the client does not know what happened.”
That is a different failure state, and it should not be retried blindly.
Do not continue after every failure
In my original workflow, the confirmation email still ran after the refund failed.
That created an even worse outcome: the customer received an email saying the refund had been processed when it had not.
Tool dependencies need to be represented in the workflow, not left for the model to guess.
async function runRefundWorkflow() {
const update = await runTool(
"update_customer",
updateCustomer
);
if (update.status === "failed") {
return summarizeExecution([update]);
}
const refund = await runTool(
"issue_refund",
issueRefund
);
if (refund.status === "failed") {
return summarizeExecution([update, refund]);
}
const email = await runTool(
"send_email",
sendEmail
);
return summarizeExecution([
update,
refund,
email,
]);
}
The confirmation email is now conditional on a successful refund.
This sounds obvious when written down. It becomes less obvious in an agent loop where the model dynamically chooses the next action and every tool result is appended to the same conversation.
What I log for every real-world action
For read-only tools, lightweight logging may be enough.
For tools that send emails, move money, change permissions, delete files, or update customer data, I want a much stronger audit trail:
{
operationId: "op_8f23...",
workflowId: "wf_19ab...",
tool: "issue_refund",
status: "failed",
attempt: 1,
startedAt: "2026-07-13T08:20:10.000Z",
finishedAt: "2026-07-13T08:20:12.413Z",
durationMs: 2413,
retryable: true,
idempotencyKey: "refund_order_789",
externalReference: null,
errorCode: "PROVIDER_TIMEOUT"
}
The exact schema is less important than being able to reconstruct what happened without relying on the agent's final sentence.
The final sentence is presentation.
The execution log is evidence.
Prompts still help, but they are not enforcement
I still tell the model to be explicit about partial failure:
Never claim that a task is complete unless every required
tool operation has status "succeeded".
If one or more required operations failed:
1. State that the request was not fully completed.
2. List which actions succeeded.
3. List which actions failed.
4. Do not imply that a failed external action occurred.
5. Do not retry actions with side effects unless the
orchestration layer explicitly permits it.
This improves the wording and reduces ambiguous responses.
But a prompt is not where I enforce the rule.
The application should:
- preserve structured tool status;
- validate required results;
- stop dependent actions after failure;
- prevent unsafe retries;
- and block completion claims when the workflow is incomplete.
The model should explain the state, not decide whether the state exists.
The test I wish I had written earlier
I now include a test that deliberately fails one required tool:
import assert from "node:assert/strict";
const execution = summarizeExecution([
{
tool: "update_customer",
status: "succeeded",
},
{
tool: "issue_refund",
status: "failed",
},
{
tool: "send_email",
status: "succeeded",
},
]);
const response = buildFinalResponse(execution);
assert.equal(
execution.status,
"partially_completed"
);
assert.equal(
response.completionAllowed,
false
);
assert.doesNotMatch(
response.message,
/^Done\b/
);
It is a small test, but it protects an important product promise:
The agent cannot say “done” unless the system has evidence that the required work is done.
"Done" is a production state
The hardest agent failures are not always crashes.
Sometimes the workflow keeps moving, produces a polished answer, and gives the user no reason to suspect that anything went wrong.
That is more dangerous than an obvious exception.
An exception asks for attention. A false success closes the conversation.
The rule I use now is simple:
A requested action is not complete because the model asked for a tool, or because the tool returned something. It is complete only when the application has validated the result.
I work on TokenBay, so I spend a lot of time thinking about what happens between model output, provider behavior, and real application state. The more actions we give agents permission to take, the less comfortable I am letting a fluent final answer serve as proof of execution.
Top comments (0)