Most AI-agent interview answers fail at the same point: the candidate describes a clever workflow but cannot say how they would know it actually completed. A small trace checker gives you a better answer. It turns tool calls into explicit invariants, catches four common failure modes, and gives you concrete evidence to discuss under follow-up questions.
What are you actually validating?
A tool-using agent emits an event stream, not just a final paragraph. It may ask to search, fetch a record, write a file, or call an internal API. The final answer is only trustworthy when the trace tells a coherent story:
- Every requested tool is on an allowlist.
- Every result refers to one earlier call.
- A call receives no more than one result.
- The agent does not claim completion while a call is still pending.
Those are deliberately modest guarantees. They do not prove that a search result was true or that an external system behaved correctly. They do prove that your orchestrator did not silently lose, duplicate, or invent a step. That distinction is the kind of boundary interviewers usually probe.
Here is a compact, dependency-free checker. Save it as trace-check.js and run node trace-check.js.
import assert from "node:assert/strict";
const knownTools = new Set(["search_docs", "get_customer", "create_ticket"]);
function checkTrace(events) {
const pending = new Map();
const completed = new Set();
for (const event of events) {
if (event.type === "tool_call") {
if (!knownTools.has(event.name)) {
throw new Error(`unknown tool: ${event.name}`);
}
if (pending.has(event.id) || completed.has(event.id)) {
throw new Error(`duplicate call id: ${event.id}`);
}
pending.set(event.id, event.name);
continue;
}
if (event.type === "tool_result") {
if (!pending.has(event.callId)) {
throw new Error(`orphan result: ${event.callId}`);
}
pending.delete(event.callId);
completed.add(event.callId);
continue;
}
if (event.type === "final") {
if (pending.size) {
throw new Error(`final with pending calls: ${[...pending.keys()].join(", ")}`);
}
return { ok: true, completed: [...completed] };
}
throw new Error(`unexpected event type: ${event.type}`);
}
throw new Error("trace ended without a final event");
}
const valid = [
{ type: "tool_call", id: "c1", name: "search_docs" },
{ type: "tool_result", callId: "c1", value: ["design notes"] },
{ type: "final", text: "Here is the answer." },
];
assert.deepEqual(checkTrace(valid), { ok: true, completed: ["c1"] });
assert.throws(
() => checkTrace([{ type: "tool_call", id: "c1", name: "delete_everything" }]),
/unknown tool/,
);
assert.throws(
() => checkTrace([{ type: "tool_result", callId: "missing" }]),
/orphan result/,
);
assert.throws(
() => checkTrace([
{ type: "tool_call", id: "c1", name: "get_customer" },
{ type: "final", text: "Done" },
]),
/pending calls/,
);
assert.throws(
() => checkTrace([
{ type: "tool_call", id: "c1", name: "search_docs" },
{ type: "tool_result", callId: "c1" },
{ type: "tool_result", callId: "c1" },
]),
/orphan result/,
);
console.log("trace checker assertions passed");
The last assertion is worth noticing. Once a call is removed from pending, a second result is no longer a valid response. The checker reports it as an orphan instead of accepting it as harmless noise. In production, that prevents an old retry response from being mistaken for the answer to a newer action.
Why is this a better interview artifact than a diagram alone?
A diagram can make an agent look orderly. A runnable artifact forces you to name the rules that make it orderly.
When an interviewer asks, “How did you make sure tool use was reliable?”, avoid opening with a list of vendors or models. Walk through the contract:
| Concern | Concrete rule | Evidence you can show |
|---|---|---|
| Tool authorization | Call names must be known before execution |
unknown tool assertion |
| Correlation | A result has one matching earlier call |
orphan result assertion |
| Idempotency | A call is settled exactly once | duplicate-result assertion |
| Completion | No final answer with unfinished work | pending-call assertion |
This answer scales beyond one framework. Whether you use an SDK trace, a queue, or a hand-rolled event stream, you still need correlation IDs, a terminal state, and observable failures.
How should you explain the trade-offs?
The in-memory Map makes the tutorial easy to run, but it would not be enough for a multi-process service. A strong interview answer should say that without being prompted.
For a production version, I would make three changes:
- Persist the call state with the workflow or job record, so a process restart does not forget pending work.
- Include an attempt or generation number in the call ID, so a late response from a timed-out attempt cannot settle a replacement attempt.
- Record failures as structured events, including the workflow ID, tool name, call ID, and the original error object.
There is also a deliberate non-goal: this checker does not decide whether a tool result is semantically correct. That needs tool-specific validation. A database lookup may require schema validation; a web lookup may need citations and freshness checks. Mixing those policies into the trace layer makes both harder to reason about.
A seven-minute rehearsal that does not sound rehearsed
Use a real project, not a fictional shopping assistant.
- Pick one agent action that calls two or three tools.
- Write a five-event trace with one valid path.
- Introduce one fault: an unknown tool, a missing result, or a late duplicate.
- Run the checker and read the error aloud.
- Explain what you would persist, what you would alert on, and what you would retry.
That gives you a clean answer to the usual follow-up: “What happens when the first tool succeeds but the second one never returns?” You can say that completion is withheld, the pending call is visible, and a retry policy must create a new attempt rather than quietly reusing an ambiguous result.
For question practice around AI-engineering work, aceround.app's AI engineer interview guide is useful as a prompt source. The preparation still works best when you can point to a small artifact and explain its limits in your own words.
What should you add next?
If you turn this into a real service, add a timeout event rather than letting a call vanish. Then make the timeout decision explicit: retry only idempotent operations, surface a failed run to the user, or hand it to a human. The correct choice depends on the tool's side effect, not on whether the model sounds confident.
The important habit is simple: treat an agent trace as data with contracts. That turns “I built an AI agent” into a discussion about state, failure modes, and evidence.
Sources
Disclosure: AI was used to help draft and edit this article. The code, technical claims, and links were reviewed before publication.
Top comments (0)