The Tool-Call Argument Your Model Gets Wrong — and the Validator That Catches It

#typescript #zod #ai #node

Book: AI That Acts
The series: AI in TypeScript — 5 books, from your first LLM call to agents in production — all five here
My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
Me: xgabriel.com | GitHub

Your tool takes a booking id and a date. The model calls it with
bookingId: "the one from earlier" and date: "next Tuesday".

Zod is happy. Both are strings, both are present, the shape is
exactly what the schema declared. The call sails through validation
and fails inside your handler, where the error message is a
database miss and the model is told "not found" — which tells it
nothing about what it did wrong.

Shape is not the same as sense. Most tool-calling code validates the
first and discovers the second by crashing.

Two categories, two owners

Shape errors are structural. Wrong type, missing field, enum
value not in the list. Zod owns these and catches them completely.

Semantic errors are about meaning against your data and your
rules. The id is well-formed and refers to nothing. The date parses
and is in the past. The filter is syntactically fine and would return
the entire table. Zod cannot see any of it — these depend on state it
has no access to.

The distinction matters because they need different responses. A
shape error means the model misread the schema; restate the schema.
A semantic error means the model made a reasonable-looking mistake
about the world; tell it what the world actually contains.

The five that recur

The referential placeholder. "the one from earlier",
"the previous booking", "<id>". The model is referring to
conversational context rather than supplying a value. Common when the
schema says bookingId: z.string() with no format constraint.

The natural-language date. "next Tuesday", "in two weeks",
"end of month". z.string() accepts them. So does
z.coerce.date() — it produces Invalid Date, which is a Date,
and instanceof Date is true.

The near-miss enum. Handled by z.enum, unless the field is a
free string because the values come from your database.

The over-broad filter. { status: undefined, limit: undefined }
meaning "everything". Valid, and a full table scan returned into a
context window.

The plausible non-existent id. "bk_00000000" — right prefix,
right length, no such row. This one is indistinguishable from a valid
id without a lookup.

Encode what you can in the schema

Before writing a validation layer, push everything expressible into
Zod. Constraints there are enforced at generation time too, if you
generate your tool's JSON Schema from it.

const BookingId = z.string().regex(
  /^bk_[0-9a-f]{12}$/,
  "Booking id like 'bk_a1b2c3d4e5f6'. Do not invent one; " +
  "use an id returned by search_bookings.",
);

const Reschedule = z.object({
  bookingId: BookingId,
  date: z.string().date().describe("ISO 8601 date, e.g. 2026-08-14"),
  limit: z.number().int().min(1).max(50).default(20),
});

The regex kills the placeholder outright. z.string().date() rejects
"next Tuesday" where z.coerce.date() would have produced
Invalid Date. .max(50) makes the over-broad filter unexpressible.

The error message inside .regex() is written for the model. It says
what a valid value looks like and where to get one, which turns a
rejection into an instruction.

The semantic layer

What remains needs your data.

export type Semantic<T> =
  | { ok: true; value: T }
  | { ok: false; message: string; retryable: boolean };

export async function checkReschedule(
  args: z.infer<typeof Reschedule>,
  ctx: Ctx,
): Promise<Semantic<Resolved>> {
  const booking = await ctx.db.booking.find(args.bookingId);
  if (!booking) {
    const recent = await ctx.db.booking.recentFor(ctx.userId, 3);
    return {
      ok: false,
      retryable: true,
      message:
        `No booking ${args.bookingId}. ` +
        (recent.length
          ? `This user's bookings: ${recent.map((b) => b.id).join(", ")}.`
          : "This user has no bookings."),
    };
  }

  if (booking.userId !== ctx.userId) {
    return {
      ok: false,
      retryable: false,
      message: `Booking ${args.bookingId} is not available.`,
    };
  }

  const date = Temporal.PlainDate.from(args.date);
  if (Temporal.PlainDate.compare(date, ctx.today) < 0) {
    return {
      ok: false,
      retryable: true,
      message: `${args.date} is in the past. Today is ${ctx.today}.`,
    };
  }

  return { ok: true, value: { booking, date } };
}

Three things this does that a thrown error does not.

It returns candidates. Listing the user's actual booking ids
turns "not found" into something the model can act on in one step
rather than three.

It distinguishes retryable from not. A past date is a mistake
worth correcting. Another user's booking is not — and the message
deliberately does not confirm the booking exists, because leaking
existence across users through an error message is a real
information disclosure.

It supplies missing context. Today is 2026-08-06 is the fact
the model lacked. Without it, it will guess again and guess similarly.

Wiring both into the loop

const parsed = t.schema.safeParse(block.input);
if (!parsed.success) {
  results.push(errorResult(block.id, formatZod(parsed.error)));
  continue;
}

const checked = await t.check(parsed.data, ctx);
if (!checked.ok) {
  if (!checked.retryable) rejections.add(block.name);
  results.push(errorResult(block.id, checked.message));
  continue;
}

const out = await t.run(checked.value, ctx);

Note that run receives checked.value, not parsed.data. The
semantic check already fetched the booking; passing the resolved
object through means the handler does not look it up again, and
cannot look it up differently.

That is the quiet benefit of parsing rather than validating — the
check produces the thing you need, so there is no second lookup that
could disagree with the first.

Error messages are prompt engineering

Whatever you put in is_error: true goes into context and steers the
next turn. It is prompt text with an unusual author.

// tells the model nothing
"Invalid input"
"Error: ECONNREFUSED"
"ValidationError at $.date"

// tells the model what to do next
"date must be ISO 8601 (2026-08-14). You sent 'next Tuesday'."
"No booking bk_zzz. Available: bk_a1b2c3, bk_d4e5f6."
"limit must be 1-50. You sent 500; use pagination instead."

Every good one has the same shape: what was wrong, what valid looks
like, and where to get a valid value. Same rules as a good API error
for a human developer — models respond to them for the same reason.

Stop the correction loop

A model that cannot satisfy a check will try again. And again.

const attempts = new Map<string, number>();

const key = `${block.name}:${hash(block.input)}`;
attempts.set(key, (attempts.get(key) ?? 0) + 1);

if (attempts.get(key)! > 2) {
  results.push(errorResult(
    block.id,
    "This call has failed repeatedly. Ask the user for the " +
    "booking id rather than retrying.",
  ));
  continue;
}

Keying on the arguments as well as the name is what makes this
useful: retrying with corrected arguments should be allowed, and
repeating the identical failing call should not. And the message
tells the model what to do instead of failing, which is what breaks
the loop rather than just capping it.

The sentence to keep

Zod tells you the arguments are well-formed. Only your data can
tell you they are right. Build the second check, return its result
as a message the model can act on, and most argument errors resolve
in one extra turn instead of surfacing as a failed request.

If this was useful

AI That Acts covers the tool
boundary in depth — schemas that prevent whole error classes, the
semantic layer, error results as a correction channel, and the guards
that keep a retry loop from becoming a bill.