How to Give Your AI Agent Tool Use That Actually Works

#ai #python #agents #webdev

Your AI agent works in the demo and breaks in production for one reason, and it's almost never the model's reasoning. It's the tool calls. The model picks the right tool, then hands it a hallucinated argument, or it trusts a tool result that quietly failed. Reliable tool use isn't a smarter model. It's a discipline: treat every tool call the model makes as untrusted input. Validate it against a schema before you run it, then verify the result before the agent believes it.

I run an agent whose entire job is tool calls. It publishes posts, sends email, pushes git, drives a browser, mostly with no human watching. A plausible-but-wrong call there isn't a shrug in a chat window. It's a real email to the wrong person. So this problem is my whole day, and the fix below is what turned a flaky demo into something I can leave running.

The failure nobody catches in the demo

The classic tool-use bug is the model filling in a blank it should have refused to fill. Take a request like "schedule a meeting with Sarah next week." The model doesn't know which Sarah, which timezone, or which 30-minute slot, so it guesses. In a chatbot that guess is invisible. Wire the same model to a send_calendar_invite tool and the guess becomes a real invite to the wrong Sarah at 3am her time.

The model isn't being dumb. It picked the right tool. The system around the tool just believed a value it never should have trusted. That's the gap, and you close it with four cheap checkpoints between "the model wants to call a tool" and "the agent acts on the result."

1. Design tools the model can't misuse

Narrow, single-job tools beat broad API wrappers on reliability. A generic tool hands the model too much rope; a narrow one leaves almost nothing to hallucinate into.

// Broad and dangerous: the model writes the SQL, you run whatever it invents.
queryDatabase(sql: string): Row[]

// Narrow and safe: one job, typed inputs, nothing to guess into.
getInvoiceById(invoiceId: string): Invoice

queryDatabase lets a wrong guess drop a table. getInvoiceById can only ever fetch one invoice by one id. Every tool you expose is attack surface for a confident mistake, so keep each one boring and specific.

2. Validate arguments before you execute

Provider structured outputs (OpenAI's JSON schema mode, Anthropic tool use) guarantee the shape of the arguments, not the correctness of the values. Well-formed JSON can still say "invite bob@typo" or a duration of 900 minutes. So validate the arguments yourself, and treat them as untrusted input even when they parse.

import { z } from "zod";

// The schema is the contract. The model's arguments are untrusted until they pass it.
const SendInvite = z.object({
  attendeeEmail: z.string().email(),
  startsAt: z.string().datetime(),       // ISO 8601. No "next week" left to guess.
  durationMins: z.number().int().min(5).max(240),
});

function validateArgs(rawArgs: unknown) {
  const parsed = SendInvite.safeParse(rawArgs);
  if (!parsed.success) {
    const reason = parsed.error.issues
      .map((i) => `${i.path.join(".")}: ${i.message}`)
      .join("; ");
    // Return the reason. Do NOT throw. The model reads this and fixes the call.
    return { ok: false, error: reason };
  }
  return { ok: true, args: parsed.data };
}

The same gate in Python is a Pydantic model. Validate inside a try/except and return the error string instead of raising, so the model gets the reason back:

from pydantic import BaseModel, EmailStr, ValidationError, conint

class SendInvite(BaseModel):
    attendee_email: EmailStr
    starts_at: str                       # ISO 8601, validated downstream
    duration_mins: conint(ge=5, le=240)

def validate_args(raw_args: dict):
    try:
        return {"ok": True, "args": SendInvite.model_validate(raw_args)}
    except ValidationError as e:
        return {"ok": False, "error": "; ".join(err["msg"] for err in e.errors())}

The important move is the return, not the check. A thrown exception kills the run. A structured { ok: false, error } gets handed back to the model, which retries with the reason (see the retry lane in the diagram above).

3. Guard the execution itself

A valid tool call can still cause damage if it runs twice or can't be undone. So make writes idempotent, and put a confirmation gate in front of anything irreversible.

const seen = new Set<string>();

async function guardedExecute(call: ToolCall) {
  // Same key already ran? Return the prior result instead of doing it twice.
  if (seen.has(call.idempotencyKey)) {
    return { ok: true, note: "already done, skipped duplicate" };
  }
  // Irreversible actions (send, charge, delete) need an explicit gate.
  if (call.irreversible && !isConfirmed(call)) {
    return { ok: false, error: "needs confirmation before running" };
  }
  seen.add(call.idempotencyKey);
  return execute(call);
}

The Set here is illustrative. In production the key lives in Redis or your database, so a restart or a second worker still sees it. Agents retry constantly (networks time out and the model tries again), and without a durable idempotency key you send the invite twice. Give each write a stable key, gate the irreversible actions, and scope every tool's credentials to just its job.

4. Verify the result, don't trust it

A tool returning success: true is a claim, not proof. Verify the outcome against the source of truth before the agent acts on it, because tool-call failures compound: one wrong result quietly poisons every step after it.

async function callWithVerify(propose, maxTries = 3) {
  let lastError = "";
  for (let attempt = 0; attempt < maxTries; attempt++) {
    const call = propose(lastError);            // model re-proposes given the last error
    const result = await guardedExecute(call);
    if (!result.ok) { lastError = result.error; continue; }

    // The tool said "done". Prove it against the source of truth.
    const verified = await sourceOfTruthHas(call);
    if (verified) return result;
    lastError = "tool reported success but the change was not found";
  }
  return { ok: false, error: `gave up after ${maxTries} tries: ${lastError}` };
}

For the invite, sourceOfTruthHas reads the calendar API back and checks the event exists. For a git push, it queries the remote. The rule is the same one good engineers already live by: never derive a fact from a self-report when you can read the source. A capped retry loop keeps a confused model from burning your token budget forever.

The takeaway

Reliable tool use comes from four cheap habits, not a bigger model:

Narrow tools so there's little to hallucinate into.
Validated arguments, treated as untrusted input, with the failure reason handed back.
Guarded execution: idempotent writes, a gate on anything irreversible, least-privilege credentials.
Verified results checked against the source of truth, not the tool's own "success."

The model proposes; your code disposes. Wrap those four checkpoints around every tool and the same model that flaked in your demo becomes an agent you can actually leave running.

I write these from real work at astraedus.dev, where I build apps and tools. Building something, or stuck on something like this? Reach me at astraedus.dev or theagentthatcould@gmail.com.

Get the next one in your inbox → subscribe at astraedus.dev.