I Wrote the Same Tool-Calling Loop Three Times, Once Per Provider, Before I Stopped

The job was an internal ops assistant. Someone asks "is order 4521 still on hold and what's the customer's plan tier?", and answering it means hitting two of our own services first - the orders API and the billing API - then coming back with a short structured summary my UI could render. Not a chatbot. The final answer had to be { status, tier, summary }, typed, every time, because a component was rendering it.

The lookups were the easy part; they're just two functions I already had. What I needed was for the model to decide which of them to call, with what arguments, see what came back, and then produce that structured answer. And I needed the arguments it invented to be trustworthy before they reached functions that hit real internal services.

The loop I wrote, then wrote again

OpenAI's tool-calling API is genuinely pleasant, so version one took an afternoon: pass tools, read message.tool_calls, JSON.parse the arguments, dispatch to the right function, push a role: "tool" message back, loop until the model stops asking. Maybe 60 lines with the turn guard.

Then I wanted Anthropic for the summarizing step, because it was better at it. Anthropic's wire format is not OpenAI's. Tool requests arrive as tool_use content blocks inside the message rather than a parallel tool_calls array, and results go back as tool_result blocks in a user message, not a dedicated tool role. So the dispatch loop got rewritten - same logic, different shape, second copy.

Then, because some of this had to run against a local model for a cost experiment, Ollama. Different again. Third copy.

Three loops, all doing the identical thing, each subtly wrong in its own way for a while. And a switch on provider name sitting in the middle of my application code, which is exactly the kind of thing I don't want to own.

The bug that actually worried me

Somewhere in copy two, a model called my order lookup with { orderId: 4521 } - a number, where every other time it had sent the string "4521". My handler did orderId.trim() and the whole request died with a TypeError from inside a function that had no business receiving a number in the first place.

That's when the real problem got clearer. It wasn't the three loops. It was that JSON.parse succeeding is not the same as the arguments being right, and I had nothing between the model's invention and a function that talks to a production service. I was one hallucinated field away from calling an internal API with garbage.

What I was already importing

I was using shapecraft for the structured-output part of this project already, so the fact that generateWithTools() was sitting in the same package was mildly embarrassing to discover this late:

import { generateWithTools, anthropic } from "@aviasole/shapecraft";
import type { ToolDefinition } from "@aviasole/shapecraft";
import { z } from "zod";

const lookupOrder: ToolDefinition = {
  name: "lookup_order",
  description: "Look up an order's current status by its ID",
  parameters: z.object({ orderId: z.string() }),
  handler: async ({ orderId }: { orderId: string }) => ordersApi.get(orderId),
};

const lookupBilling: ToolDefinition = {
  name: "lookup_billing",
  description: "Look up a customer's billing plan tier",
  parameters: z.object({ customerId: z.string() }),
  handler: async ({ customerId }: { customerId: string }) => billingApi.tier(customerId),
};

const result = await generateWithTools(
  anthropic({ model: "claude-haiku-4-5-20251001" }),
  [lookupOrder, lookupBilling],
  z.object({ status: z.string(), tier: z.string(), summary: z.string() }),
  "Is order 4521 still on hold, and what's that customer's plan tier?"
);

console.log(result.data);      // { status: "on_hold", tier: "enterprise", summary: "..." }
console.log(result.toolCalls); // every call it made, in order, with results

This uses each provider's native tool API underneath - it isn't a prompt convention pretending to be tool calling. The OpenAI-wire-format backends share one implementation, and Anthropic and Ollama get their own shapes normalized behind the same interface. Which means the three loops collapse into one call, and swapping anthropic(...) for openai(...) is a one-line edit rather than a rewrite.

The part I actually cared about: parameters is the same schema machinery as everywhere else in the library, and arguments are validated against it before the handler runs. My { orderId: 4521 } case now fails that check, gets handed back to the model as an error it can see, and the model corrects itself. It never reaches ordersApi.get().

Where my first attempt actually broke

I had lookupOrder throw on a missing order, because that's what the function already did:

handler: async ({ orderId }) => {
  const order = await ordersApi.get(orderId);
  if (!order) throw new Error(`No order ${orderId}`);  // don't
  return order;
},

First time the model guessed a wrong order number, the entire call died with ToolExecutionError. No retry, no recovery.

That's deliberate, and once I read why, I agreed with it: a handler throwing means my code failed, and re-prompting a model cannot fix a broken function. So it aborts immediately rather than looping. Bad arguments are the recoverable case, because the model can fix those - and those it does feed back.

The catch is that "order not found" isn't really my code failing. It's a legitimate result the model should get to react to. So it belongs in the return value, not in a throw:

handler: async ({ orderId }) => {
  const order = await ordersApi.get(orderId);
  return order ?? { error: `No order ${orderId}` };  // model sees this and adapts
},

Now a wrong guess comes back as data, the model tries a different ID or says it couldn't find one, and the loop carries on. Throw for "my database is down", return for "no such record." That distinction is worth getting right on day one instead of day three.

What this doesn't promise

Tool selection isn't validated, and can't be. Nothing checks the model picked the sensible tool, or that it should have called one at all - that's model behaviour, and it's the same structural-vs-semantic gap that applies to every schema-validated call. Arguments are guaranteed well-formed; the judgment to use them isn't.

Handler correctness is entirely mine. The library guarantees my return value gets fed back to the model as-is, and nothing about whether it's the right value.

There's a turn cap, defaulting to 10, and a model that keeps requesting tools forever hits MaxToolTurnsExceededError rather than spinning. Blunt, but I'd rather have a blunt guard than a runaway bill.

And it's non-streaming in v1, which for my use case is fine - I'm rendering a structured object into a component, not streaming prose at anyone. It's also on every cloud backend but not local GGUF models via llamaCpp(), which simply have no tools API to call. There's a model.capabilities.toolCalling flag to check rather than memorizing that.

Where that leaves it

Three provider-specific loops and a switch statement became one function call and two plain { name, parameters, handler } objects. Roughly 180 lines of dispatch plumbing deleted, and the thing I was actually nervous about - a hallucinated argument reaching a service that mutates real orders - is now a validation failure the model gets to correct instead of a TypeError in production.

import { generateWithTools } from "@aviasole/shapecraft";

Repo's at github.com/aviasoletechnologies/shapecraft, package is @aviasole/shapecraft on npm.