My Tool-Calling Loop Worked Fine, Until Compliance Wanted a Second Model to Check It

Small ask, on paper. A clinician types something like "any allergy conflicts for this patient's current meds?", and before the model answers it needs to actually go get the medication list and the allergy list rather than guess at what's plausible. Two functions, both of which already existed. The interesting part was never the lookups, it was getting a model to decide when to call them and hand back arguments I could trust.

Then compliance sat in on the review and asked the question I should have seen coming: "what checks this model's answer?" Fair question, this is going in front of a clinician. Their answer was a second model, from a different provider, running the same lookups independently and flagging if it disagreed. Reasonable. Also, as it turned out, the thing that broke my code.

The first version worked. That was the problem.

I had the OpenAI SDK already wired into this project, so version one was straightforward: define tools, send the request, read message.tool_calls, run whichever function it asked for, push a role: "tool" message back with the result, loop until it stops asking. Twenty minutes, maybe. It worked on the first real test and I remember thinking this was going to be a short ticket.

It was a short ticket, right up until "second model, different provider" landed in the same sprint. I went to point the exact same loop at Claude and it just doesn't speak that dialect - Anthropic sends tool requests back as tool_use blocks sitting inside the message content, not a separate tool_calls array, and the result has to go back as a tool_result block inside a user message. There's no tool role at all on their side. Same idea, completely different shape, and I was about two minutes from just writing a second version of the loop and calling it a day.

I'd already installed the thing that fixes this

I only stopped because I already had @aviasole/shapecraft in this project for the FHIR schema work, and figured it was worth thirty seconds to check whether generateWithTools() covered this too before I wrote loop number two:

import { generateWithTools, openai, anthropic } from "@aviasole/shapecraft";
import type { ToolDefinition } from "@aviasole/shapecraft";
import { z } from "zod";

const lookupMedications: ToolDefinition = {
  name: "lookup_medications",
  description: "Get the patient's current active medication list",
  parameters: z.object({ patientId: z.string() }),
  handler: async ({ patientId }: { patientId: string }) => ehr.getMedications(patientId),
};

const lookupAllergies: ToolDefinition = {
  name: "lookup_allergies",
  description: "Get the patient's recorded allergies",
  parameters: z.object({ patientId: z.string() }),
  handler: async ({ patientId }: { patientId: string }) => ehr.getAllergies(patientId),
};

const NoteSchema = z.object({
  conflicts: z.array(z.string()),
  recommendation: z.string(),
});

const draft = await generateWithTools(
  openai({ model: "gpt-4o-mini" }),
  [lookupMedications, lookupAllergies],
  NoteSchema,
  "Any allergy conflicts for patient 8823's current meds?"
);

const check = await generateWithTools(
  anthropic({ model: "claude-haiku-4-5-20251001" }),
  [lookupMedications, lookupAllergies],
  NoteSchema,
  "Any allergy conflicts for patient 8823's current meds?"
);

Same two tools, same call, both providers - the only line that changes between draft and check is which model constructor gets passed in. No tool_use blocks to unpack, no manually tracking a tool_call_id so the result lands back in the right place. That's the whole rewrite I was bracing for, gone.

The bug I blamed on the library, before I actually read why it was there

Not every test patient has an allergy on file, which is realistic and also exactly the case I hadn't tested:

handler: async ({ patientId }) => {
  const allergies = await ehr.getAllergies(patientId);
  if (!allergies) throw new Error(`No allergy record for ${patientId}`);
  return allergies;
},

First patient with no record, the whole request died with ToolExecutionError instead of the model just saying "no known allergies on file," which is a completely normal thing for a note to say. My first reaction, not proud of this, was to assume the library had a rough edge around error handling. It didn't. A handler throwing means my code broke in a way a retry can't fix, so it bails immediately rather than pretending otherwise. Bad arguments get looped back to the model because the model caused those and can fix them. A patient with no allergy record isn't my code failing, it's just an answer, so it never belonged in a throw:

handler: async ({ patientId }) => {
  const allergies = await ehr.getAllergies(patientId);
  return allergies ?? { note: `No allergy record on file for ${patientId}` };
},

One-line fix once I stopped being annoyed at the library and actually read the code. Both models now say "no known allergies on file" for that patient instead of my server handing a clinician a 500.

Two things worth knowing before you assume more than you get

The model still decides whether to call a tool at all and which one - no schema can check that, only the shape of the arguments once it's decided. And there's a turn cap, 10 by default, so a model that gets stuck re-asking for the same lookup fails loudly with MaxToolTurnsExceededError instead of quietly burning through your API budget while nobody's watching. Neither one bit me here, but I'd rather know that going in than find out from an invoice.

Where that leaves it

Two tool definitions, written once, running unmodified in front of two different providers, and a bug that turned out to be mine for treating "no record found" as an exception instead of an answer. Kudos to shapecraft for that one, again - I'd already reached for it once for the FHIR side of this same project and honestly expected to write the second tool loop by hand anyway.

If you're about to hand-roll a tool-calling loop for a second provider, check generateWithTools() in @aviasole/shapecraft first. It might already be sitting there.