DEV Community: deep patel

My Tool-Calling Loop Worked Fine, Until Compliance Wanted a Second Model to Check It

deep patel — Wed, 05 Aug 2026 12:51:50 +0000

Small ask, on paper. A clinician types something like "any allergy conflicts for this patient's current meds?", and before the model answers it needs to actually go get the medication list and the allergy list rather than guess at what's plausible. Two functions, both of which already existed. The interesting part was never the lookups, it was getting a model to decide when to call them and hand back arguments I could trust.

Then compliance sat in on the review and asked the question I should have seen coming: "what checks this model's answer?" Fair question, this is going in front of a clinician. Their answer was a second model, from a different provider, running the same lookups independently and flagging if it disagreed. Reasonable. Also, as it turned out, the thing that broke my code.

The first version worked. That was the problem.

I had the OpenAI SDK already wired into this project, so version one was straightforward: define tools, send the request, read message.tool_calls, run whichever function it asked for, push a role: "tool" message back with the result, loop until it stops asking. Twenty minutes, maybe. It worked on the first real test and I remember thinking this was going to be a short ticket.

It was a short ticket, right up until "second model, different provider" landed in the same sprint. I went to point the exact same loop at Claude and it just doesn't speak that dialect - Anthropic sends tool requests back as tool_use blocks sitting inside the message content, not a separate tool_calls array, and the result has to go back as a tool_result block inside a user message. There's no tool role at all on their side. Same idea, completely different shape, and I was about two minutes from just writing a second version of the loop and calling it a day.

I'd already installed the thing that fixes this

I only stopped because I already had @aviasole/shapecraft in this project for the FHIR schema work, and figured it was worth thirty seconds to check whether generateWithTools() covered this too before I wrote loop number two:

import { generateWithTools, openai, anthropic } from "@aviasole/shapecraft";
import type { ToolDefinition } from "@aviasole/shapecraft";
import { z } from "zod";

const lookupMedications: ToolDefinition = {
  name: "lookup_medications",
  description: "Get the patient's current active medication list",
  parameters: z.object({ patientId: z.string() }),
  handler: async ({ patientId }: { patientId: string }) => ehr.getMedications(patientId),
};

const lookupAllergies: ToolDefinition = {
  name: "lookup_allergies",
  description: "Get the patient's recorded allergies",
  parameters: z.object({ patientId: z.string() }),
  handler: async ({ patientId }: { patientId: string }) => ehr.getAllergies(patientId),
};

const NoteSchema = z.object({
  conflicts: z.array(z.string()),
  recommendation: z.string(),
});

const draft = await generateWithTools(
  openai({ model: "gpt-4o-mini" }),
  [lookupMedications, lookupAllergies],
  NoteSchema,
  "Any allergy conflicts for patient 8823's current meds?"
);

const check = await generateWithTools(
  anthropic({ model: "claude-haiku-4-5-20251001" }),
  [lookupMedications, lookupAllergies],
  NoteSchema,
  "Any allergy conflicts for patient 8823's current meds?"
);

Same two tools, same call, both providers - the only line that changes between draft and check is which model constructor gets passed in. No tool_use blocks to unpack, no manually tracking a tool_call_id so the result lands back in the right place. That's the whole rewrite I was bracing for, gone.

The bug I blamed on the library, before I actually read why it was there

Not every test patient has an allergy on file, which is realistic and also exactly the case I hadn't tested:

handler: async ({ patientId }) => {
  const allergies = await ehr.getAllergies(patientId);
  if (!allergies) throw new Error(`No allergy record for ${patientId}`);
  return allergies;
},

First patient with no record, the whole request died with ToolExecutionError instead of the model just saying "no known allergies on file," which is a completely normal thing for a note to say. My first reaction, not proud of this, was to assume the library had a rough edge around error handling. It didn't. A handler throwing means my code broke in a way a retry can't fix, so it bails immediately rather than pretending otherwise. Bad arguments get looped back to the model because the model caused those and can fix them. A patient with no allergy record isn't my code failing, it's just an answer, so it never belonged in a throw:

handler: async ({ patientId }) => {
  const allergies = await ehr.getAllergies(patientId);
  return allergies ?? { note: `No allergy record on file for ${patientId}` };
},

One-line fix once I stopped being annoyed at the library and actually read the code. Both models now say "no known allergies on file" for that patient instead of my server handing a clinician a 500.

Two things worth knowing before you assume more than you get

The model still decides whether to call a tool at all and which one - no schema can check that, only the shape of the arguments once it's decided. And there's a turn cap, 10 by default, so a model that gets stuck re-asking for the same lookup fails loudly with MaxToolTurnsExceededError instead of quietly burning through your API budget while nobody's watching. Neither one bit me here, but I'd rather know that going in than find out from an invoice.

Where that leaves it

Two tool definitions, written once, running unmodified in front of two different providers, and a bug that turned out to be mine for treating "no record found" as an exception instead of an answer. Kudos to shapecraft for that one, again - I'd already reached for it once for the FHIR side of this same project and honestly expected to write the second tool loop by hand anyway.

If you're about to hand-roll a tool-calling loop for a second provider, check generateWithTools() in @aviasole/shapecraft first. It might already be sitting there.

I Almost Hand-Wrote a FHIR Schema. Then I Found Out I Didn't Have To.

deep patel — Wed, 15 Jul 2026 12:45:39 +0000

A hospital-network client wanted our system to output patient data in actual FHIR format - the standard interoperability format healthcare systems use to talk to each other - instead of whatever shape we felt like inventing. Made total sense from their side, their EHR software only accepts FHIR resources, not our custom JSON. From my side, it meant I now had to get an LLM to produce a Patient resource that was FHIR R4 compliant, field for field.

I opened the FHIR R4 spec page for Patient to see what I was dealing with. Closed the tab about four minutes later. It's not one flat object - names have their own nested structure with use/family/given arrays, telecom is a list of typed contact points, addresses have their own multi-field shape, and half the fields have specific allowed value sets straight out of a separate FHIR terminology spec. This was not going to be a quick z.object({...}).

Two days into hand-writing it, and I wasn't even done

I started anyway, because what else was I going to do:

const PatientSchema = z.object({
  resourceType: z.literal("Patient"),
  identifier: z.array(
    z.object({
      system: z.string(),
      value: z.string(),
    })
  ),
  name: z.array(
    z.object({
      use: z.enum(["official", "usual", "nickname", "maiden"]),
      family: z.string(),
      given: z.array(z.string()),
    })
  ),
  telecom: z.array(
    z.object({
      system: z.enum(["phone", "email", "fax"]),
      value: z.string(),
      use: z.enum(["home", "work", "mobile"]).optional(),
    })
  ),
  gender: z.enum(["male", "female", "other", "unknown"]),
  birthDate: z.string(),
  address: z.array(
    z.object({
      use: z.enum(["home", "work", "temp"]).optional(),
      line: z.array(z.string()),
      city: z.string(),
      state: z.string(),
      postalCode: z.string(),
      country: z.string(),
    })
  ),
  // ...and I still hadn't gotten to maritalStatus, communication,
  // contact, generalPractitioner, managingOrganization...
});

Two days in and I still wasn't done, and worse, I had no real confidence I'd gotten the parts I had written correct. I kept cross-referencing the spec for things like whether gender really only allows those four values or whether I was missing an unknown case, whether telecom.system needed a url variant too, whether I'd nested name right. This was medical data going to an actual hospital system - if I got a field wrong and it silently passed validation because I'd defined it wrong in the first place, that's not a bug someone catches in a code review, that's a bad record in someone's patient chart. That thought alone was enough to make me stop and look for literally any other option before shipping my own guesswork as the source of truth.

Turned out I didn't have to write it at all

I was already using shapecraft for the extraction itself, so on a hunch I checked whether anyone had already solved the "FHIR schema in TypeScript" problem as a package rather than a spec page. Shapecraft ships presets for exactly this:

import { PatientSchema } from "@aviasole/shapecraft/fhir";
import { generate, openai } from "@aviasole/shapecraft";

const result = await generate(
  openai({ model: "gpt-4o-mini" }),
  PatientSchema,
  patientIntakeText
);

// result.data is a typed, validated FHIR R4 Patient resource
console.log(result.data.name[0].family);
console.log(result.data.telecom[0].system);

Same generate() call I already knew from using it elsewhere, same retry/timeout/guaranteeLevel behavior - I didn't have to learn a new API, just point it at a different schema. The nested name/telecom/address structure I'd been hand-rolling badly for two days was already there, already correct, already typed. Observation, Condition, MedicationRequest, and Encounter were sitting right next to it too, which meant when the client's second request came in ("can you also send us the medication list in FHIR") it was a five-minute change instead of another two-day spec-reading marathon.

Where that left me

Deleted my hand-rolled PatientSchema entirely - it wasn't even fully correct, I found out later while diffing it against the preset, I'd missed that telecom.system also allows a url/sms/pager variant I'd never gotten to in the spec. Two days of careful, cautious work, still wrong in a spot I hadn't reached yet.

What actually shipped was a preset import and the same generate() call I'd have written anyway. Kudos to shapecraft for that one - I went from "I might get medical data wrong because I misread a spec doc" to "the schema was already someone else's solved problem" in about ten minutes of searching plus a five-minute swap.

If you're staring down a FHIR resource and reaching for a blank Zod schema, check @aviasole/shapecraft/fhir first. It might already be sitting there.