Force structured outputs and make your LLM calls testable

Free text is a test you can never write

Here is a question that used to stump me. How do you write a CI test for an LLM call that returns a paragraph of English?

You cannot. Not really. You can check that the string is non empty. You can grep for a keyword and pray. But the model can rephrase the same answer a thousand ways, and every rephrasing breaks a naive assertion while being perfectly correct. So people give up and ship prose, and then the call quietly drifts over weeks, and nobody notices until a downstream parser chokes in production.

The fix is to stop asking for prose. Ask for a shape.

If the model must return data that fits a schema, the call stops being a vibe and becomes a function with a contract. And a function with a contract is something you can assert on, in CI, on every commit, like any other code.

From a wish to a schema

Say I want to extract action items from a meeting transcript. The lazy version asks "summarize the action items." What comes back is a bulleted list, or a numbered list, or a chatty paragraph, depending on the model's mood that afternoon. Good luck parsing that reliably.

Now I define what I actually want.

{
  "type": "object",
  "properties": {
    "action_items": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "task": { "type": "string" },
          "owner": { "type": "string" },
          "due": { "type": ["string", "null"], "format": "date" },
          "priority": { "type": "string", "enum": ["low", "medium", "high"] }
        },
        "required": ["task", "owner", "priority"]
      }
    }
  },
  "required": ["action_items"]
}

Most current model APIs let you pass a schema like this, either as a structured output format or through function calling, where the "function" is just the shape you want back. The mechanism has different names across vendors. The idea is the same. You hand over the schema, the provider constrains generation so the response has to fit it, and you get JSON that matches or an error you can catch.

Look at what this bought me. priority can only ever be one of three strings, so a typo like "urgent" cannot slip through. owner is required, so an action item with no assignee fails loudly instead of hiding. due is a date or explicitly null, not the phrase "sometime next week." The schema encodes the rules that used to live in my head.

What breaks without it

Let me be concrete about the failure I used to eat.

Without a schema, the model returns something like "priority": "High" one day and "importance": "high" the next, and "pri": 2 the day after. My downstream code did item["priority"].lower(). It threw a KeyError in production on a transcript that happened to trigger the third variant. No test caught it because my test used a fixed transcript that happened to produce the first variant.

That is the quiet killer. Free text output means your test only proves the model behaved once, on one input, on the day you wrote it. The model can drift under you after a version bump, and your green CI tells you nothing, because the assertion was too loose to notice.

A schema flips this. Now I can write tests that mean something.

def test_extract_returns_valid_shape():
    result = extract_action_items(SAMPLE_TRANSCRIPT)
    jsonschema.validate(result, ACTION_ITEM_SCHEMA)  # raises on mismatch

    for item in result["action_items"]:
        assert item["priority"] in {"low", "medium", "high"}
        assert item["task"].strip()

This test fails the moment the output shape drifts, on any input, on any model version. That is a real regression signal. I run it in CI against a few representative transcripts and I sleep better.

Validate, then retry on mismatch

Constrained generation is good but not a guarantee. Weird inputs, token limits, and edge cases still produce the occasional response that does not fit. So I never trust the output blindly. I validate, and if it fails, I retry with the error fed back in.

def extract_with_retry(transcript, max_tries=3):
    last_error = None
    for attempt in range(max_tries):
        raw = call_model(transcript, schema=ACTION_ITEM_SCHEMA,
                         prior_error=last_error)
        try:
            data = json.loads(raw)
            jsonschema.validate(data, ACTION_ITEM_SCHEMA)
            return data
        except (json.JSONDecodeError, jsonschema.ValidationError) as e:
            last_error = str(e)
    raise ValueError(f"Schema never satisfied after {max_tries} tries: {last_error}")

The trick is passing prior_error back into the next call. The model gets to see exactly why its last attempt was rejected ("priority must be one of low, medium, high") and usually fixes it on the second pass. Three strikes and I raise, because an infinite retry loop is its own outage.

None of this is tied to a particular provider or framework. Define the schema. Ask the model to fill it. Validate the result against the same schema. Retry with the error on failure, and cap the retries. The whole approach is portable because it lives in your code, not in a vendor's SDK.

Free text answers are a promise you can never verify. A schema is a contract you can test. Once your LLM calls return shapes instead of paragraphs, they slot into your test suite like any other function, and the drift that used to bite you in production shows up as a red build instead.

AGINE Academy is an independent product by AGINE AI (not affiliated with Anthropic). We teach building with Claude by doing the work, not watching lectures.