A strict schema gives the model nowhere to say I do not know, and that costs you the errors you most need to see.

When you force a model into required fields and tight enums, you have not removed hallucination. You have removed the model's ability to admit it, so it fills the blank with something plausible instead.

We tightened a classification schema last spring and watched our error rate look better while our incidents got worse. The schema went from loose (a free-text answer field) to strict (a required category enum with eleven allowed values, no nulls). Validation-failure rate dropped to near zero, which read like a win on the dashboard. What actually happened is that inputs the model could not classify used to come back empty and get routed to a human, and now they came back as a confidently-typed category that happened to be wrong. We measured it on a held-out set of genuinely-ambiguous tickets: roughly 1 in 6 of them got a crisp, schema-valid, incorrect label, where before they would have been flagged as unclassifiable. We had not reduced the errors. We had hidden them behind a green checkmark.

The claim in one line. A strict schema with no representable "I cannot answer" outcome does not make the model more reliable, it makes the model's uncertainty invisible, and an invisible fabrication is more expensive than a visible refusal. Below is the argument in cases, and the schema shape I now reach for first.

Case 1: the abstain-shaped input is the one that hurts you

Most inputs are answerable and the schema is fine. The failure lives in the tail: the malformed record, the question the context does not cover, the enum that has no member for what the model is actually looking at. For those, a strict schema offers exactly one path. Pick a value. The model is not choosing to lie, it is doing the only thing the contract permits, which is to emit the most probable allowed token given an input it has no real answer for.

The tell is that these fabrications are indistinguishable from confident correct answers at the type level. A wrong-but-required category and a right category are the same shape (both pass), so no downstream validator, no retry loop, nothing catches it. The signal you needed (this input was unanswerable) existed for one moment inside the model and your schema threw it away.

Case 2: a discriminated union makes abstention a first-class outcome (my default)

The fix that has held up best is to stop pretending every response is an answer. Model the response as a tagged union of two shapes: an answer, or an explicit abstention that carries the reason it could not answer. In pydantic v2 this is a discriminated union on a literal kind field, and the discriminator is what lets you branch without guessing.

python
from typing import Literal, Union
from pydantic import BaseModel, Field, TypeAdapter

class Answer(BaseModel):
kind: Literal["answer"] = "answer"
category: Literal["billing", "bug", "feature_request", "account", "other"]
confidence: float = Field(ge=0.0, le=1.0)

class Abstain(BaseModel):
kind: Literal["abstain"] = "abstain"
reason: Literal[
"insufficient_context",
"ambiguous_between_categories",
"input_malformed",
"out_of_scope",
]
note: str = Field(default="", max_length=280)

Result = Union[Answer, Abstain]
_adapter = TypeAdapter(Result) # discriminates on kind

def classify(raw_json: str) -> Result:
return _adapter.validate_json(raw_json)

The caller cannot ignore the abstain arm, because the two shapes are different.

res = classify(model_output)
if res.kind == "abstain":
route_to_human(reason=res.reason, note=res.note) # a clean, typed refusal
else:
apply_label(res.category, res.confidence)

The property that matters: Answer and Abstain are structurally different, so the caller has to handle both arms. There is no way to accidentally treat an abstention as an answer, because it does not have a category field to read. The reason for abstaining is itself a constrained enum, so "I could not answer" arrives with a machine-actionable cause attached, not as a shrug.

Case 3: Optional fields are the weak version, because they lose the why

The reflexive fix, once you see this, is to make everything optional. Let category be str | None, let the model return null when it is stuck. This is better than a forced enum, but it is the weak form, and it fails for one specific reason: None tells you a field is missing, it does not tell you why it is missing.

python
from pydantic import BaseModel

Weak fix: null-as-abstain. Works, but throws away the reason.

class LossyResult(BaseModel):
category: str | None # None could mean:
# - genuinely ambiguous input
# - context did not cover it
# - the model malfunctioned
# - a real answer of "none of the above"

Now every null looks the same at the call site. "This ticket is ambiguous between two categories" and "the input was garbage" and "this is legitimately none-of-the-above" all collapse into one None, and you have to reconstruct the cause from logs you probably did not keep. Worse, None is often a valid answer to some questions (no middle name, no discount applied), so you have overloaded one token to mean both "the answer is nothing" and "I have no answer." The discriminated union keeps those separate on purpose. If you truly cannot afford a union, the next best thing is a pair of fields, an abstained: bool plus a reason, so the caller has an explicit flag to check rather than inferring intent from a null. And a sentinel enum member (a NOT_ENOUGH_INFORMATION value inside the category enum itself) is the cheapest retrofit of all when you cannot change the response shape, though it muddies the enum's meaning by mixing a control signal in with real categories.

My working rules, stated plainly:

1. Every strict output schema needs a representable "I cannot answer," or you are converting refusals into silent fabrications.
2. Prefer a discriminated union of answer-or-abstain, so the two outcomes are different shapes the caller must branch on.
3. If a union is too heavy, use an explicit abstained flag plus a reason the caller checks, not a bare None.
4. A reason for abstaining should be a constrained enum, so downstream code can route on it (retry, human, drop) without parsing prose.
5. "Just make it optional" is the weakest fix, because a null loses the reason and collides with legitimately-empty answers.

Where I'd push back on this

The honest counter is that giving a model an abstain arm is giving it an escape hatch, and escape hatches get overused. A model that can say "insufficient_context" will sometimes say it on inputs that were perfectly answerable, because abstaining is easy and being right is hard. If your abstain rate creeps to 30 percent, you have not built reliability, you have built a very confident way to route everything to a human, and now the humans are the bottleneck you were trying to remove. That failure is real and I have seen it: the union made refusal cheap, so the model took the cheap path more than it should have. You manage it by measuring the abstain rate as its own metric, sampling abstentions for ones that should have been answers, and tightening the prompt (or the examples) when the model is hiding behind the escape hatch instead of using it.

So I will concede the boundary. If your task is genuinely closed-world (a finite set of inputs you fully control, where every input has a correct answer by construction), then there is nothing to abstain about, and the abstain arm is dead code that only invites the model to misfire. A strict schema with no escape hatch is the right call there, and adding a union is over-engineering a problem you do not have. Where I do not move is anything open-world: user free text, scraped documents, anything where the input can be malformed or out of scope. There, the inputs that break your schema are exactly the inputs you most need to know broke it, and a schema that forces a confident answer on an unanswerable input is not being strict, it is being wrong on purpose and calling it valid.