Pydantic caught three distinct JSON failures from a model output

Pydantic raised a validation error on three separate content drafts across two days. Two were truncated: one stopped mid-value at column 20914, one mid-string at column 21862. The third carried a trailing comma at column 1308. Each surfaces in Pydantic as a single json_invalid error with a column number.

The three alerts fired at different times: the trailing comma on 2026-08-23, the two truncation errors on 2026-08-24. The record does not say why any of the three arrived in that state.

In each case the topic was left in the plan, and the alert noted that a second failure on the same topic would point to the topic or the gateway rather than the model.

EOF mid-value, EOF mid-string, and trailing comma are distinct failure shapes that all surface as a single Pydantic json_invalid error, so the column number and the error subtype together are what tell them apart.

---

Originally published at neuragrowth.co. NeuraGrowth is a one-person digital-products studio; this is the log of what its pipeline does and where it breaks.

Comments

[Vinh Nguyen]

The subtype you want is already machine-readable, just not in the field where the error type lives. On pydantic 2.13.5 all three shapes come back as type='json_invalid', so the type cannot separate them, but ctx['error'] carries the parser string verbatim: EOF while parsing a value, EOF while parsing a string, and trailing comma. So classification does not need the raw payload — a prefix match on ctx['error'] gets you the three buckets, and the payload is then for diagnosis rather than for routing. One collision worth knowing before you build on that: truncation in the middle of a key also reports EOF while parsing a string, the same wording as truncation inside a value, and I got column 18 against column 36 for the two, so the column is the only thing separating them and you need the schema to read it.

[Raknaos]

That distinction between the three failure shapes collapsing into a single json_invalid error is the painful part. I've had the same experience feeding model output into Pydantic schemas: truncation from a max-token cutoff, a trailing comma from the model imitating JS-style JSON, and silent encoding issues all look identical at the error site.

What helped me was logging the raw payload next to the error before any retry logic — column number alone isn't enough when the same topic can fail three different ways. Your note about a second failure on the same topic pointing at the gateway rather than the model matches what I saw: consecutive identical truncations were almost always the proxy cutting responses, not the model itself.