How to Stop an LLM From Returning Broken JSON — and Why Valid JSON Is the Easy Half

Here is a model response from a pipeline that reads invoices. It parses. It matches the schema. Every field is the right type.

{
  "verdicts": [
    {
      "doc_id": "doc_a1",
      "risk": "block",
      "reasons": ["already_paid"],
      "explanation": "INV-001 from Acme Supplies was already settled in \"July payables\".",
      "evidence": [
        { "doc_id": "doc_x9", "quote": "Invoice INV-001 · Acme Supplies Ltd · Paid 1,000.00 USD" }
      ],
      "confidence": 0.94
    }
  ]
}

JSON.parse is happy. A validator checking types is happy. And the whole thing is fiction: doc_x9 was never sent to the model, and nothing in the batch says the word paid. The model wrote a citation the way it writes everything else — by producing the most plausible next token — and a citation is a string, so a string is what came out.

That response is going to be shown to a human next to a button that posts money.

Getting valid JSON out of a model is the easy half of this problem, and it's the half everyone writes about. The hard half is JSON that is valid, well-typed, on-schema, and wrong. Below are nine steps, in the order I'd apply them, that take a pipeline from hoping to enforcing. Every one of them is a few lines of ordinary code, and none of them are model-specific — they survive your next provider swap.

---

First: the four ways model JSON goes wrong

They need different fixes, and lumping them together is why "just add a retry" doesn't work.

#	Failure	Looks like	Fixed by
1	Unparseable	fences, a preamble sentence, a trailing comma	JSON mode, then bracket salvage
2	Parses, off-schema	confidence: "high", a risk you don't have a branch for	field-by-field coercion
3	On-schema, invented	an id you never sent, a quote nobody wrote	pointers instead of prose
4	On-schema, true, useless	"confidence 0.97 is below the 1.00 bar"	writing rules the schema can't hold

Steps 1–3 below fix category 1. Step 4 fixes category 2. Steps 5–7 fix category 3 — the expensive one. Steps 8–9 are what happens when all of it fails anyway, which it will.

---

Step 1: Stop asking for JSON in prose

If your provider has a JSON or structured-output mode, use it, and set temperature to 0 for anything that is a judgement rather than a piece of writing.

async def complete_json(system: str, user: str) -> dict[str, Any]:
    """One call per batch — never per document.

    Ask for JSON (response_format / json mode if the provider has it) and temperature 0: two runs
    over the same books should not disagree about which bill is a repeat. On a parse failure return
    {} — `validate` turns that into "not checked" per item, which is the honest outcome and the
    safe one.
    """

Temperature 0 is not about quality here. It's about two runs over the same input not disagreeing, which is the difference between a check and a coin flip. If a user re-runs the same batch and gets a different verdict, you no longer have a feature — you have a slot machine with a business logo on it.

Rule 1: sampling temperature is a product decision, not a tuning knob. Anything a user can re-run and compare belongs at 0.

---

Step 2: Shrink the schema until most of it is a closed set

Every free-form string in your schema is a place the model can be creative. Most of them don't need to be strings.

RISK = Literal["block", "review", "clear"]

REASONS = {
    "already_paid", "duplicate_invoice", "vendor_alias", "overpaid", "unallocated_payment",
    "instalment_ok", "credit_note_offsets", "restated_invoice", "currency_mismatch",
}

# Fields a citation may point at. Anything outside this set is dropped — it is how "cite the
# document" stays a checkable instruction instead of a request for a nice sentence.
CITABLE = ("invoice_number", "vendor", "total", "issue_date", "currency", "notes", "ledger")

Three things this buys you that a prose description of the same schema does not:

• An unknown value is detectable. "risk": "suspicious" is a bug you can see. "risk": "this looks suspicious to me" is a bug you can only see once a user reports the UI is blank.
• The set is a contract you can version. Ours says, in the spec: extend by asking us, not by inventing. A model that invents a tenth reason gets it dropped, not rendered.
• You stop translating slugs into sentences on the fly. Keep the closed set for grouping and filtering; keep one free-form explanation for the human. One creative field, not nine.

Leave exactly as much free text as a person actually reads, and no more.

---

Step 3: Salvage the parse — once, cheaply, and never with a regex

Even in JSON mode, content arrives wrapped in fences or introduced by a sentence often enough to be worth handling. The whole fix is two indexOf calls:

/** Models wrap JSON in fences or a sentence often enough that reading the first bracketed array is
 *  worth more than trusting the response to be clean. */
function parseRanking(raw: string): { id: string; fit: number; reason: string }[] {
  const start = raw.indexOf('[')
  const end = raw.lastIndexOf(']')
  if (start < 0 || end <= start) return []
  try {
    const arr = JSON.parse(raw.slice(start, end + 1)) as unknown
    if (!Array.isArray(arr)) return []
    // …validation continues in step 4
  } catch {
    return []
  }
}

That's the entire acceptable surface area of "JSON repair". First bracket, last matching bracket, one JSON.parse, and a catch that returns your empty case.

What I'd argue against: the libraries and hand-rolled fixers that close unbalanced braces, strip trailing commas, and re-quote keys. They work, which is the problem — they turn a loud failure into a quiet guess. A response truncated mid-object is a response where you don't know what was cut. Repairing the braces gives you a well-formed object that is missing half its verdicts, and nothing downstream can tell.

Rule 2: salvage formatting, never content. If the bytes that describe the answer are incomplete, you do not have an answer, and step 8 is what happens next.

---

Step 4: Parsing is not validating — coerce every field, individually

This is the step that most pipelines skip, because after JSON.parse returns an object it feels done. Types from a model are suggestions:

return arr
  .map((x) => x as Record<string, unknown>)
  .filter((x) => typeof x?.id === 'string')
  .map((x) => ({
    id: x.id as string,
    fit: Math.max(0, Math.min(100, Number(x.fit) || 0)),
    reason: typeof x.reason === 'string' ? x.reason : '',
  }))

Read what each line assumes will go wrong, because each one has:

• filter(typeof x?.id === 'string') — an entry with no id is not an entry. Drop it; don't default it to "" and carry a ghost through the rest of the pipeline.
• Number(x.fit) || 0 — handles "87", null, undefined and NaN in one expression.
• Math.max(0, Math.min(100, …)) — the model has been asked for 0–100 and will occasionally return 120, or 0.87 because it decided the scale was a fraction. Clamping means your progress bar never renders off the end of its container.
• typeof x.reason === 'string' ? x.reason : '' — a missing sentence is an empty sentence, not undefined printed into your UI.

The Python side does the same thing to the same values:

risk = item.get("risk")
if risk not in ("block", "review", "clear"):
    risk = "review"                                    # unknown → the safe branch, not a crash
reasons = [r for r in (item.get("reasons") or []) if r in REASONS]   # unknown members dropped
try:
    confidence = min(1.0, max(0.0, float(item.get("confidence") or 0.0)))
except (TypeError, ValueError):
    confidence = 0.0

Note what isn't here: an exception thrown at the caller. A model returning something odd in one field of one item is an ordinary Tuesday, and it should cost you that field, not the request.

Rule 3: every field gets a defined behaviour for "the model said something else." Written down, in code, next to the field. "That shouldn't happen" is not a behaviour.

---

Step 5: Let the model point. You render.

Here is the single structural idea, and if you take one thing from this article take this one.

The model does not write the citation. It returns a pointer — {doc_id, field} — and your code renders the text from its own copy of that record.

def _render_evidence(raw: list[dict[str, Any]], index: dict[str, dict[str, Any]]) -> list[Evidence]:
    """Turn (doc_id, field) pointers into text, from our own records.

    This is the step that makes a citation checkable: the model chooses WHICH fact to stand on, and
    the value comes from the data, so it cannot be embellished. A pointer at a document or field
    that was never sent simply does not render.
    """
    out: list[Evidence] = []
    for item in raw or []:
        doc_id = str(item.get("doc_id") or "")
        field = str(item.get("field") or "")
        record = index.get(doc_id)
        if not record or field not in CITABLE:
            continue
        value = str(record.get(field) or "").strip()
        if not value:
            continue
        ref = record.get("invoice_number") or record.get("vendor") or doc_id
        out.append(Evidence(doc_id=doc_id, quote=f"{ref} · {value}"))
    return out

Go back to the response at the top of this article. Under this scheme it cannot exist: doc_x9 isn't in index, so the pointer doesn't resolve, so there is no evidence — and by step 7 the block becomes a review. The invented citation didn't get caught by a checker. It became structurally impossible to express.

The same idea in three lines of TypeScript, on the ranking case:

const byId = new Map(jobs.map((j) => [j.id, j]))
return parsed
  .map((r) => (byId.get(r.id) ? { ...byId.get(r.id)!, fit: r.fit, reason: r.reason } : null))
  .filter((j): j is JobMatch => j !== null)

The model ranks ids we sent. An id it invented finds nothing in the map and disappears. We never render a title, a company or a URL that the model produced — those come from our own row, keyed by an id the model was only allowed to choose.

Rule 4: a model may choose among your facts. It may never author one. Ids, quotes, prices, URLs, names — pointer in, render out.

This is also the honest answer to "how do we stop hallucinated citations?" You don't detect them. You remove the ability to write one.

---

Step 6: Enforce cardinality, and make a missing answer say so

You asked for one verdict per invoice. You will get: two for the same document, none for another, and one for a document that isn't in this batch.

wanted = [d.doc_id for d in req.invoices[:MAX_INVOICES]]
seen: dict[str, Verdict] = {}

for item in raw.get("verdicts") or []:
    doc_id = str(item.get("doc_id") or "")
    if doc_id not in wanted or doc_id in seen:
        continue  # not ours, or said twice
    # …coerce and store…

# Exactly one verdict per invoice. A missing one is not silence — the client reads absence as
# "not checked", so say that rather than letting the row look clear.
for doc_id in wanted:
    if doc_id not in seen:
        seen[doc_id] = Verdict(doc_id=doc_id, risk="review", reasons=[],
                               explanation="This invoice was not checked.", confidence=0.0)
return [seen[doc_id] for doc_id in wanted]

Three separate guarantees in fifteen lines: only ours (unknown ids dropped), at most one (first wins), at least one (gap filled), and the return is ordered by your input rather than by the model's output order.

The filled gap matters more than it looks. An item with no verdict renders as a row with no warning on it, which a person reads as checked and fine. Absence of an answer must be rendered as an answer, or your UI is quietly lying in the exact place it is meant to be reassuring.

---

Step 7: Downgrade, don't reject

When a rule is broken, the tempting move is to throw the item away. Usually the better one is to demote it to the outcome that costs least if you're wrong:

evidence = _render_evidence(item.get("evidence") or [], index)
# Cite or downgrade. The prompt says it; this is what makes it so.
if risk == "block" and not evidence:
    risk = "review"
explanation = str(item.get("explanation") or "").strip()
if not explanation:
    risk = "review"

For this pipeline the asymmetry is: "held for a human to look at" costs a few seconds; "cleared automatically" costs a duplicate payment. So the spec says the thing that makes every rule above resolvable:

Default to review when unsure. clear is a claim, not a fallback.

Find the equivalent sentence for your own pipeline before you write the validator, because it decides every else branch in it. A moderation pipeline's safe default is not a search pipeline's safe default. What you must not do is let the safe default be whatever the model happened to say when it broke a rule.

Rule 5: name your cheap direction of failure, in one sentence, and make it the value of every fallback.

---

Step 8: Choose your failure posture before you need it

The model call will fail. Time out, 500, return prose, return {}. There are three defensible answers and you must pick one per surface:

try:
    raw = await complete_json(SYSTEM_PROMPT + "\n" + FEW_SHOT, user_prompt)
except Exception:  # noqa: BLE001 — a model failure must not 500 someone's books
    raw = {}
    note = "; ".join(filter(None, [note, "the check could not be completed"]))

verdicts = validate(raw if isinstance(raw, dict) else {}, req)

{} flows into the same validator, which fills every row with "This invoice was not checked." The user gets their books, plus an honest note, and nothing auto-posts. Compare the ranking case, where the failure posture is different because the stakes are:

} catch {
  return []   // the caller still has the unranked list, and a search that shows nothing is
}             // better than an error card

Two rules that fall out of doing this a few times:

• The enrichment path degrades; the safety path escalates. Ranking, summaries, suggested titles: fail to the unenriched version silently. Anything guarding money, deletion or publication: fail to held for a human, loudly.
• Retry structural failures, not semantic ones. An unparseable response or a 503 is worth one retry. A response that parsed and then failed your rules will fail them again — at temperature 0, identically. Change the input or fall back; don't spend a second call proving the first one wasn't a fluke.

---

Step 9: If the model generated logic, run it before you return it

Last one, for the case where the JSON isn't data but a small program — a formula, a filter, a query, a tool definition. Schema validation says nothing about whether it works.

Our tool builder turns "age calculator from a birth date" into a spec with an expression in it. The checklist it has to pass before it's allowed out:

1. Valid JSON matching the envelope shape.
2. Every field id matches ^[A-Za-z_][A-Za-z0-9_]*$, is unique, snake_case.
3. The expression references only existing field ids and the functions in the allowed list.
4. The expression parses and evaluates on a sample input — every number field 2, every date field 2000-01-01. If it throws, fix it or fall back.
5. icon and category are members of their allowed sets.

Rule 4 is the one people leave out and the one that catches the most: a formula referencing a field that isn't there, a division that can only produce Infinity, a date function given a number. You have a whole interpreter sitting right there. Use it as a validator. It costs microseconds and it is the only check in the list that tests the thing the user actually asked for.

The fallback matters too: fix it or fall back to a safe generic tool. Not "return the spec and let the UI throw."

---

The 60-second audit

Run these four greps against your own codebase. Each one took me under a minute:

• JSON.parse( on anything that came from a model, with no .filter or type guard after it. That's steps 4 and 6 missing. Look at what the very next line assumes.
• Any string field in your schema that holds an id, a quote, a URL or a price. That's step 5 missing — every one of those is a value the model can invent and your UI will print with a straight face.
• catch {} around a model call. Read what the caller does with the empty result. If the answer is "renders as success", you have a silent-lie path.
• Enum-ish fields typed string. risk: string, status: string, category: string. Any value outside your branches lands in whatever your else does, and your else was written for a bug you didn't have in mind.

What I'd stop saying

"Just use structured outputs" is now a complete answer to about a fifth of this problem — the fifth where the JSON is malformed. It's a genuine improvement and you should turn it on today.

It does nothing about the other four fifths. Structured output mode will happily hand you a perfectly-formed citation of a document that does not exist, a verdict for an id you never sent, no verdict at all for the one row that mattered, and a confidence of 0.97 where your threshold is 1.00, rendered to a user as a sentence they cannot act on.

The schema you describe to the model is a request. The schema you enforce in code is the only one your product actually has. Nine steps, a few dozen lines, and most of them are filter.