Build a Bounded JSON Repair Loop for LLM Output in Python

“Return JSON” is a request, not a guarantee. An LLM can produce invalid syntax, valid JSON with the wrong shape, or a structurally valid answer that violates the task.

A repair loop should classify those failures and stop after a small number of attempts.

A runnable validator

This example uses only Python's standard library:

import json

CATEGORIES = {"bug", "feature", "question"}

def validate(raw: str):
    try:
        value = json.loads(raw)
    except json.JSONDecodeError as exc:
        return None, f"syntax: {exc.msg} at position {exc.pos}"

    if not isinstance(value, dict):
        return None, "shape: root must be an object"
    if set(value) != {"category", "summary"}:
        return None, "shape: expected category and summary only"
    if value["category"] not in CATEGORIES:
        return None, "semantic: category is not allowed"
    if not isinstance(value["summary"], str) or not value["summary"].strip():
        return None, "semantic: summary must be non-empty text"

    return value, None

Try three fixtures:

samples = [
    '{"category":"bug","summary":"Crash on save",}',
    '{"category":"bug","title":"Crash on save"}',
    '{"category":"urgent","summary":"Crash on save"}',
]

for sample in samples:
    print(validate(sample)[1])

Expected categories are syntax, shape, and semantic.

Bound the repair

MAX_ATTEMPTS = 2

raw = first_model_response
for attempt in range(MAX_ATTEMPTS + 1):
    value, error = validate(raw)
    if error is None:
        break
    if attempt == MAX_ATTEMPTS:
        raise RuntimeError(f"invalid model output: {error}")
    raw = ask_model_to_repair(raw, error)  # your API adapter

Send the original output, the exact validation error, and the required schema to the repair call. Do not add an expanding conversation history. It increases cost and can distract from the one correction needed.

Know what not to repair

A parser can identify malformed output. It cannot decide whether “Crash on save” really describes a bug. Validate factual support against source evidence in a separate stage. Automatically repairing an unsupported claim can make it more polished without making it true.

Also log attempt count, error class, model/version, and final disposition—but avoid logging sensitive prompts or full user data by default.

Learning outcome

The important pattern is not the particular schema. It is the control flow:

generate -> parse -> validate -> bounded repair -> accept or fail

Once you can observe each transition, you can measure how often repair helps and when it only burns tokens. “Usually valid JSON” becomes a testable rate instead of a hopeful integration plan.