DEV Community

AgentBasics
AgentBasics

Posted on Originally published at agentbasicshq.com

LLM JSON output truncated? A deterministic repair ladder

#debugging #llm #python #tooling

You prompted for JSON. The model returned something that is 95% JSON. json.loads throws. Now what?

The two common reactions are both wrong. Reacting with "the API errored" wastes a perfectly good response. Reacting with a pile of regex hacks silently corrupts data — a greedy re.sub that strips a comma inside a string, a naive brace-matcher that trips over a quote in a value. The fix is a repair ladder: a deterministic sequence of progressively more aggressive fixes, stopping at the first that parses, and — this is the part people skip — reporting which rung saved you.

What actually breaks LLM JSON

  • Markdown fences: triple-backtick json blocks wrapped around valid JSON. Harmless, constant.
  • Prose around the object: "Sure! Here's the JSON:" then the object, then "Let me know!" Your parser needs the first balanced {...}, string-aware.
  • Trailing commas: {"a": 1,} — JSON forbids them, JavaScript habits produce them.
  • Python literals: True/None instead of true/null, single-quoted keys.
  • Truncation mid-string: max_tokens fires in the middle of a value. The output is not recoverable in full — and pretending otherwise is how agents invent the missing half.

The ladder 

def repair_json(text):
    rungs = [
        ("clean",      lambda s: s),
        ("fence",      strip_code_fences),        # strip markdown fences
        ("extract",    first_balanced_span),      # string-aware brace matcher
        ("trailing",   remove_trailing_commas),   # ,} -> }
        ("pyliterals", normalize_true_none),      # True -> true, None -> null
        ("quotes",     single_to_double_quotes),
        ("truncation", close_open_strings_and_brackets),
    ]
    for name, fix in rungs:
        try:
            return json.loads(fix(text)), RepairReport(rung=name)
        except (json.JSONDecodeError, NoBalancedSpan):
            continue
    return None, RepairReport(rung="hopeless")
Enter fullscreen mode Exit fullscreen mode

Two design details matter:

1. The brace matcher must be string-aware. Counting { and } naively breaks the moment a value contains a brace. Track quote state and escapes while scanning — ten lines, no dependencies.

2. The truncation rung is a last resort, and it must announce itself. Closing open strings and brackets turns {"items": [1, 2, 3 into something that parses — but you recovered 3 of maybe 60 items. That's why the function returns the report: rung="truncation" is your signal to log it, retry with a higher token cap, or tell the user the result is partial. A repair that silently pretends nothing happened is just hallucination with extra steps.

What this looks like in practice 

obj, report = repair_json(raw_llm_output)
# -> obj = {"items": [1, 2, 3]}, report.rung = "truncation"
Enter fullscreen mode Exit fullscreen mode

That's a real run from our test suite: fence stripped, trailing comma inside the array dropped, the truncated string closed — and the report says the answer came from the desperate rung, so the caller decides whether to trust it. A clean parse reports rung="clean"; over a week of production runs the rung distribution tells you whether your prompts or your token caps are the real problem.

This is the json_repair module in AgentOps Autopilot (MIT, stdlib only). Pair it with the retry pattern from the previous post: a truncated response should count as a retryable failure, not a success.

Next in the series: timeout handling so 40-step runs stop dying at step 41.

Top comments (0)