llm-json-repair: Three-Pass Local JSON Repair for LLM Responses in Rust

The panic that started it

The model returned this:

Here is the JSON you requested:

json
{
"status": "ok",
"items": ["a", "b", "c"]
}

Let me know if you need anything else.

toml

serde_json::from_str panicked.

The JSON itself was valid. The problem was the code fence, the intro sentence, and the trailing prose. The actual structured data was buried in there. Every project I have worked on had a version of this fix somewhere, usually a 10-line helper that someone wrote once and never exported.

I wrote it as a shared crate so it does not have to be written again.

Shape of the fix

[dependencies]
llm-json-repair = "0.1"

Pass in the raw model output:

use llm_json_repair::repair;

let raw = r#"Here is the JSON you requested:

json
{
"status": "ok",
"items": ["a", "b", "c"],
}

Let me know if you need anything else."#;

let repaired = repair(raw)?;

// repaired is now: {"status":"ok","items":["a","b","c"]}
let parsed: serde_json::Value = serde_json::from_str(&repaired)?;

rust

The function returns Ok(String) with the repaired JSON or Err(RepairError) if none of the passes produced valid JSON.

Handling the error:

use llm_json_repair::{repair, RepairError};

match repair(raw) {
    Ok(json_str) => {
        // parse and use
    }
    Err(RepairError::NoValidJson) => {
        // the model output was not recoverable
        // log the raw output and return an error to the caller
    }
}

What it does NOT do

• No LLM call. This is fully local string manipulation. No network round-trip.
• No semantic repair. If the model output has wrong field names or wrong value types, those will parse successfully and be wrong. This crate only fixes structural syntax issues.
• No schema validation. After repair, feed the result to a validator if you need type-checked output.
• No streaming repair. The input must be a complete string. If you are streaming model output, wait for the full response before calling repair.

Inside the lib

The three passes run in order. Each pass is tried only if the previous pass fails to produce valid JSON.

Pass 1: code fence stripping. The most common failure mode is a code fence. The pass looks for the pattern json ` or ` and extracts the content between the opening and closing fences. It tries serde_json::from_str on the extracted content. If that parses, the pass succeeds and the function returns immediately.

This pass is the cheapest. It does a single string scan for the fence delimiters. Most calls to repair exit here.

Pass 2: JSON extraction from prose. If there is no code fence, the pass walks the string looking for the first { or [. From that position, it tracks brace and bracket depth and finds the matching close. It extracts the substring and tries serde_json::from_str. If that parses, the pass succeeds.

This covers the case where the model outputs a JSON object or array directly but adds an intro sentence before it or a sign-off after it. The depth tracker handles nested objects and arrays correctly. It does not handle escaped brackets inside strings, which is a known limitation in v0.1.

Pass 3: trailing comma and brace repair. If the extracted substring from pass 2 does not parse (or pass 2 found nothing), this pass applies a set of string-level mutations:

• Remove trailing commas before } or ].
• Add a missing closing brace or bracket if the depth tracker ended with unclosed depth.

This handles the case where the model truncated mid-object or added a trailing comma in an array. After the mutations, it tries serde_json::from_str again.

The order matters. Pass 1 before pass 2 because fence stripping is a cheap constant-time check. Pass 2 before pass 3 because extraction without mutation is safer than extraction plus mutation.

When useful

• Any pipeline where the model is supposed to return structured JSON and you do not control the model's system prompt enough to guarantee clean output.
• Tool-call fallback when the model uses text-mode JSON instead of the API's structured output feature.
• Eval pipelines where you want to parse model responses programmatically and do not want to add retry calls for fixable formatting errors.
• Data extraction pipelines where the model wraps results in prose and you need the object reliably.

When NOT

• If the model returns consistently clean JSON (no fences, no prose), skip this crate. Adding a repair step introduces a parsing layer you do not need.
• If you are using the Anthropic or OpenAI structured output / tool-call JSON mode, the API enforces valid JSON for you. This crate targets text-mode responses.
• If the repair fails and you retry with a different prompt, the retry handles correctness at the semantic level. This crate only handles syntax.

Install

[dependencies]
llm-json-repair = "0.1"

Crates.io: llm-json-repair
GitHub: MukundaKatta/llm-json-repair

Siblings

Lib	Boundary	Repo
bedrock-kit	Python: repair + throttle + cache-cost, includes a repair pass	MukundaKatta/bedrock-kit
agentcast-rs	Repair + validate + LLM-retry loop for structured output	MukundaKatta/agentcast-rs
agentvet-rs	Validate tool call args before execution	MukundaKatta/agentvet-rs
llm-output-validator	Rule-based validator for LLM strings after parsing	MukundaKatta/llm-output-validator
tool-arg-coerce-py	Coerce parsed JSON values to expected Python types	MukundaKatta/tool-arg-coerce-py

What is next

The depth tracker in pass 2 does not handle escaped brackets inside string values. A proper tokenizer that respects JSON string escaping would make pass 2 more robust for pathological inputs.

Pass 1 could also handle Python-style single-quoted JSON, which some smaller models produce. That would be a fourth pass or an extension to pass 1.

The main thing v0.1 does not do is explain which pass succeeded. A RepairResult that includes the pass number and what was changed would help with debugging and monitoring in production pipelines.

---

Part of the Hermes Agent Challenge sprint. All crates shipped on crates.io.