I Chased Malformed JSON for 48 Hours. My Parser Was Only Half the Problem.

Can a free model produce dependable structured output around the clock, or does the quality quietly rot after the twentieth hour? I spent two full days running an extraction job that turned messy support-ticket text into strict JSON, and the answer was less dramatic than I hoped. The model made mistakes, no question, but my parser and my assumptions failed me just as often. This is the field log, the repair loop I settled on, and the parts I would repeat without a second thought.

The experiment I actually ran

Every fifteen minutes, a small script pulled the newest ticket from a test inbox, asked the model for a fixed JSON shape, and stored whatever survived parsing and validation. The target schema had five fields — id, priority, category, summary, and requested_at — with priority as an integer from one to three. I ran the whole loop on MonkeyCode's free server option, used the free model access for every call, and recorded each failure in a SQLite table for later triage. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The honest headline is that roughly nine out of ten responses parsed cleanly on the first pass, which sounds fine until you realize that one in ten of a continuous stream becomes uncomfortable fast. The remaining failures were not one disease but several, and each one demanded a different medicine. What follows is the taxonomy of those failures, ordered by how much they annoyed me.

Failure one: valid JSON, useless shape

The biggest surprise was that json.loads almost never threw, because the model almost always emitted syntactically valid JSON. The real damage arrived as missing keys, extra keys, and values that were the right type but the wrong meaning. A response would return "priority": "urgent" when the schema demanded an integer, or it would quietly drop category after a verbose summary. My parser smiled and moved on, and the bug surfaced somewhere else entirely.

import json
import jsonschema

SCHEMA = {
    "type": "object",
    "required": ["id", "priority", "category", "summary", "requested_at"],
    "properties": {
        "id": {"type": "string"},
        "priority": {"type": "integer", "minimum": 1, "maximum": 3},
        "category": {"type": "string"},
        "summary": {"type": "string"},
        "requested_at": {"type": "string", "format": "date-time"}
    },
    "additionalProperties": False
}

def get_usable_data(raw: str):
    try:
        parsed = json.loads(raw)
        jsonschema.validate(parsed, SCHEMA)
        return parsed
    except (json.JSONDecodeError, jsonschema.ValidationError) as exc:
        return None

Parsing with json.loads alone was a lie, because it told me the output was valid when the shape was useless. Adding jsonschema turned every silent drift into an explicit error that I could feed back to the model. That single change converted a mystery into a diagnosis loop, and it is the first thing I would add to any LLM pipeline tomorrow.

Failure two: truncation made retries duplicate everything

Truncation was the nastiest failure because the JSON stayed syntactically valid right up to the cutoff, and my first retry made things considerably worse. When the output hit the token ceiling in the middle of an array, my naive repair simply asked the model to try again, and the model cheerfully re-emitted records it had already sent. My ledger showed the same id twice in the final payload, which broke the uniqueness constraint a few services downstream. Retrying without context was not a repair; it was a generator of new bugs.

What actually worked was sending the last successfully parsed element along with the validation error, so the retry could continue from the break instead of restarting from zero. The repair call cost a bit more input, but it stopped the duplication and reduced the number of failed records dramatically. One targeted retry beat three blind ones every single time.

Failure three: numbers wearing string costumes

Type drift sounds trivial until a string "2" flows into an API that expects an integer and the service starts rejecting entire batches. The model returned numbers as strings often enough that I stopped being surprised and started writing coercion rules with explicit limits. I let int() try the conversion, and anything that refused went straight to the dead-letter queue.

def coerce_field(value, expected_type):
    if expected_type == "integer" and isinstance(value, str):
        try:
            return int(value)
        except ValueError:
            return None
    return value

This feels like a rookie problem, and it is, but it is also the kind of problem that hides until your data lands in front of a customer. The schema validation above already knows the expected type, so the fix is a tiny function that the validator calls before it makes its final decision. Do not cast blindly, because a corrupted value should fail loudly instead of becoming a confident zero.

The decision table I eventually lived by

Failure class	Symptom	First attempt	What worked
Shape drift	Missing or extra keys	Accept whatever parsed	jsonschema + feed the error back
Truncation	Valid JSON cut mid-array	Retry from scratch	Resume from last valid element
Type drift	"2" instead of 2	Cast everything	Explicit coercion, fail loudly
Repetition	Duplicate entries	Trust the retry	Resume context, then dead-letter

The table looks obvious in hindsight, and that is exactly why I wrote it down at hour six instead of hour forty. Every row came from a real failure, and every repair took about an hour longer than I expected. If you run a similar pipeline, keep a failure ledger from day one.

The repair loop I'd repeat

By hour thirty, the pipeline had settled into a boring shape: parse, validate, repair once, then dead-letter the rest. One retry carrying a specific error message saved most of the remaining failures, while a second retry just burned tokens without saving anything. The skeleton below is the whole trick, and the boring parts are the point.

def handle(raw, prompt):
    data = get_usable_data(raw)
    if data:
        return data
    feedback = build_feedback(raw)       # exact validation error
    repaired = model_call(prompt, raw, feedback)
    return get_usable_data(repaired) or dead_letter(repaired)

Every record that failed twice went to a dead_letter table instead of being retried forever, and that table became the most valuable artifact of the whole run. At hour forty-eight I opened it and saw exactly four failure patterns, and all four were fixable in the pipeline. The ledger was the only honest health check I had.

The costs I actually tracked

Tokens were the real tax of the repair layer, because every retry doubled the input for that call and the output rarely shrank in return. Roughly a third of my total token spend went to repair calls that saved only a small slice of records, which is a terrible exchange rate for most paid APIs. Was that a fair trade? Not for a paid API, and barely for a free one, so I would rather pay one targeted retry than three blind ones. The free server option handled the loop without drama, which surprised me more than it should have.

What I would not repeat

The biggest mistake was treating the parser as the source of truth, because every hour spent debugging a malformed string was an hour of ignoring the schema question. I also retried too eagerly during the first day, and the duplication bugs that came out of it took longer to untangle than the original failures. And I trusted a single successful parse as proof that the pipeline was healthy, which hid a slow drift in category values until the dead-letter table exposed it.

Who should not use this approach

If your downstream system cannot tolerate two records with the same id, or a string where an integer belongs, then you need a stricter contract or a human in the loop, not another retry. This workflow is for low-stakes extraction where a dead-letter table and a fix in the next deploy are acceptable outcomes. For medical, financial, or customer-facing decisions, validation alone is not a safety net.

Would I run the same 48-hour experiment again? Absolutely, but with the schema check first, one retry carrying a specific error, and a dead-letter table that I actually read. The model was not the fragile part — my parser was only half the problem, and the other half was the assumption that valid JSON means useful data. Build the ledger, keep the repair loop boring, and let the failures teach you where the real contract lives.

If you are about to wire a free model into a pipeline, spend the first hour writing the failure table instead of polishing the prompt. That hour will save you a night of chasing ghosts, and you will end up with an artifact more useful than any dashboard.