When Your AI Returns Markdown and Your TypeScript App Expected JSON

#typescript #ai #node #json

Book: AI That Answers
The series: AI in TypeScript — 5 books, from your first LLM call to agents in production — all five here
My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
Me: xgabriel.com | GitHub

You asked for JSON. You said "respond with JSON only, no other text." You said
it in capitals. And every so often you get:

Sure — here's the extracted data:

```json
{"total": 1240, "currency": "EUR"}
```

Let me know if you need anything else.

JSON.parse dies on the first character. Your error says Unexpected token 'S', which points at nothing useful, and the retry usually produces the same
shape because the prompt did not change.

The right fix is constrained generation. But you will not always have it — a
model that does not support it, a provider you cannot change, a path where
tool-calling is not appropriate. So the extractor is worth having, and it is
worth writing properly rather than as accumulated regex.

The five shapes you actually get

1. Fenced. The most common by far, either a json-tagged fence or an
untagged one.

2. Prose then fence then prose. Conversational padding on both sides.

3. Bare JSON with a preamble. No fence, just Here you go: {"total": …}.

4. Truncated. Hit max_tokens mid-object. Structurally unrecoverable, and
importantly, a different failure from the others.

5. Valid JSON, wrong shape. Parses fine, fails your schema. Not this
post's problem, but the extractor must not confuse it with the others.

Extract in layers, cheapest first

export type Extract =
  | { ok: true; value: unknown; how: "direct" | "fenced" | "scanned" }
  | { ok: false; reason: "truncated" | "no-json" };

export function extractJson(text: string, stopReason?: string): Extract {
  if (stopReason === "max_tokens") return { ok: false, reason: "truncated" };

  const direct = tryParse(text.trim());
  if (direct.ok) return { ok: true, value: direct.value, how: "direct" };

  for (const block of fencedBlocks(text)) {
    const p = tryParse(block);
    if (p.ok) return { ok: true, value: p.value, how: "fenced" };
  }

  const scanned = scanBalanced(text);
  if (scanned) {
    const p = tryParse(scanned);
    if (p.ok) return { ok: true, value: p.value, how: "scanned" };
  }
  return { ok: false, reason: "no-json" };
}

Checking stop_reason first is the part that gets skipped. Truncation
needs a different response from every other case — a bigger max_tokens or a
smaller request, not a re-prompt, and if you do not check it here you will
spend a while debugging "malformed JSON" that was actually a length problem.

Returning how matters too: a rising fenced rate means your prompt is
drifting, and you would never see that from a boolean.

Fence extraction without regex soup

function* fencedBlocks(text: string): Generator<string> {
  const re = /```
{% endraw %}
(?:json|JSON)?\s*\n([\s\S]*?)\n
{% raw %}
```/g;
  for (const m of text.matchAll(re)) yield m[1];

  // unterminated final fence — model ran out before closing it
  const last = text.lastIndexOf("```
{% endraw %}
");
  if (last !== -1 && (text.match(/
{% raw %}
```/g)?.length ?? 0) % 2 === 1) {
    yield text.slice(last + 3).replace(/^(?:json|JSON)?\s*\n/, "");
  }
}

The odd-count branch handles the model closing its object but not its fence,
which happens often enough to be worth eight lines. A generator means we stop
at the first block that parses rather than extracting all of them.

Scanning for a balanced object

Last resort, when there is no fence at all. Naively taking indexOf("{") to
lastIndexOf("}") breaks the moment prose contains a brace, so track depth,
and respect strings:

function scanBalanced(text: string): string | null {
  const start = text.search(/[[{]/);
  if (start === -1) return null;

  const open = text[start];
  const close = open === "{" ? "}" : "]";
  let depth = 0, inStr = false, esc = false;

  for (let i = start; i < text.length; i++) {
    const c = text[i];
    if (esc) { esc = false; continue; }
    if (c === "\\") { esc = true; continue; }
    if (c === '"') { inStr = !inStr; continue; }
    if (inStr) continue;
    if (c === open) depth++;
    else if (c === close && --depth === 0) return text.slice(start, i + 1);
  }
  return null;
}

The inStr tracking is what stops a brace inside a string value from
unbalancing the count. A version without it works on your test fixtures and
fails on the first record whose description contains {.

Do not repair invalid JSON

There is a tempting next step: strip trailing commas, quote unquoted keys,
convert single quotes. Libraries exist for it.

Avoid it in a data path. Repair changes the meaning of what the model
produced, silently, and you can no longer tell whether the value you stored is
what it generated. {"total": 1240,} becoming {"total": 1240} is harmless;
{"items": [1, 2, becoming {"items": [1, 2]} invents a complete list from a
truncated one.

If the text is not valid JSON, that is signal. Feed it back:

const ex = extractJson(text, res.stop_reason);
if (!ex.ok) {
  if (ex.reason === "truncated") throw new ResponseTruncated(res.usage);
  messages.push({ role: "assistant", content: text });
  messages.push({ role: "user", content:
    "That was not valid JSON. Reply with the JSON object only — no prose, " +
    "no code fence, no explanation." });
  continue;
}

Naming the specific problems ("no prose, no code fence") corrects far better
than "invalid JSON, try again", because the model can see what to change.

Prevention beats extraction

Two things reduce how often you need any of this.

Prefill the assistant turn. If the API allows starting the assistant
message, open the object yourself:

messages: [
  { role: "user", content: prompt },
  { role: "assistant", content: "{" },
]

The model continues from {, so a preamble is not reachable. Remember to
prepend the { back before parsing.

Force a tool call. Where supported, this is strictly better than asking
for JSON in prose — the arguments arrive as a structured object rather than as
text you have to recover:

tools: [{ name: "record", input_schema: zodToJsonSchema(Invoice) }],
tool_choice: { type: "tool", name: "record" },

Then block.input is already an object, and the whole extraction problem
disappears. Reach for the extractor only on paths where this is not available.

Instrument which layer fired

metrics.increment(`json.extract.${ex.ok ? ex.how : ex.reason}`);

direct should dominate. A climb in fenced means prompt drift — something
changed and the model started wrapping. A climb in scanned means it stopped
fencing too, which usually means your instruction got buried by a growing
prompt. truncated climbing means inputs got bigger.

Each of those has a different fix, and the counter is what distinguishes them.
Without it, all three look like "the JSON parsing is flaky."

If this was useful

AI That Answers covers getting
structured data out of a model reliably — constrained generation, prefill,
extraction as a fallback, and the parse boundary that turns any of it into a
TypeScript type.