DEV Community

Denis LaCombe
Denis LaCombe

Posted on

I got tired of LLMs returning almost-JSON, so I built a strict prompt generator

Hi, I'm Denis, a self-employed developer from Quebec, Canada. I build small free tools for people, and lately I've been building tools that talk to LLMs.

A few months ago I was working on a simple job: read a book catalog entry and pull out the title, the author, and the ISBN. Easy, right? I asked the model for JSON. What came back looked like JSON.

It was almost JSON. And almost-JSON broke my code.

Three ways "almost-JSON" broke my code

1. It didn't parse at all. A trailing comma here, a missing bracket there. JSON.parse threw, my script died, and I was debugging at 2 a.m. for a comma.

2. Right shape, wrong type. The ISBN came back as a number:

{ "title": "The Hobbit", "author": "J.R.R. Tolkien", "isbn": 9780261102217 }
Enter fullscreen mode Exit fullscreen mode

My database expected a string. ISBNs need to stay strings — an ISBN-10 can even end with the letter X, which is a valid check digit, not a typo. A number can't hold that.

3. Valid JSON, wrong contract. This one is the sneakiest, because it parses just fine:

{
  "title": "The Hobbit",
  "author": "J.R.R. Tolkien",
  "isbn": "978-0-261-10221-7",
  "price": "$12.99"
}
Enter fullscreen mode Exit fullscreen mode

I never asked for a price. My code didn't expect one. Downstream, something broke — quietly, three steps later, where it was hard to find.

The lesson: the model wasn't the problem

The problem was that I never gave the model a contract. I asked for "JSON" the way you'd ask a stranger for "directions" — and then I was surprised when the directions were vague.

A real contract has three parts:

  1. A strict system prompt — "return ONLY a valid JSON object, no explanations, no markdown, no extra fields."
  2. A JSON Schema — the exact shape, the exact types, nothing else allowed.
  3. Validation code — check the answer before your code touches it. If it fails, reject it, don't patch around it.

Here is the contract I use now for that book job. The schema comes straight from my tool's library:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "additionalProperties": false,
  "properties": {
    "author": {
      "minLength": 1,
      "type": "string"
    },
    "isbn": {
      "minLength": 1,
      "pattern": "^(?:ISBN(?:-1[03])?:?\\s*)?(?:(?:\\d[-\\s]?){9}[\\dX]|(?:\\d[-\\s]?){12}\\d)$",
      "type": "string"
    },
    "title": {
      "minLength": 1,
      "type": "string"
    }
  },
  "required": [
    "title",
    "author",
    "isbn"
  ],
  "title": "BookIsbn",
  "type": "object"
}
Enter fullscreen mode Exit fullscreen mode

And the system prompt that goes with it:

You are a precise book extraction engine. Read the provided library catalog entry and return ONLY a valid JSON object matching the schema below. Do not add explanations, markdown, or extra fields. Every field is required: title, author, and isbn. Extract the best value printed in the entry and never invent a value that is not printed. Never use null or an empty string for a required field. The isbn must be the full ISBN-10 or ISBN-13 exactly as printed, keeping its hyphens or spaces; it may start with the literal prefix "ISBN", "ISBN-10", or "ISBN-13" followed by a space or a colon. If any required field (title, author, or isbn) cannot be found in the entry, do not produce any output.

Then the check. Ten lines of Zod, and the three failure modes above all get caught before they reach my code:

import { z } from "zod";

export const BookIsbnSchema = z
  .object({
    title: z.string().min(1),
    author: z.string().min(1),
    isbn: z
      .string()
      .min(1)
      .regex(
        /^(?:ISBN(?:-1[03])?:?\s*)?(?:(?:\d[-\s]?){9}[\dX]|(?:\d[-\s]?){12}\d)$/
      ),
  })
  .strict();
Enter fullscreen mode Exit fullscreen mode

I ran the outputs through a validator to be sure. The clean answer passes. The one with the extra "price" field fails with a clear message: Additional properties are not allowed ('price' was unexpected). The ISBN-as-a-number fails too: 9780261102217 is not of type 'string'. No more silent breakage three steps later.

Writing all of this by hand got old fast

A system prompt, a schema, and validation code — for every little task, in three formats (Zod, Pydantic, plain JSON Schema)? That's a lot of careful typing for a small job. So I built a tool that does the careful part for me.

It's called StrictJSON — https://strictjson.com

You describe the data you need in plain words. It writes you the optimized system prompt, the strict JSON Schema, and the ready-to-use validation code. It checks its own output on the server and repairs it before you get it — because a prompt that only says "be strict" isn't strict enough.

No account, no API key. Ten free generations a day, made by server-side models. There's a library of twelve ready-made examples (receipts, invoices, meeting notes, job posts, and this book extractor) if you want to see what a good contract looks like.

And when you already have JSON and a schema and you just want to know "is this valid, and if not, what's wrong?" — that's the companion tool, SchemaBox: https://schemabox.dev

Paste your JSON and your schema. It tells you in plain language if it's valid — or exactly what's wrong. One hundred percent in your browser, nothing is sent anywhere, no account, free and unlimited. Ten tools: validator, data generator, formatter, converter, and guides in English and French.

What I'd love to know from you

Which format do you validate with — Zod, Pydantic, or plain JSON Schema? And what's the worst "almost-JSON" an LLM ever handed you? I read every comment, and I really want to hear the war stories — that's how the tools get better.

Try it, break it, and tell me what breaks. That's the deal.

— Denis

Top comments (0)