DEV Community

lixingliangsy
lixingliangsy

Posted on

I built a validator that refuses to use AI. Business is fine.

Everyone building dev tools in 2026 has an AI story. Mine is that my most popular tool has an entire FAQ entry explaining that it contains no AI at all.

SchemaSafe is a JSON Schema validator. You paste a schema and a JSON instance, it tells you every way the instance fails the schema: wrong types, missing required fields, bad formats, unexpected properties. That's it. There's no model in the loop, and honestly, the first few times someone asked "but where's the AI?", I almost added one out of embarrassment.

I'm glad I didn't.

The case against the model, for this job

JSON Schema validation is a solved problem with a written specification. The spec says exactly what "valid" means: types, required keys, enums, format constraints, additionalProperties. There is no ambiguity for a model to reason about, and no fuzzy case where an LLM's judgement adds value.

What an LLM does add, in this context, is failure modes. A model that validates JSON can be talked into saying things are fine. It can't be trusted to enumerate every violation — enumeration is exactly what language models are worst at. And when it misses a required field violation, you find out in production, from your webhook handler, at 2am.

A ruleset doesn't miss violations. It walks the schema and reports every mismatch with a JSON-pointer path, like /items/2/quantity — which, by the way, was the other thing I kept getting wrong by hand. Telling a teammate "the third item's quantity field is a string" is much less useful than handing them the pointer.

Where AI does belong in this picture

I'm not anti-model. The SQL tool I built uses one, behind a deterministic safety screen, because writing SQL from natural language is genuinely open-ended. The dividing line I've settled on: use the model where the input space is unbounded, use rules where correctness is definable.

Validation correctness is definable. It's a spec.

The failure modes I actually had to handle

The interesting engineering was all edge cases. Invalid JSON input should produce "your JSON is broken at character N", not a model confabulating schema errors. A schema that itself is invalid needs to be rejected with the reason. Empty instances. Nested arrays. additionalProperties: false interactions with patterns. None of it exciting, all of it the reason the tool exists.

If your payload validation is currently "the model usually catches it," and your payloads arrive from third parties — webhooks, partner APIs, client-side forms — it might be worth an afternoon to move that check to rules that can't be sweet-talked.

SchemaSafe runs in the browser, no account, no key. If it misses a violation that your own validator catches, that's my bug, and I'd like to hear about it.

Top comments (0)