How Do You Compare LLM Answer Verifiers? Run ZTC, JEV and Laya on the Same Answer.

Short answer: paste a question and an answer into the
Verifier Playground, and ZTC, JEV and
Laya each return their own score on that exact input, side by side. It is free, needs no signup, and
the code is Apache-2.0.

Benchmarks tell you which verifier wins on average. They do not tell you what any of them does with
your input. This is the tool for the second question.

What an answer verifier is, in one paragraph

An answer verifier — also called a typed decision model — reads a question and an answer
produced by some LLM, and returns a structured verdict on whether that answer can be trusted. A
boolean, a choice, or a score. Not free text.

The defining property is that it runs one forward pass and generates zero tokens. That is what
makes it one to three orders of magnitude cheaper and faster than asking a frontier model "is this
right?" TypeSafe AI's Jev popularised the category under the name System One models.

If you are building an agent that needs to decide retry, escalate, or ship, this is the component
that makes that call.

The three systems in the playground

System	Vendor	Weights	Output scale	Measured AUC
ZTC-Judge-27B	VIDRAFT	open	unbounded real	0.7282
JEV	TypeSafe AI	closed (API)	0-1 probability	0.7350
Laya-Multilingual 322M	Convai Innovations	open, Apache-2.0	0-1 probability	0.4796

The AUC column comes from the
Typed Decision Leaderboard,
which scores these systems on 2,018 items with identical labels. The playground shows that figure
under each card so you always see the measured number next to the one-off result.

How to use it

Open the Space. Nothing to install.
Paste a question and an answer — or click one of eight built-in cases.
Pick a domain.
Read three cards: each system's score, its scale, its latency, and its generated-token count.

The eight built-in examples are not softballs. They are the failure modes that actually get past
review:

Swapped constant — a photon-energy calculation that uses the right formula and then reaches for the reduced Planck constant halfway through.
Confident wrong mechanism — statins described as binding circulating LDL and marking it for macrophage clearance. Fluent, authoritative, wrong.
Clean work, wrong answer — an inclusion-exclusion problem worked correctly and then answered with the wrong quantity.
Multi-step trap — a train arrival time where the time-zone step is applied in the wrong direction.

Every one of them reads well. That is the point: the answers that survive human review are the
fluent wrong ones, and a verifier earns its place by catching exactly those.

Three things to know before you read the output

A single case cannot rank these tools. One input is one sample. The leaderboard figure under
each card is the measured one; the live result is an illustration of behaviour, not evidence of
quality. This is stated in the Space itself rather than left to be inferred.

The scales differ and are not forced onto one axis. ZTC returns an unbounded real number; JEV
and Laya return 0-1 probabilities. Normalising them onto a shared 0-100 bar would look tidier and
would be false. Compare rank within a system, not raw values across systems.

ZTC returns no score for domains it has not measured. That refusal is deliberate. A number
produced for an unmeasured domain has nothing behind it, so the card reports unmeasured instead of
guessing. If you are evaluating verifiers, this is a property worth asking every vendor about.

What it costs to run yourself

The Space is Apache-2.0 and the whole app is 173 lines. Duplicate it and set two secrets:

Secret	Purpose	Required?
`HF_TOKEN`	read access to the ZTC model repository	yes
`AI_GATEWAY_API_KEY`	JEV calls via the Vercel AI Gateway	no

Without the gateway key, the JEV card reports that no key is configured and the other two still run.
So you can evaluate the two open systems at zero cost, on your own data, in a private Space.

Frequently asked questions

Can I test a hallucination detector without writing code?

Yes. The Verifier Playground runs three systems in the browser with no signup and no local install.
Paste a question and an answer, or use one of the eight bundled cases.

Is there a free alternative to Jev I can try right now?

Two of the three systems in the playground are open-weight: ZTC-Judge-27B and Laya
(Apache-2.0). Both run without an API key. On the shared test set ZTC-Judge-27B scores 0.7282 and
Laya scores 0.4796, against JEV's 0.7350 — so "free" and "equivalent" are not the same thing here,
and the playground lets you see the difference on inputs you care about.

Why do the three systems return numbers on different scales?

Because they were built that way. ZTC emits an unbounded real value; JEV and Laya emit 0-1
probabilities. The playground reports each on its native scale rather than rescaling them, because
rescaling implies a comparability that does not exist between a probability and an unbounded score.

What does "zero generated tokens" mean and why does it matter?

The verifier produces a verdict from a single forward pass without emitting any text. No decoding
loop means latency in the tens of milliseconds and cost roughly two orders of magnitude below an
LLM judge. Laya, for example, runs at about 0.015 s per call.

Can one test case tell me which verifier is better?

No. Use the playground to understand behaviour — where a system is confident, where it abstains,
how it handles your domain's phrasing. Use the leaderboard for ranking, where 2,018 items and
bootstrap intervals do the work a single case cannot.

DEV Community