The confidence number is the product (and here is ours, with the caveat)

TuringCorp — Fri, 18 Sep 2026 13:35:40 +0000

Most AI tools answer the question. The harder job is judging between two answers you already have: two drafts, two plans, two offers. You usually lean one way already — what you cannot do is price your own uncertainty.

That is the only thing we built: Decider, a judge that takes a question plus two candidate answers and returns the stronger one, a calibrated confidence value, and the reasoning for the call.

The number is the product, and it comes with a caveat

Here is the part most products would leave out. On raw pick rate, Decider ties a plain single-model baseline — roughly 92.5% against 92.7% on JudgeBench. We do not claim to be more accurate.

What a single pass cannot give you is how close the call was. Every judgment carries a confidence value calibrated against outcomes:

Confidence reported	Share of judgments	Observed accuracy
≥ 90%	45.6%	99.6%
80–90%	29.7%	94.0%
70–80%	13.2%	84.1%
< 70%	10.5%	67.7%

JudgeBench, 614 judgments. These are our own runs on the official protocol; the runs that failed are disclosed rather than dropped, and the raw judgments are published.

Two outcomes, both useful

A high band is permission to commit and stop re-reading both options. A low band is not a failure — it is the answer you actually needed: the two are genuinely close, so take the one you already preferred and spend your time somewhere else. Terminating the comparison is the value.

Judge it on real work, not a demo script

We recorded 27 real runs across nine domains — writing, engineering, business, research, career, money, people, travel, everyday — and published each one end to end: the question, both candidate answers, the pick, the confidence and the full reasoning, verbatim. They are browsable as static pages:

https://api.turingcorp.net/platform/poe/examples

Nothing is charged for reading them, and the same set ships inside the app as free recorded runs.

Try it

Decider runs as an app on Poe: https://poe.com/TuringCorp-Decider — pay per decision, no subscription (list $0.50, launch offer $0.25). What it is, how it is measured and what it costs: https://api.turingcorp.net/platform/poe

If your pipeline needs a quality gate rather than a chat, there is also an OpenAI-compatible endpoint: https://api.turingcorp.net/

Disclosure: this post is written by the team that builds Decider.

DEV Community: TuringCorp

The confidence number is the product (and here is ours, with the caveat)

The number is the product, and it comes with a caveat

Two outcomes, both useful

Judge it on real work, not a demo script

Try it