DEV Community

Cover image for Judgment has two axes. Jev collapsed one. Almost nobody is racing on the other.
TuringCorp
TuringCorp

Posted on

Judgment has two axes. Jev collapsed one. Almost nobody is racing on the other.

Judgment has two axes. Jev collapsed one. Almost nobody is racing on the other.

Two offers on the desk.

The smaller company pays about 12% less, and the work sits closer to what you want to be doing in five years. The bigger one pays more, the commute is forty minutes shorter, and the team you would be joining lost its lead six weeks ago. You have been carrying this for two weeks. You have written the same two-column list four times, and both columns still hold up.

Notice what is missing from that scene: information. You are not short of it. The problem is that both answers survive scrutiny, the choice is not reversible at the same price, and if it goes wrong, nobody else signs for it. That is a hard decision, and it is the most common kind of decision that actually changes a person's year.

Now put a different kind of decision beside it. A support ticket arrives — billing dispute or account question? A tool call has to pick one of four data sources. A patch needs a yes or no before it merges. There are thousands of these a day, the answer is one of a set you defined yourself, and a wrong call costs a re-run.

Both of these are called "judgment". They do not live on the same axis.

The two axes

Every judgment sits somewhere on two independent quantities.

  • Latency — how fast and how cheaply you can get an answer. This is the axis the industry is racing on, and it is the axis Jev compressed to its physical limit.
  • Consequence — what you pay if the answer is wrong, and who pays it. This axis has no product category, no leaderboard, and almost no competition.

Confidence — the number both systems emit — is neither of these. Confidence tells you how much the judge knows. Consequence tells you how much you should care. A 0.99-confidence call on an irreversible decision is still an irreversible decision.

Jev's win is real, and it is a big one

TypeSafe AI shipped Jev on September 15, 2026. You send program state and typed questions, and you get back choices, scores and probabilities, computed in parallel, each with a confidence value — and not one word of prose. Latency runs 70–500 ms. Pricing is $0.042 per million input tokens with output free, which puts a million triage calls under twenty dollars. The founder, Diogo Almeida, was an OpenAI researcher and a co-author on InstructGPT and the RLHF work, and the company raised a $40M seed round led by DCVC.

The adoption numbers say the same thing. The launch post reached 1,800+ points on Hacker News the day it went up. Vercel's AI Gateway reported that within 24 hours of listing it, nearly 13% of its paid teams had already used it. Inside a week, traffic on X ran to roughly 37 million views, and community "Jev Router" projects had already appeared to let Jev itself decide which model should handle a request.

None of that is hype about nothing. Turning a judgment into a typed value that code can switch on is a genuine interface change, and cheap is cheap. If your problem is the thousands-a-day kind, Jev is the right tool and that argument is settled.

What it did not do is move the other axis. It made cheap decisions cheaper. It did not become the place where an expensive decision gets the treatment its cost implies.

The axis nobody is racing on

Here is the part worth sitting with. The consequence axis is not merely unoptimized. It has been declared solved by routing.

Jev's own documentation hands you the pattern: set a confidence threshold. Above it, act automatically. Below it, route to a human, or escalate to a more deliberate judge. That is good engineering, and as a policy it is exactly right. But look at where the pattern ends. The low-confidence branch is an arrow pointing at a box labeled "someone else". The millisecond layer has declared that it does not know, and the cost of that not-knowing has been moved off the model and onto a queue.

Three things follow, and none of them are about model quality.

Confidence is not consequence. Threshold routing sorts by how sure the judge is, not by what being wrong costs. Those are different quantities, and the mismatch runs in the dangerous direction: the more consequential a decision is, the more likely it is to be one where both options look defensible — which is precisely where confidence comes back low. Route on confidence alone and the hardest calls arrive with no plan attached to them.

Automating the cheap end makes the remaining end more expensive per case. This is a selection effect, not a paradox. If a system quietly absorbs 80% of your tickets, the person who used to see the whole distribution now sees the 20% the model was least sure about. Their job got harder, not easier, and the queue receiving that work has no new tooling. Judgment gets cheap at the volume end and stays expensive at the tail — and the tail is where the money and the exposure live.

The tail is where the minutes are. For the decisions that do reach a person, what is being bought is not a faster answer. It is a slower one that can be defended. That is what the extra time purchases: not a longer opinion, but the ability to say "this is the call, here is why, and here is how confident I am" — and to have that sentence survive a review six weeks later.

The end of the arrow

We build at the far end of that arrow.

Decider takes one hard question and two candidate answers, and returns which one is the stronger call, a calibrated confidence for it, and a 900–1,700 character argument for the pick. It takes tens of seconds rather than hundreds of milliseconds, and it is priced per decision rather than per token. We are not competing with a triage model, and we would be bad at triage: Decider.

On accuracy we claim nothing over a plain direct baseline, and we publish the number that shows why. On JudgeBench, self-run with the official protocol over 620 pairs, with the 6 pairs whose first verdict failed disclosed rather than imputed, we measured 92.5% against 92.2% for a direct model baseline. That is level. It is written that way on our own benchmark page, because any claim of a lead there would not survive one afternoon of checking.

What we do publish is the calibration, because that is what a person needs before acting on a pick. On that same self-run JudgeBench pass, calls we reported at 90% confidence or above came back right 99.6% of the time (283 judgments), and calls in the 80–90% band 94.0% (184 judgments). The benchmark, the method and the failures are disclosed, and the recorded runs are public. Separately, on ContextualJudgeBench — again self-run with the official protocol, over the full 2,000 pairs, with the 12 orders (0.3%) excluded after repeated platform failures disclosed rather than imputed — we scored 67.1% consistent accuracy against a 65.4% reference. "Consistent" means the same pick in both presentation orders, a rule baked into the protocol with a 25% random floor, on the grounds that an answer which flips when you swap A and B was never a judgment to begin with.

That is the honest version of what a few minutes buys: a pick, a number attached to it, and text you can argue with. If the article you were hoping for was "the slow judge is the smart one", this is not it. The slow judge is the one you call when the bill for being wrong is not a re-run.

A rule you can use

Sort the decision, not the model.

Send it to the millisecond layer when: the answer is one option from a set you defined; the outcome is reversible for a bounded cost; a downstream check would catch a wrong call anyway; being wrong is roughly as cheap as being right; and you will make this call thousands of times.

Escalate when: being wrong is not recoverable at the same price; a named person has to sign it; both candidates survive an adversarial reading; you will make the call once or twice and live with it for years; or you need to show the reasoning to someone who was not in the room when it was made.

The one-line version: ask what the wrong branch costs, and who pays it. If the answer is "another run", milliseconds are enough. If the answer is a quarter, a hire, or a relationship, you are on the second axis, and the minutes are the cheapest part of that decision.

And when the slow judge comes back with low confidence, treat it as a finding rather than a failure. A near-tie means the two candidates do not contain the answer; the deciding criterion is one you hold and have not written down yet. That is worth knowing before you sign — and it is the one output no threshold has ever produced.

Top comments (0)