A Decision Model That Writes Nothing Just Took #1 on a 137-Benchmark Board

The result first

Darwin-27B-ZTC-v9 is #1 on the S1MB english-v1 leaderboard, read from the live board on 20 October 2026.

Rank	Model	Borda
1	Darwin-27B-ZTC-v9	90.38
2	kettei-fat v1.0	90.14
3	Superion S1	89.53
4	D3 (27B)	89.28
5	Darwin-27B-ZTC-v5	88.44

197 models, 137 benchmarks. Weights are Apache-2.0.

Before anyone quotes that number, the rest of this post explains what it does and does not mean, because "first on a leaderboard" is close to meaningless without the aggregation method attached.

What Borda aggregation actually measures

Most boards average a score over tasks. S1MB does not. It ranks each of the 137 benchmarks separately, then aggregates the ranks. That is a Borda count.

The difference bites. Under averaging, a benchmark whose metric happens to spread from 0.1 to 0.9 contributes far more to your average than one packed between 0.80 and 0.84. Nobody chose that weighting. It fell out of metric scaling. A model that is ordinary everywhere and extraordinary on three wide-spread tasks can land near the top.

Under Borda every benchmark contributes the same thing: your position among the entrants. Winning by 0.3 and winning by 0.001 earn the same rank. To finish first you have to place well on most of the 137 boards simultaneously.

What Borda gives up is margin. A board where the top five differ by a hair on every task still produces a confident-looking order. So read both columns. S1MB publishes a task average next to Borda, and on the current English board they disagree: this model is second on task average at 72.20, while the highest task average belongs to the model sitting third on Borda. Neither column is wrong. They answer different questions.

The model generates no text

ZTC stands for zero token choice. You give it a situation and a typed question. It returns a probability for every option in one forward pass and writes nothing.

Type	Returns	Example
noul	Probability the statement holds	Does this ticket need a reply today?
choice	A probability per option	Which team should own this?
score	A probability per level	How satisfied is this review, 1 to 5?

No output to parse. No format that can drift on the thousandth call. No reasoning tokens to wait through.

Calibration is the property that matters

Here is the part teams underrate. Accuracy alone does not let you build a gate.

A gate needs a threshold, and a threshold needs probabilities that mean what they claim. If 0.9 is right about nine times in ten, you can auto-approve above 0.9, route the rest to a human, and predict your error rate before shipping. If the model is accurate on average but its 0.9 means anywhere from 0.6 to 0.99, no threshold is safe.

This is why decision benchmarks score calibration directly. Brier score for yes or no, probability mass on the correct option for choice, expected score error for scales. A model can top an accuracy table and be useless as a gate.

Selected per-task results against the published baseline:

Task	Baseline	v9
boardgameqa-choice	0.500	0.953
snli-choice	0.790	0.968
gretel-pii-choice (entity type)	0.653	0.981
argument-quality-choice	0.857	0.959
ud-ewt-choice (part of speech)	0.433	0.833

What cheap checking changes

Count where an agent's tokens go. A large share is the agent asking itself small questions. Is this tool call safe. Does this passage answer the question. Did the last step succeed. Should this escalate.

Every one of those is a yes, a pick, or a score, and every one currently costs a full generation. At a thousand a minute the checking costs more than the work, so teams quietly drop the checks. Not because they decided the checks were unnecessary, but because they were unaffordable.

Make the check one forward pass and the calculus flips. Gate every tool call. Score every retrieved passage. Confirm every step. Keep a confidence number on each decision so escalation is a threshold rather than a guess.

Serving it

hf download FINAL-Bench/Darwin-27B-ZTC-v9 systemone_config.json --local-dir .
transformers serve FINAL-Bench/Darwin-27B-ZTC-v9 --decision-config systemone_config.json

It runs on the standard /v1/systemone endpoint of Hugging Face transformers serve, so clients written against that public shape work unchanged.

How to check this claim

1. Open the board and read the column and date yourself. It is live and 197 entries move.
2. Note that most public boards mix self-reported and verified entries, and S1MB says so.
3. Best test: run your own decision log through it and see whether the probabilities hold at the threshold you intend to use. Open weights make that a one-afternoon experiment rather than a procurement decision.