Local 144.3M or a hosted API for your approval gate? Three questions and one honest table.

Half of what my coding agent "thinks" about overnight isn't thinking. It's permission questions: can I run this command? should this ticket go to billing or support? is this output safe to post? Each one is a small, structured decision — and if you serve each one with a chat API, you pay in three places: tokens per call, a network round trip, and the decision context leaving your machine.

Cloudflare's Clef release earlier this month put decision models on a lot of radars, and the community write-ups on dev.to are worth reading. This post is about the other lane: a small decision model you run yourself. I'll use Phocinae-Largha-150M-v1 — 144.3M parameters, Apache-2.0 — as the worked example, and show the selection logic I use for an approval gate: what stays local, what still goes to a hosted model, and the numbers behind both, with their caveats attached.

What a decision model actually does

It doesn't chat. You give it a state (a short description of the situation) plus typed questions — noul (yes/no), choice (pick one of N), score (2–10) — and one forward pass returns an answer per question with a confidence. The answer set exists before the call: nothing is generated, so nothing drifts. In the model card's own usage example, a decision reports input_tokens: 146, output_tokens: 0 — zero output tokens, because there is no output text.

That's the point for a gate: deterministic, auditable, replayable decisions — the card's own phrasing.

Three questions that pick the lane

1. How often will this decision fire? If the same shape of decision comes up a few dozen times a day, a hosted API call is a fine trade — rare, varied, convenient. If it fires thousands of times a day — approvals, triage, screening, routing — the arithmetic changes shape. A local forward pass has no per-call fee; the cost moves to hardware you control, and it doesn't grow with volume.

2. Can the input leave the machine? Approval gates see command lines, file paths, log tails, sometimes customer content. If policy — or common sense — says that context stays on the box, the decision has to be made on the box. This is the hardest filter of the three, because there's nothing to negotiate.

3. Is the answer set closed? If the possible answers are enumerable in advance — allow / ask / deny, queue A or B, a 2–10 severity score — a typed decision model fits, and generating prose to pick from a closed list is waste. If the answer is genuinely open-ended — "write the incident summary", "explain this stack trace" — keep the big model. The dividing line isn't model size; it's whether the answer exists before you call.

One practical footnote before the numbers: latency and hardware. On GPU fp16 the published p50 is 21.0 ms per decision (RTX 5090). On CPU it's slower — 1.64 s per case (one state plus five questions), single-threaded — or 8–20 decisions/second with 8 threads. That's background-gate territory, not interactive-typing latency. And temper the "local is automatically faster" instinct: in the third-party benchmark below, the hosted API measured 225 ms p50 while the author's own local GPU setup measured 315 ms p50 on a 3090. Local's wins are about owning the box and skipping the network — not about the model being small.

The numbers, including the awkward ones

Same-protocol readings, each row carrying its own annotation — all from the public model card:

reading (typed decisions)	value	as published
Largha-150M-v1, en	0.906	400 cases / 2000 decisions; specialist fitted on that dataset's train split; the card notes this sits above the 0.735 teacher self-agreement reference, and the dataset card flags far-above-reference scores as label-specific overfitting risk.
Laya	0.766	self-measured, native interface
JEV	0.727	generalist, zero-shot
meraGPT	0.768	generalist, zero-shot

Lineage note: the fine-tune mix includes a 500-item error-correction pool mined from model predictions on typed-decisions test rows — full details in the card's Training-overlap disclosure (corrected 2026-10-10). We report 0.906 as a fitted reading, not a zero-shot result.

And the awkward one, in the same breath: JevBench public-231 reads 0.5455 (126/231) — below its 58.4% acceptance gate, not passed. That number comes from a public 231-item set measured in a different environment and protocol than the 400-case self-eval above — the two don't merge. The card publishes it as measured, and those JevBench rows were held out of training.

There's a zh reading too: 0.848 — measured on machine-translated cases, and the training mix itself includes ≈2,400 machine-translated Chinese rows alongside ≈1,400 native ones. Treat it as a fitted Chinese result, not zero-shot transfer.

For outside evidence, the most useful public write-up I've found so far ran 42 real decisions from one person's overnight coding agent: Jev (hosted) at 71.4% overall vs Clef-Flash (9B, run locally on his 3090) at 66.7% — third-party numbers, his protocol, link below. His framing is the one worth keeping: the exercise doesn't tell you which model is best; it tells you which calls you're wasting money on.

Same post, for scale contrast: Clef-27B in bf16 is estimated at ~54 GB of VRAM (a third-party estimate), while Largha's weights are 288.6 MB with a 1.6 GB inference peak, and the project's deployment doc lists a 4 GB-RAM, no-GPU floor. Those are lane differences — different machines for different jobs — not a verdict.

Mix, don't replace: the escalation gate

The design that keeps both sides honest is a confidence gate. Every decision comes back with a calibrated confidence; at τ=0.6 the gate escalated 45.0% of official-set decisions and cut LLM calls by 55.0%, with kept-subset accuracy moving 0.906 → 0.9936 (full routed accuracy 0.8135). The fine print is on the card, verbatim: "τ swept on the eval set — re-scan per domain." Your traffic will want its own threshold; tune it and re-measure.

So local vs hosted isn't a fork. The small model takes the steady state, and the hosted model gets exactly the calls the gate flagged as uncertain, novel, or high-stakes. Escalation isn't failure — it's routing.

One honesty detail that matters if you build on the gate: on the card, the shipped confidence column carried an ECE of 0.2519 (en); the bundled recalibration column brings it to 0.0168. If you set thresholds from confidence, recalibrate first — the project ships the column and the temperature constants to do it.

Read these before you pick

• Tool routing is the weak spot. The project's server confuses semantically close tool names — the card's reproducible case returns fetch_url instead of web_search at ≈0.33 confidence. Low-confidence picks are escalate-worthy, never final.
• Not a safety oracle. The card's words, and they matter at a gate: first-line signal with escalation — pair it with your own rules and a human path, never as the sole guard.
• Long-document reasoning is out of scope. Base context is 8192 tokens, and "not for long-document reasoning" is on the card's "what it is not" list. Keep gate inputs to states and questions, not articles.
• These are single-setup measurements. Latency and quality numbers come from the release run; your hardware and your traffic will differ, and they deserve their own measurement.
• The card self-corrects in public. A training-overlap disclosure was corrected on Oct 10, and the current numbers were re-measured on the current weights with a revision history. Read the disclosures before citing any single number — including mine.

What's worth trying (and what isn't)

Try a local gate when:

• the decision is structured and repetitive — command approvals, ticket triage, output screening, document routing (the project's public gallery has 27 scenario demos along these lanes);
• the inputs can't leave the machine — on-prem, air-gapped, regulated context;
• the volume makes per-call fees the wrong shape, or you want the steady state to work offline;
• or you want a first-line gate that escalates its uncertain cases instead of answering everything.

Keep a hosted model when:

• the answer is open-ended text — summaries, explanations, long-document reasoning;
• the decisions are rare and varied, with no pattern to amortize;
• or the call is the escalated tail — the part the gate explicitly refused to guess on.

pip install phocinae-server
hf download Phocinae/Phocinae-Largha-150M-v1 --local-dir ./largha
PHOC_MODEL_DIR=./largha python -m phocinae.main   # 127.0.0.1:8155

Third-party benchmark referenced above: I benchmarked Cloudflare's new open decision model against the hosted API it's trying to replace — 42 cases, his protocol, his caveats; read it before quoting my one-line summary of it.

Model card and full numbers

If you try it, the failure report I'd most want to read is in the tool-routing caveat above — that's where it breaks first. And if your selection math lands somewhere else, that's the interesting part: the gate is supposed to be re-scanned per domain, including yours.

Transparency: this post was drafted with AI assistance; every number above comes from the public sources linked here, and no new benchmarks were run for this piece.