You Don't Need an LLM for Every Decision

Most of the AI calls in my code never needed words. They needed a choice.

Route this ticket. Is this chunk useful. Should we retry. Is this tool call safe. All of them went to a text model. The model wrote tokens. My code parsed the JSON, checked it, and then a normal if made the real decision.

That is a lot of work for a yes or no.

Last week a new kind of model appeared. It does only the decision part. I tried it for a few days. Here is what it is, where it helps, and where it does not.

All examples are JavaScript, and I kept them small on purpose.

What a System One model is

TypeSafe released Jev on September 15, 2026, in early access. They call it the first public System One model.

The idea is simple. You send your application state. You send typed questions. You get back probabilities. There is no text. A normal LLM writes JSON one token at a time. This one returns typed answers directly.

All the answers come back in a single forward pass. That is why it is fast.

There are three question types:

Type	You give	You get
noul	a yes or no question	the probability of yes
choice	instructions plus named options	the chosen option, probabilities, confidence
score	instructions plus ordered levels	a number, probabilities, confidence

choice accepts up to 255 options. score accepts 2 to 10 levels.

The smallest example

Install it and run it. You need Node 20 or newer.

npm install @typesafe-ai/sdk
export TYPESAFE_API_KEY="your-key"

One question. One answer.

import { noul, TypeSafeClient } from "@typesafe-ai/sdk";

const client = new TypeSafeClient();

const response = await client.systemOne({
  state: "I was charged twice. Please fix this.",
  questions: {
    isBilling: noul("Is this about billing?"),
  },
});

console.log(response.answers.isBilling.noul); // 0.97

That number is the probability of yes. Nothing else comes back.

Now a question with options.

import { choice, TypeSafeClient } from "@typesafe-ai/sdk";

const response = await client.systemOne({
  state: ticketText,
  questions: {
    team: choice("Which team should read this?", {
      billing: "Payments and invoices",
      support: "Bugs and how-to questions",
      sales: "New customers and pricing",
    }),
  },
});

console.log(response.answers.team.choice); // "billing"
console.log(response.answers.team.confidence); // 0.88

The types come from the questions you wrote. No schema file. No parser. No retry when the JSON is broken.

You can also send several questions at once. They all travel in one request, which matters for the cost numbers below.

const response = await client.systemOne({
  state: ticketText,
  questions: {
    isBilling: noul("Is this about billing?"),
    isAngry: noul("Is the customer angry?"),
  },
});

Case 1: put it in front of your model, not instead of it

This is where I would start. The common advice at launch was not to replace your model. Let the small decision happen first, so the expensive model runs only when you really need it.

const gate = await client.systemOne({
  state: userMessage,
  questions: {
    needsWriting: noul("Does answering this need a written reply?"),
  },
});

if (gate.answers.needsWriting.noul < 0.5) {
  return sendCannedReply(userMessage);
}

return await callClaude(userMessage); // the expensive path, now less common

That 0.5 is my code, not the model. It is an example value, not a measured or recommended one. Choose yours from your own data.

Some numbers were reported from an email tool that made this change. Two AI calls per email became one decision call. A test with 120 tickets took 42 seconds instead of 22 minutes, and cost $0.0003 instead of $0.059. The person who ran the test said his slow version was also slowed down by retry errors, so the real gap is smaller.

Read that as a direction, not as a benchmark.

Case 2: check the output before you send it

This is the shape I liked most. The LLM writes the reply. The small model checks it.

const draft = await callClaude(prompt);

const check = await client.systemOne({
  state: { question: userMessage, draft },
  questions: {
    isGood: noul("Does the draft answer the question?"),
  },
});

if (check.answers.isGood.noul < 0.8) {
  return sendToHuman(draft);
}

return send(draft);

Use an object for state when you have more than one piece of context, like here. A plain string is fine when you have one.

The official advice is to ask only for judgments that a person who knows the topic could make in about a second. If a question needs slow thinking, split it into smaller questions and combine the answers in your own code.

That rule is the whole design. The model judges. Your code decides.

Case 3: check relevance before you build the prompt

This one is cheap and people talk about it too little. Before you put twenty retrieved chunks into a prompt, rate them.

Build the questions in a loop, one per chunk.

const questions = {};

chunks.forEach((chunk, i) => {
  questions[`chunk_${i}`] = noul("Does this passage help answer the question?");
});

const result = await client.systemOne({
  state: { question: query, passages: chunks },
  questions,
});

const keep = chunks.filter((chunk, i) => {
  return result.answers[`chunk_${i}`].noul > 0.6;
});

Send many questions at once, then filter in code. The prompt that reaches the expensive model gets smaller, and that is usually where the bill comes from.

If you want more than yes or no, score gives you levels instead.

score("How relevant is this passage?", ["not at all", "a little", "a lot"]);

There is already an unofficial LlamaIndex integration that does this. The model rates the retrieved passages and picks which tool answers a query. Note that score mode is a rubric, not cosine similarity.

Case 4: when the data cannot leave your servers

For agency work this is the important part. I would rather not explain to a client why their data goes to a third party on every routing call.

Jared Palmer, who built Turborepo and v0, released Kev. It is an open source family that follows the same interface. There are three models, 0.8B, 4B and 9B, and you can train, inspect and run them on your own hardware under Apache-2.0. The server, the weights and the training code stay local.

So the change is a base URL. The SDK appends /v1/systemone to it.

const client = new TypeSafeClient({
  baseURL: "http://localhost:8000", // your own server
});

Everything else in your code stays the same. You can also skip the constructor and set TYPESAFE_BASE_URL in the environment, so the same code runs against the hosted model in development and against your server in production.

You pay for this in accuracy. On the published numbers, Kev-9B gets 0.837 on the locked test set, against 0.857 for the hosted model. The cost of building it still surprises me. The whole port to Qwen3.5 was about $95 of Modal H100 time plus three cents of API calls.

Latency is not free either. Kev-4B on an Apple M5 took 779ms for five questions, up from 174ms on the earlier Qwen3 version. Measure it on your own hardware first.

Why this is worth an afternoon

1. You delete the parsing layer. No schema prompt, no JSON repair, no error branch that nobody tests.
2. The options are yours. You write the list of possible answers in your own code. The model does not invent them.
3. You get real probabilities. Confidence is a number you can compare against a threshold. That is how you build a safe fallback.
4. One request, many questions. You can classify, score and gate a ticket in a single call.
5. You can undo it. Put it behind a flag, keep the old call, run both for a week, compare, then delete.

The ecosystem is already usable. There are provider integrations for Cloudflare AI, the Netlify AI Gateway, OpenRouter and the Vercel AI Gateway. On the JavaScript side there are integrations for LangChain.js and Effect. In the Vercel AI SDK, the Boolean type is what TypeSafe calls Noul. Knowing that will save you ten minutes.

Honest caveats

I like this, and I am still not putting it in a client's critical path this month.

• It is one week old. Early access, launched September 15, 2026.
• The launch numbers are marketing numbers. The claims of 20 to 200 times faster and 40 to 400 times cheaper came from launch posts. The most careful writeup I found says clearly that it is an analysis of the documentation and architecture of Jev 1.13, not an independent latency test and not a production deployment.
• The model is uneven. The TypeSafe docs list limits around arithmetic, dates, distractors, adversarial state and inconsistent structure. Do not ask it to do math.
• The training data is synthetic. That is confirmed, and it is 100% synthetic. The architecture is still a guess from outside. The best guesses are ModernBERT and diffusion.
• The open models are not the same model. The Kev README says clearly that no outputs from the hosted model were used for training. That is good practice, and it also means you get the same interface, not the same quality.
• Confidence is not accuracy. Choice and Score report confidence separately from the answer probability, and Noul has no confidence field at all. Read that page before you write a threshold.

What I would do next

1. Search your code for AI calls whose output only ever feeds an if. Those are your candidates.
2. Pick the one with the most traffic. Ticket routing, moderation, relevance filtering.
3. Write the list of options in your own code, not in a prompt.
4. Put the decision call in front of the existing call, behind a feature flag, and keep the old path as a fallback.
5. Run both for a week and compare on your data, not on the launch benchmarks.
6. If the data cannot leave your servers, run Kev locally and point the SDK at it.
7. Only then delete the old call.

Here is the idea I keep coming back to. An agent is a long chain of small judgments with a little writing at the end. We have been paying the writing price for the whole chain.