Triage support tickets locally with a 449 MB decision model

Every support inbox asks the same three questions of every ticket: who should handle it, does the customer want their money back, and how much is on fire. The usual fix is to ask an LLM, write a parser for whatever it decides to say, and then write a second parser for the day it answers "Billing (probably)" instead of "billing".

NobodyWho's new decision model support for Flutter, React Native, Swift, Kotlin and Godot bindings, remove the parser from that story. A decision model scores answers you have already written. You give it the options up front, it returns a probability for each one, and it never generates a single token, so it has no way to invent a fourth department.

TypeSafe calls this kind of model a System One model, after the fast, intuitive half of Kahneman's Thinking, Fast and Slow, and describes its own model, Jev, as "unstructured state in, typed probabilistic decisions out". NobodyWho runs the same kind of decision on your own hardware, using Laya, an Apache-2.0 decision model whose Q8_0 GGUF file is 449 MB.

Load the model

Install NobodyWho with pip install nobodywho, then point a Model at the Laya file on Hugging Face, or at a local GGUF path if you already have it.

from nobodywho import Decision, Model

model = Model(model_path="hf://ggml-org/Laya-GGUF/Laya-Q8_0.gguf")
decisions = Decision(model=model, n_ctx=1024)

Ask the ticket three questions

Each question has an ID, a type and plain-English instructions. A choice question picks one option, a noul question returns the probability that the answer is yes, and a score question places the ticket on a scale you define, from two to ten levels.

message = "I was charged twice. Please refund the duplicate payment."

questions = {
    "department": {
        "type": "choice",
        "instructions": "Which team should handle this?",
        "criteria": {
            "billing": "Payments and refunds",
            "technical": "Bugs and login problems",
        },
    },
    "refund": {
        "type": "noul",
        "instructions": "Is a refund requested?",
    },
    "urgency": {
        "type": "score",
        "instructions": "How urgent is this?",
        "criteria": ["can wait", "soon", "urgent"],
    },
}

result = decisions.decide(state={"message": message}, questions=questions)

team = result["answers"]["department"]["choice"]
probabilities = result["answers"]["department"]["probabilities"]
refund_probability = result["answers"]["refund"]["noul"]
urgency = result["answers"]["urgency"]["score"]

assert result["usage"]["output_tokens"] == 0

That last line is the whole pitch in one assert. The model read the ticket, answered three questions and wrote nothing.

What actually comes back

NobodyWho's test suite runs a very similar ticket through Laya: "I was charged twice for my order last week and nobody has replied." It asks which of billing, shipping or technical should take it, whether the customer is angry, and how urgent it is on a four-step scale from "can wait" to "right now". These are the reference answers the suite checks against on CPU, rounded:

{
  "route": {
    "choice": "billing",
    "probabilities": {"billing": 0.975, "shipping": 0.014, "technical": 0.011},
    "confidence": 0.962
  },
  "angry": {"noul": 0.784},
  "urgency": {
    "score": 1.914,
    "legend": {"0": "can wait", "1": "this week", "2": "today", "3": "right now"},
    "probabilities": {"0": 0.040, "1": 0.458, "2": 0.050, "3": 0.452},
    "confidence": 0.006
  },
  "usage": {"input_tokens": 166, "output_tokens": 0}
}

The routing answer is as clean as it gets: billing at 97.5%, with a confidence of 0.96. The model also puts the chance that this customer is angry at 78%, which anyone who has been charged twice will find conservative.

The urgency answer is where it gets interesting. A score is a weighted average of the levels, and 1.914 rounds neatly to "today", but the model only gives "today" 5%. What it actually believes is that this ticket is either a "this week" problem (45.8%) or a "right now" problem (45.2%), and the confidence of 0.006 is its way of telling you it can't pick. If your code only read the score, it would file the ticket under an answer the model gave a 5% chance.

Send the unsure ones to a person

Choice and score answers both carry that confidence value, from 0 to 1, which measures how concentrated the probabilities are. That gives you a simple rule for triage: route confident answers automatically and put the rest in front of a human.

answer = result["answers"]["department"]
queue = answer["choice"] if answer["confidence"] >= 0.6 else "human-review"

Applied to the reference output above, the route goes straight to billing at 0.96 and the urgency goes to a person at 0.006, which is exactly the split you want. Treat 0.6 as a starting point to tune against your own tickets, and keep an eye on the queues for the first week, because the docs are clear that high confidence does not guarantee a correct answer.

Running it in your app

DecisionAsync gives you the same API with await, which suits a web handler that triages tickets as they arrive. Decision models are also in the Flutter, React Native/Expo, Swift, Kotlin and Godot bindings, so the same questions can run inside a mobile or desktop app, and the ticket text never leaves the device.

The full Python guide can be found here.

If this saved you a parser, a star on the repo is the easiest way to tell the team to keep going ❤️