Hugging Face published a post describing decision models and the tooling it has added around them. The aim is a common interface that any such model can implement, whatever architecture sits underneath.
What a decision model does
A decision model receives a state, which can be a string or a structured JSON object. It also receives one or more named questions, each with a fixed set of allowed answers. It returns a typed decision plus probabilities, and it generates no text along the way. The sample response in the post reports zero output tokens.
There are three question types:
- Choice picks one option, such as which team should handle a ticket, and returns a probability for every option.
- Score rates something on an ordered scale, such as low, medium or high urgency, and returns the expected value of the level.
- Noul, named after Bernoulli, answers a yes/no question with a single probability, for example whether the customer is asking for a refund.
Choice and Score answers also carry a confidence value that shows how concentrated the probabilities are.
The post's running example is a support message about a double charge. One request can ask for the department, the urgency and the refund intent at once. The department comes back as billing with a probability close to one.
How it differs from what you may already use
The post contrasts decision models with two older approaches. A conventional classifier has its labels fixed in the model, so adding a category means retraining. Zero-shot classification takes labels at request time but handles one kind of question. Hugging Face says decision models did not invent runtime labels or probabilistic output. What they add is a single interface that covers choices, ordered scores and yes/no questions together.
The post also lists the weaknesses of asking an LLM for JSON. The output may not follow the schema, constrained decoding still adds computation, and generating a whole response is extra work when the application only needs to pick an option.
What shipped
- Transformers gains a decision pipeline, an AutoModelForDecision class and a DecisionConfig class. It can also serve a /v1/systemone endpoint locally.
- llama.cpp serves GGUF decision models through the same /v1/systemone API.
- The Hub has a decision-model tag. The models listed use different backbones, from ModernBERT-style encoders to larger generative ones.
- Training: decision models can be fine-tuned with the Trainer API. The post shows a deliberately small example that routes support requests with a cross-entropy loss over the options.
The post adds that compatible autoregressive language models can also be run through the decision interface. Transformers builds the prompts, reads the logits for the answer labels and applies softmax without generating text, so the interface can be tried without training a new model.
Caveats
Hugging Face calls decision models a young, fast-moving space, and the post gives no benchmark comparison with LLM prompting. The examples use models published under the autotrust and ggml-org names. Accuracy on your own tickets is untested until you run it.
Top comments (0)