A lot of the LLM calls inside an agent aren't about generating text. They're tiny decisions: which team owns this ticket? Did the user actually say that tool argument, or did the model make it up? Is this message urgent? Cheap model or expensive model? For each one, we ship the full context to a frontier model, wait a second or two, and parse free text that comes with no measure of how confident the model is.
Strands Labs, the experimental arm of the Strands Agents team at AWS, released Strands Decider 2B, an Apache 2.0 model built for exactly that gap. It doesn't generate text. It takes a state plus a set of questions and, for each one, picks from closed options and returns a calibrated probability. It runs locally, answers in about 115 ms median on a desktop GPU, and plugs into Strands Agents' intervention system to review tool calls before they execute.
TL;DR
- A decision model (the authors call it a "system one" model): picks options, answers yes/no, or scores on a scale, always with a probability. No text generation.
- Architecture: the Qwen3.5-2B-Base torso with its LM head removed, a ~1M-parameter pointer head, and rank-16 LoRA adapters. One forward pass, no decoding loop.
- First release latency: 115 ms median (299 ms p95) on an RTX 3090; 153 ms on an M3 Pro.
- On JevBench (231 public tasks) the current reference model gets 180 right (0.779). Gemma 4 variants reach 207 with the 26B model.
-
pip install strands-decider, then use it from the CLI or as a local HTTP server (/v1/systemone). - It's an experiment, not a managed AWS service. You run it.
Why now
The "decision model" idea got attention in September when TypeSafe AI came out of stealth with Jev, a proprietary model that returns typed decisions instead of text, available via waitlist at $0.042 per million input tokens. Strands credits Jev for putting the approach on the map and evaluates against JevBench. The difference is distribution: Jev is closed; Strands Decider ships weights, code and the full training recipe.
There's also a design argument. Ask a chat model "are your own tool arguments grounded?" and it can be talked into yes. A classifier that returns a number your if statement branches on doesn't negotiate.
How it works
The decider keeps the transformer torso of Qwen3.5-2B-Base, throws away the vocabulary head, and replaces it with a pointer head. The input has a fixed format: state, question, numbered options, and an <answer> marker. The head scores each option's final hidden state against the hidden state at <answer>, a softmax turns those logits into probabilities, done. No tokens generated.
Two practical consequences: the head has no per-option parameters, so it can't learn "option 1 is usually right" and you can rename options without retraining; and only the LoRA adapters and the head are trained, which is why the repo can publish the recipe (about 11 hours on one RTX 3090, or ~70 minutes on eight H100s).
Three question types are supported: noul (yes/no, returns P(yes)), choice (one of N), and score (a level on an ordered rubric). You can ask many questions about one state in a single request; the state is encoded once.
| Model | JevBench (of 231) | Brier (lower is better) |
|---|---|---|
| 2B hobson-v19 (launch) | 167 | 0.342 |
| 2B hobson-v21 | 176 | 0.323 |
| 2B qwen3.5-v1-2610 (reference) | 180 | 0.280 |
| E4B gemma4-v1-2610 | 187 | 0.243 |
| 26B-A4B gemma4-v1-2610 | 207 | 0.162 |
The README itself warns that gaps under ~10 tasks between single runs aren't reliable. For v19, Techstrong.ai reports 100% on easy tasks and 50.5% on hard ones: great at routine, mediocre at complex, which is the pitch.
Hands-on
pip install strands-decider
strands-decider ask StrandsAgents/strands-decider-2B-qwen3.5-v1-2610 \
--state "Help! My payouts have been failing for 3 days! " \
--choice "Which team should handle this?=billing,sales,retail" \
--noul "Does this convey urgency?" \
--score "How frustrated is the writer?=calm,frustrated,depressed"
Sample output from the README (CPU run): billing at 0.773, urgency 0.858, and a frustration score of 1.03 with confidence 0.509. That last number is the model telling you it isn't sure, which is information an LLM doesn't give you.
As a server:
strands-decider serve StrandsAgents/strands-decider-2B-qwen3.5-v1-2610 --port 8000
curl -s localhost:8000/v1/systemone -H 'content-type: application/json' \
-d '{"state": "Help! My payouts have been failing for 3 days!",
"questions": {"is_urgent": {"type": "noul", "instructions": "Does this convey urgency?"}}}'
The server binds to 127.0.0.1 and has no authentication. Don't put it behind a public Service.
Gating tool calls in a Strands agent
This is a trimmed version of examples/strands/ from the repo. The agent has an eager system prompt that makes it guess a city; the decider catches it:
from strands import Agent, tool
from strands.interventions import Guide, InterventionHandler, Proceed
YES = 0.45 # illustrative threshold; tune it on your own traffic
class ToolCallReviewer(InterventionHandler):
name = "strands-decider-tool-call-reviewer"
def __init__(self, decider):
self._decider = decider
def before_tool_call(self, event, **kwargs):
state = build_state(event.agent.messages, event.tool_use) # conversation + proposed call
answers = self._decider.ask(state, QUESTIONS) # args_grounded, premature (noul)
if answers["args_grounded"]["noul"] < YES:
return Guide(feedback="The arguments look guessed. Ask the user to confirm them.")
if answers["premature"]["noul"] >= YES:
return Guide(feedback="Too early to call this tool. Clarify with the user first.")
return Proceed()
Guide doesn't block the call; it hands the turn back to the model with feedback, and the agent ends up asking "which city?" instead of reporting Seattle's weather. Strands interventions also support Deny and Confirm (human approval).
Running it on Amazon EKS
Because the server only listens on localhost, the natural Kubernetes pattern is a sidecar in the agent's pod. A native sidecar (an init container with restartPolicy: Always) starts before the agent and lives as long as the pod. Two gotchas: there's no published container image, so build your own and bake the weights in; and httpGet probes will always fail because the kubelet hits the pod IP, not 127.0.0.1, so use an exec probe against http://127.0.0.1:<port>/health. A g6-family node (NVIDIA L4, 24 GB) is plenty for the 2B model.
Cost
The model is free; you pay for hardware. A g6.xlarge is about $0.805/hour on demand in us-east-1, roughly $588/month always on. Back-of-the-envelope (mine): ~8 sequential decisions/second at ~115 ms is on the order of 20M decisions a month; at 500 tokens each, that volume on a per-token service like Jev would be around $450. Self-hosting wins with sustained volume or privacy requirements; at low volume an idle GPU costs more. Start on CPU, share the GPU, and use the decider as a first-pass filter that escalates low-confidence cases to the LLM.
Limitations worth knowing
Per the model card: rephrased questions often get the same answer, long multi-step documents are the weak spot, score and noul transfer poorly to unfamiliar rubrics, and calibration was fitted on short classification, so validate thresholds on your own traffic. Training data and examples are English.
My take
The benchmark isn't the point (231 tasks is noisy). The point is the separation: the model classifies and returns a number, your code decides. That turns decisions hidden inside prompts into something you can test, version and audit with explicit thresholds. It's the same logic platform teams apply with Kubernetes admission policies: a deterministic checkpoint before something runs. It's not production-ready, and the official example says its thresholds are illustrative. But I'd run it as a tool-call guardrail for an internal agent, in a pod with a shared GPU, and compare its decisions against what your LLM decides today.
Originally published in Spanish on CloudAcademy.ar, with the full Kubernetes manifest and the complete intervention example.
Top comments (0)