Determinism as a feature: when to let your agent call a math API instead of reasoning

Disclaimer: This article was drafted with AI assistance and reviewed and edited by the author. The technical design and opinions are my own.

LLM agents are great at deciding what to do and unreliable at computing it. Ask one to allocate traffic across five variants, price tail risk, or solve a scheduling constraint and you'll get a confident, plausible, subtly-wrong number — tokens burned included.

The fix usually isn't a better prompt. It's the same instinct that gave us the calculator: move the deterministic math out of the probabilistic engine.

The tell

You have a determinism problem the moment your agent's output needs to be:

• reproducible — same inputs → same answer, every run,
• auditable — someone can check why it's 0.62 and not 0.61, or
• correct under adversarial inputs — a fat-tailed return, an infeasible constraint.

An LLM gives you none of those for free. A tool call does.

What to offload (and a cheap test for each)

1. "Which variant should I ship?" → a multi-armed / contextual bandit. The agent picks the question; Thompson sampling picks the allocation. Test: ask your agent to allocate 1,000 users across 4 arms with the same conversion counts, twice. Different answers? Offload it.
2. "Is this metric anomalous?" → score the series against a baseline; don't eyeball it inside the context window.
3. "What's the 95% VaR / CVaR?" → Monte Carlo paths, not a vibe.
4. "Schedule these tasks under these limits" → an LP/MIP solver. LLMs can't reliably satisfy hard constraints; solvers can't violate them.

The pattern

Expose the math as MCP tools so the agent calls them like any other tool — intent stays in the model, the number comes from code:

// agent decides intent; the tool computes the answer
const alloc = await callTool("optimize_contextual", {
  arms: variants,          // [{ id, name }]
  context: userFeatures,   // segment, prior_open_rate, hour_of_day
  history: pastRewards
});
// `alloc` is reproducible, sub-millisecond, and you can show your work

Two design details that bite people:

• Delayed reward. If reward trickles in (email opens over hours), set a fixed attribution window before crediting an arm — otherwise the bandit over-exploits early openers and collapses variant diversity.
• Cold start. Start each arm on a Beta(1,1) prior (or an informed prior from past campaigns) so exploration doesn't die on run one.

When not to offload

Determinism is a constraint, and constraints have cost. If the task is genuinely fuzzy — summarizing a doc, routing an intent, drafting copy — keep it in the model. A rule of thumb that's served me well:

If you'd want a unit test for the output, it belongs in a tool, not a prompt.

---

If you want a batteries-included set of these as MCP tools — bandits, forecasting, Monte Carlo, optimization, anomaly/risk — I maintain OraClaw (npx -y @oraclaw/mcp-server; 11 of the tools are free, no key). But the pattern matters more than the tool — wire in whatever solver you like. Disclosure: I built it.

Comments

[Armorer Labs]

Determinism is also an audit feature.

When an agent calls a deterministic API, you can store the input, output, tool version, and result in a way that is much easier to replay than a reasoning-only answer. That matters when the output later affects a write, approval, or user-visible decision.

I like this pattern broadly: let the model choose when appropriate, but move anything factual, calculable, or state-changing through tools that leave receipts.

[Whatsonyourmind]

"Leave receipts" is the right phrase. I'd push it one notch past replay: if the tool returns a certificate — inputs, tool/solver version, plus a verification artifact (a content hash, or for an optimizer the optimality/KKT conditions) — the receipt stops being something an auditor has to trust and becomes something they can independently recompute to a byte-identical result. "We logged it" → "anyone can re-run it and get the same answer." A reasoning-only step can't offer that no matter how thorough the logging, which is exactly why moving the factual/state-changing parts through deterministic tools is the audit win you're describing.

[Armorer Labs]

Exactly. The part I would add is that the certificate should sit next to the agent's decision boundary, not only inside the tool's logs.

If the agent chose not to call the deterministic tool, or called it with lossy inputs, the receipt should make that visible too. In practice I like three checks: can the artifact be recomputed, can the operator see why the tool was selected, and can a later run prove it used the same tool/version/policy.

That turns audit from "read the transcript" into "verify the chain of custody for the state-changing step."

Disclosure: I work on Armorer Labs.

[Whatsonyourmind]

@armorer_labs This is the sharp version of it -- signing the decision, not just the action. The "chose not to call the tool" case is hard precisely because it's an absence, and you can't sign an event that didn't happen. What I keep landing on is emitting the selection decision inline at each step: the candidate tool set plus the selector's verdict ("determinism-eligible=true, chose to reason anyway, reason=..."), so "didn't call the math API" becomes a positive signed record rather than a silent gap in the transcript. Absence becomes presence.

For the lossy-inputs case, binding the call to a hash of the exact inputs passed -- and requiring that hash to chain back to the upstream artifact it was derived from -- makes degraded inputs detectable: if the input hash doesn't descend from the source, the receipt flags it instead of the operator having to catch it. Stack that on your third check (pinned tool + version + policy) and you get an unbroken recompute chain: source -> inputs -> outputs, each hop hashed.

That's "verify the chain of custody" made literal -- every link either recomputes or it doesn't. Curious how you handle the absence case at Armorer Labs in practice: does the runtime emit the selection decision inline at each step, or do you reconstruct the candidate set post-hoc from the trace?

[Armorer Labs]

Yes - the useful record is emitted inline at the selection boundary. Post-hoc reconstruction is a debugging aid, not the receipt we want an operator to trust.

The shape we use/aim for in Armorer is:

• candidate set observed by the agent at that step;
• selection policy/version that ranked or excluded each candidate;
• decision outcome: selected, rejected, escalated, or intentionally not called;
• reason code and minimal provenance for the inputs that influenced the decision;
• digest of the normalized args when a call is made, or a digest of the candidate/input context when no call is made.

That last piece is the absence case. A non-call still becomes a positive event: tool_not_called or reasoned_instead, bound to the candidate set and policy state. Then the run record does not need to infer silence from a trace gap.

For lossy inputs, I agree with the chain-of-custody framing. The runtime receipt should point back to the source artifact or prior step digest; if the adapter or prompt path dropped fields, the receipt shows that the dispatched input did not descend cleanly from the source.

Disclosure: I work on Armorer Labs.