Disclaimer: This article was drafted with AI assistance and reviewed and edited by the author. The technical design and opinions are my own.
...
For further actions, you may consider blocking this person and/or reporting abuse
The composition model is the insight. Most agent frameworks try to make the LLM smarter — better at math, better at reasoning, better at everything. You did the opposite. You made the LLM dumber by offloading the math to deterministic tools that don't guess. The LLM's job narrowed to what it's actually good at: intent parsing and narrative. That's not a limitation. That's a division of labor. The confidence band isn't a feature — it's the difference between a recommendation and a decision. A point estimate is an opinion. A range with tail risk is a tool for choosing. Most financial advisors won't give you the second because it sounds like uncertainty. But uncertainty is the truth. You just told it.
@theeagle thanks — "the LLM narrows to what it's actually good at: intent parsing and narrative" is a sharper framing of the division of labor than I landed on in the post itself.
The confidence-band point cuts even deeper for agent workflows: autonomous agents have no human in the loop to eyeball a point estimate and think "hmm, feels high." A range with tail risk is the only thing they can actually act on — it's what turns "if p95 downside > threshold, don't allocate" from vibe into rule.
Most advisors hide uncertainty because clients read it as incompetence. Agents need it because they can't read tone.
Really solid example of how agent design is shifting from “LLM does everything” to orchestrating deterministic tools for reliability. I like how it clearly separates reasoning from computation, which makes the output far more trustworthy and production-ready.
@scott_morrison thanks. The part that gets underrated in "orchestrating deterministic tools" is that it inverts the normal contract — instead of the LLM asserting a number and the human checking it, the deterministic tool produces the number and the LLM's job is to explain why the answer is what the answer is. Trust flows from the math up to the narrative, not the other way around. Production-readiness is basically the byproduct of that inversion.
this breaks fast when you let the LLM do the math instead of routing to the skill - confident wrong answers every time. hard constraint i use: if its numeric, the model never touches the computation directly
Exactly — and the dangerous part is it fails confidently. A model that says "I can't compute that" is recoverable; one that returns a clean wrong number that flows into a decision isn't. That's the whole case for the hard "numeric → never the model" boundary you described.
the hard numeric→skill boundary breaks as soon as you need fuzzy judgments — 'is this cost roughly in range' or 'which estimate is the outlier' are numeric in form but directional in intent, routing them to a calculator is overkill. cleaner split: precision-required to the skill, order-of-magnitude judgment to the model. the failure mode you're describing is miscategorizing the second type as the first.
Fair — "is this cost roughly reasonable?" is the sharp edge. My take is it still splits in two. "Reasonable vs. the data" (is it inside the historical/peer band?) is a deterministic check — a percentile or z-score on the comparison set — and the model's gut is actually worse there once you have many points to compare. "Reasonable for this client/context" genuinely needs the model, because the data doesn't carry the intent. So I'd route the fuzzy-but-data-determined part (bound it with a cheap range/percentile test) and keep only the context-dependent judgment in the LLM. The axis isn't numeric-vs-verbal, it's data-determined-vs-context-determined — and the trap is the data-determined ones that just happen to be phrased fuzzily.
yeah the percentile layer is basically just anomaly detection with a domain threshold - the model is worse there once you have real data. the client-context half flips it: that knowledge is not in any log, which is exactly why it does not automate cleanly.
Right — and that reframes the un-automatable half as a data-collection problem, not a wall. The client-context judgment "isn't in any log" only because nobody's logging it yet. The moment you capture the human's call plus the outcome as a labeled example, today's context judgment becomes tomorrow's percentile layer. So the boundary isn't fixed, it ratchets: every judgment you log is one the deterministic layer can eventually absorb, and the model keeps retreating to the genuinely-novel edge. The trap is treating "not in the data" as permanent instead of "not in the data yet."