Originally published on the Ramsud Technologies blog.(https://ramsudtechnologies.com/tool-decision-architecture)
The core question in AI system design isn't "should we use an LLM?" It's: who decides which tools the system calls?
In an agentic system, the LLM sees a set of available tools and decides which ones to call, in what order, with what arguments. In a deterministic system, code decides the tool sequence. The LLM never sees the tools.
Agentic LLM sees tools → LLM decides which to call → execution follows Deterministic: Code decides tool sequence → tools execute → no LLM in the decision loop*
The agentic pitch is seductive for many workflows — it handles edge cases you didn't code for, reads messy natural language, adapts to new situations. For triage, drafting, and summarization, that's often the right call.
For refunds, we chose deterministic. Here's why, and what we're learning building it.
Why refund issuance is different
A refund decision is binary and consequential: money leaves the business or it doesn't. What matters — delivery status, days since delivery, order amount, customer history — is all structured data. No messy prose to interpret. And when it goes wrong, it's not "slightly worse UX." It's:
Chargebacks from customers disputing refunds they claim never to have requested
Fraud vectors a model might learn to exploit
Support cleanup when an LLM's hallucinated policy exception commits the company to something legally inconsistent
Audit failures when someone asks why a model made a financial decision and you can't explain it
So we built it the boring way: a fixed sequence of data lookups, a deterministic eligibility check against policy, a hardcoded decision path. No model decides whether money moves. Ever.
The rule we're using: an LLM can inform a decision about money. It should not make the decision alone.
What we rejected: LLM decides which tools to call
In the agentic pattern, the LLM sees your tools and decides which ones to call. For a refund system, that means the model is deciding things like "should I call issue_refund() now, or gather more info first?" or "this customer's tone suggests they deserve a refund."
The problem: on day 1, with one phrasing, the LLM calls tools in order A. On day 2, with slightly different context, it calls tools in order B. The decision isn't reproducible. Not auditable. Not defensible if anyone asks why.
What we kept: tool patterns, not tool autonomy
We borrowed patterns from agent infrastructure but removed the autonomy. We built a scoped tool gateway that:
Enforces token-based authentication on every tool call
Logs every access in an immutable audit trail
Restricts which tools can be called — no dynamic discovery
Scopes each tool to specific data (customer data only, order data only)
This is defensive infrastructure, not AI autonomy. The gateway doesn't decide when to call tools — the code does. The LLM never talks to the gateway; the code does.
We initially named it SecureMCPGateway, implying Model Context Protocol compliance. It wasn't — it was a hardcoded dispatcher with three allowed function names and no dynamic discovery. We renamed it ScopedToolGateway. Clearer, and honest about what it actually does.
The lesson: adopt the infrastructure patterns that matter (audit logs, scope boundaries, access control). Reject the framework patterns that give LLMs autonomy they shouldn't have.
Where we'd actually use a model
As this system grows, the natural place for an LLM is upstream: parsing free-text support requests into structured categories, drafting a friendlier decline email than a template allows, triaging which of several policies applies when a request is genuinely ambiguous.
All of that is advisory. A human or a deterministic rule still gates the money-moving step. The model makes a human decision faster — it doesn't replace it.
The trap: using an LLM because it's available, not because you actually need it.
The architecture
Request flow — code controls every tool decision:
Parse input — extract customer ID and order ID from the free-text request. LLM not involved yet.
Call tools in fixed sequence — code decides to call get_customer() → get_order() → get_delivery(). Same sequence, every time.
Scoped gateway validates — each call is authenticated with a token. No other tools can be called. No dynamic discovery.
Policy lookup — code looks up policy rules and returns eligible status and window. Advisory only, no decision here.
Deterministic decision — a fixed rule: IF status in [damaged, lost] AND days_since_delivery <= 30 THEN eligible ELSE decline. Binary logic, no model judgment.
Create ticket — eligible → pending_approval; not eligible → declined (auto-resolved).
Human approval — a support agent reviews the ticket and calls approve_refund(). The human makes the actual decision to move money.
Refund issuance — a database constraint prevents duplicate refunds for the same order. The transaction logs everything.
Two gates, not one: code decides eligibility, a human decides approval. Neither gate is an LLM.
The database constraint in step 8 matters more than it sounds like it should — a naive idempotency check (an in-memory set, a per-ticket status flag) will pass a single-threaded happy-path test and still let two refunds through under concurrency, because most implementations end up enforcing uniqueness on the wrong entity. The constraint has to live on the order, not the ticket, since a new ticket gets created per request:
sql
CREATE UNIQUE INDEX one_issued_refund_per_order
ON support_tickets (customer_id, order_id)
WHERE status = 'refund_issued';
A partial unique index scoped to status = 'refund_issued' means Postgres physically cannot hold two issued-refund rows for the same order, regardless of how many tickets get created or how many processes are racing.
What this buys you
Fraud surface: an LLM learns from patterns — if it sees "customer called 3 times → refund approved," it might learn "call a lot, get refunds." A rule engine can't learn; it only does what you told it to.
Explainability: "delivery status was damaged and within 30 days" is a defensible answer to "why did this refund happen?" "The model thought the tone suggested the customer was upset" is not.
Predictable failure modes: an LLM needs a human in the loop for edge cases it can't resolve confidently. A deterministic system routes edge cases to a human ticket by design — same outcome, but the failure path is visible and logged, not discovered after the fact.
When you're tempted to use an agent
Before reaching for an agent framework for a workflow like this, ask:
Is this decision actually non-deterministic, or would a rule engine or state machine work?
Can I test and audit every decision path?
If the model gets it wrong, what's the blast radius?
Does it make the system easier to operate — or just more impressive?
If the honest answer to the last one is "more impressive," you're adding an attack surface and a failure mode for no capability gain. Deterministic code that's well-tested and auditable beats a model with judgment you can't fully verify — especially anywhere real money, PII, or irreversible actions are involved.
We'd rather ship the boring version that's provably correct than the exciting version that's probably fine.
This isn't a retrospective — it's a live project. We're building this refund agent in production right now and documenting the architectural decisions as we make them. If you're weighing the same LLM-vs-deterministic call for a workflow that touches money, the full writeup has the rest of the production pattern (audit trail design, the human-approval flow, and the compliance angle).
Top comments (0)