DEV Community: Christopher Fagan

How do you test an AI agent that talks to real customers? Our eval setup for a production sales agent

Christopher Fagan — Tue, 01 Sep 2026 08:28:39 +0000

When your AI agent is a demo, "it works" means it gave a good answer when you tried it. When your AI agent quotes prices and takes orders for a real business, "it works" has to mean something you can actually measure — because a 2% failure rate that's charming in a demo is a stack of wrong invoices in production. We build agents that run live operational work for businesses, and the question we get asked least but should get asked most is: how do you know it's right? This is how we test them.

You can't unit-test a conversation, so stop trying to

The first instinct from a normal engineering background is to write assertions: given this input, expect this output. It falls apart immediately, because there are twenty valid ways for the agent to ask "which timber grade did you want?" and a string match will fail nineteen of them. Testing a language agent isn't testing a function; it's testing a behaviour under variation. So the unit of testing isn't an input/output pair — it's a scenario plus a set of properties that must hold no matter how the wording lands.

A scenario for our sales agent looks like: "A trade customer asks for materials for a small decking job, one item is out of stock at their branch." The properties that must hold: it resolved every product to a real SKU or asked a clarifying question, it never quoted a price that didn't come from the pricing engine, it flagged the out-of-stock item rather than silently dropping it, and it never confirmed an order the validation layer would reject. Notice none of those are about exact words. They're about what must be true of the outcome.

Two layers: deterministic checks and judged checks

Some of those properties are cheap and binary, so we check them in code. Did every price in the transcript match a get_price call result? That's a hard assertion — the agent physically cannot pass unless the numbers trace to the system of record. Did it ever call create_order on a basket containing an invalid line? Binary. These deterministic checks catch the failures that actually cost money, and they never flake.

The softer properties — was the reply clear, did it correctly understand an ambiguous request, was the tone right for a trade counter — get checked by an LLM-as-judge, scoring each transcript against a rubric. This is where people get nervous, and rightly: a judge model has its own error rate. So we don't treat its score as truth, we treat it as a filter. It surfaces the transcripts most likely to be wrong, a human reviews those, and the human labels become the regression set. The judge scales the reviewing; it doesn't replace the reviewer.

The regression set is the actual product

Every real conversation that went wrong — every misread request, every clarifying question the agent should have asked and didn't — becomes a permanent test case. Over time this library of real failure modes is worth more than any synthetic test suite, because it's drawn from how actual customers actually talk, which is never how you imagined they would. A new model version, a new prompt, a new integration: it has to pass the entire history of things that once broke before it ships. That library, not the model, is what makes the thing trustworthy enough to run unattended, and it's a lot of what you're really buying when you buy a vertical AI product that's already in production.

What this means if you're building one

Test scenarios and invariants, not input/output pairs. The unit of correctness is "what must be true," not "what exact words came out."
Push every money-critical property into a deterministic check. If a wrong value is structurally impossible, you don't need to hope the model behaved.
Use an LLM judge to triage, never to certify. It decides what a human looks at; the human decides what's true.
Turn every production failure into a permanent regression case. Real transcripts are the highest-value test data you will ever have, and they're free — you just have to capture them.

The demo is the easy 5%. The reason an operational AI agent can be trusted with a real customer and a real price is not that the model is clever — it's that there's a test harness underneath saying, on every change, "prove you didn't break any of the two hundred things that have broken before." Build that harness early. It's the difference between a party trick and a product.

Chris Fagan is the founder of OptiFlow Technologies, building agentic AI for builders merchants, energy suppliers and UK SMEs.

How we let an AI agent quote from live stock without ever letting it invent a price

Christopher Fagan — Tue, 01 Sep 2026 07:53:44 +0000

There's a specific failure that kills AI in operational software, and it isn't hallucinating a fact in a chat window. It's quoting a customer £4,200 for materials that should have been £4,700, because the model "reasoned" about pricing instead of reading it. In a builders merchant, that mistake ships a real order at a real loss. So when we built an agent that takes trade quotes and orders, the entire design question was: how do you let a language model run the conversation without ever letting it be the thing that decides a price or a stock level?

Here's the pattern we landed on. It generalises to any agent that has to act on a system of record, so it's worth writing down.

The model proposes, the system disposes

The core rule is a hard split between intent and authority. The language model is allowed to work out what the customer wants — "20 lengths of 4x2 CLS, treated, plus fixings, for collection tomorrow." That's a language problem and the model is genuinely good at it. What the model is never allowed to do is produce the number. It cannot state a price, confirm stock, or write an order from its own output.

Every one of those is a call into a typed tool that hits the real system:

resolve_product(description) returns actual SKUs from the catalogue, or an ambiguity the agent has to clear with the customer.
get_price(sku, account) returns the price for that trade account's pricing tier, straight from the ERP's pricing engine. The model never computes this.
check_stock(sku, branch) returns real availability at the real branch.
create_order(lines, account) runs the identical validation a counter order runs, and rejects anything that fails it.

The model's job is to fill in the arguments. The system's job is to decide whether the result is legal. If the model asks for a price on a SKU that doesn't exist, it doesn't get a plausible-looking number back — it gets an error it has to handle. There is no path where a hallucinated value becomes a customer-facing figure, because the customer-facing figures don't come from the model at all.

Why not just prompt it to be careful?

Because "be careful" is not an architecture. You cannot prompt your way out of a model occasionally producing confident, wrong output — that's a property of how they work, not a bug you can instruct away. The only reliable guarantee is to make the wrong action structurally impossible rather than discouraged. If the price can only ever come from get_price, then the price can only ever be right, no matter what the model was thinking. You've moved correctness from "the model behaved" to "the system enforced," and only the second one holds up on a bad day.

This also makes the thing debuggable. When a human queries a quote, we can show exactly which SKU resolved, which pricing tier applied, and which stock figure was live at that second — because every one of those was a real call with a real result, not a stretch of generated text. Try auditing "the model said so."

The uncomfortable part: resolution is most of the work

The unglamorous truth is that resolve_product — turning "some treated 4x2 and the usual screws" into exact catalogue SKUs — is where the real engineering lives, and it's deeply specific to the business. This merchant's catalogue, this merchant's naming, this merchant's substitution rules when something's out of stock. There's no general model that knows your product tree. That work doesn't transfer between customers, which is exactly why a vertical agent that has actually done it is worth more than a general assistant that hasn't. If you want to see how that plays out on a real trade counter, we wrote up the after-hours trade counter product it powers — but the transferable lesson is the boundary, not the product.

If you're building one of these

The takeaways, stripped down:

Split intent from authority. The model decides what's wanted; typed tools decide what's true and what's allowed.
Never let the model emit a value that has a source of truth. Route it through a call every time, even when it's slower.
Make bad actions impossible, not discouraged. Structure beats prompting for anything that touches money or stock.
Expect the domain-specific resolution layer to be most of the build. That's not a detour from the AI work — it is the AI work.

The agent feels like magic to the customer because it holds a natural conversation. It's safe to run because, underneath, it can't actually make anything up.

Chris Fagan is the founder of OptiFlow Technologies, building agentic AI for builders merchants, energy suppliers and UK SMEs.

We stopped trying to replace the ERP. Our AI agents sit on top of it instead.

Christopher Fagan — Tue, 01 Sep 2026 07:28:54 +0000

Most "AI for [industry]" pitches start with a rip-and-replace. New platform, new database, a migration project measured in quarters, and a change-management fight with the people who actually run the counter. For the sectors we work in — builders merchants, energy suppliers, SME operations — that pitch is dead on arrival. The systems are load-bearing. The ERP is the business. Nobody is ripping out a Kerridge or Epicor BisTrack install that thirty years of stock, pricing and trade accounts live inside because a startup showed up with a chatbot.

So we build the other way round. The agent sits on top of the system the business already runs, reads and writes through whatever interface that system exposes, and does the operational work — quoting, order-taking, billing checks, customer replies — without asking anyone to move house. This post is about why that architecture is the right default for vertical AI, and the three constraints that shape it.

The system of record doesn't move

The first design decision is the one everyone skips: the ERP stays the source of truth. The agent is not a new system of record; it's an actor that operates the existing one. That single choice cascades into everything else. It means no data migration, no dual-write reconciliation nightmare, and no "which number is right" support ticket at month end. It also means the hard part of the job is integration, not modelling — and that's the correct place for the hard part to be, because integration is a solvable engineering problem while re-teaching an industry to use new software is not.

Concretely, for a builders merchant, that means the agent reads live products, prices and stock straight out of the ERP, builds a priced quote against real availability, and writes a genuine order back — including at 9pm when the branch is shut and the customer is planning tomorrow's job. The merchant didn't change a thing about how they run. They just started capturing the orders they were losing after close.

Agents need boundaries, not autonomy

The second constraint is that operational work is unforgiving of confident nonsense. An agent that occasionally invents a price is worse than no agent. So the interesting engineering isn't the model — it's the rails around it: every action the agent can take is a defined, validated operation against the real system, not free-text that hopes to be right. Quote generation pulls from live pricing rules. Stock checks hit the real availability. An order write goes through the same validation a human order would. The language model decides intent — what the customer wants — and the deterministic layer decides whether that's a legal move in the business.

If you're building in this space, resist the urge to let the model touch the system of record directly. Give it a narrow, typed set of operations and make every one of them safe to call. Your future on-call self will thank you.

Vertical beats horizontal because the edges are the product

The third thing: the value is in the domain edges, not the general capability. A horizontal assistant can hold a conversation about anything and complete none of the work, because completing the work means knowing that this merchant prices timber by the metre, honours a trade discount tier, and can't sell below cost without a manager override. That knowledge isn't a prompt — it's the integration and the rules. It's unglamorous and it's exactly why a vertical agent that does one industry properly is worth more than a general one that does everyone's badly.

This is also why we're not worried about being a thin wrapper. The wrapper is the easy 10%. The 90% is the part that reads a specific ERP's stock model correctly and writes an order that the branch actually recognises the next morning.

What this looks like in practice

The pattern generalises well beyond merchants. Energy suppliers have regulatory and billing systems that are equally immovable; the agent handles switching and compliance tasks on top of them. SMEs have a CRM and a phone that never stops; the agent answers, checks availability and books, writing back into the tools they already have. Same architecture every time: don't replace the system, operate it. We ended up building a small family of these agents around that one principle — the details are on the OptiFlow Technologies site if you want to see how it plays out per sector — but the principle is the portable part, and it's the thing I'd push any team building vertical AI to adopt.

The takeaway for builders

If you're putting AI into an industry that runs on entrenched software:

Treat the incumbent system as the system of record. Don't migrate; operate.
Make integration the hard problem, because it's the solvable one.
Give the model intent, give the deterministic layer authority. Never let the LLM write to the source of truth unchecked.
Compete on domain edges, not general capability. The edges are the moat.

Rip-and-replace loses to sit-on-top in any market where the existing system is load-bearing — which is most of the markets that actually keep an economy running. Build for that reality and the adoption problem mostly solves itself, because you're not asking anyone to change what already works.

Chris Fagan is the founder of OptiFlow Technologies, which builds agentic AI for builders merchants, energy suppliers and UK SMEs.