Here are 200 lines of the most boring code I have ever written, compressed to twelve:
// intent-router.ts
const ROUTES = [
{ when: /\b(invoice|receipt|bill)\b/i, handler: 'billing' },
{ when: /\b(refund|charge ?back|dispute)\b/i, handler: 'refunds' },
{ when: /\b(password|2fa|locked out)\b/i, handler: 'auth' },
// … fourteen more, each with a test, each added the day a user hit it
]
export function route(text: string): Handler {
for (const r of ROUTES) if (r.when.test(text)) return r.handler
return 'fallback'
}
It took eight months to accumulate. Every regex in it is a scar. It was 91% accurate on our 1,200-row eval set and nobody enjoyed touching it.
Here is what replaced it:
export async function route(text: string): Promise<Handler> {
const { handler } = await agent.classify(text, HANDLERS)
return handler
}
Six lines. Same eval set, same afternoon: 96%. Five points better than eight months of regexes, and I deleted 194 lines to get it.
I want to be completely clear about this, because the rest of the article is going to sound like a warning and it isn't one: the agent was better at the job. It never got worse. That number held. The measurement was real.
It still cost me three weeks.
What broke was never the accuracy. What broke was all the stuff the 200 lines were doing that nobody had written down, because nobody had ever needed to.
What actually happened
Three weeks in, support forwarded a ticket. A customer had asked about a duplicate charge and been routed to billing instead of refunds. Wrong queue, four-day delay, angry customer.
I went to look for the error.
There was no error.
The request had succeeded. Status 200. The agent had returned billing, which is a real handler, spelled correctly, in the enum, on schema. Every check in the pipeline had passed because every check in the pipeline was checking whether the answer was well-formed, and it was. The answer was well-formed and wrong, and those two things had never needed to be distinguished before — because when a regex is wrong it falls through to fallback, and fallback is loud.
So I went to reproduce it. Same text, straight into the router.
It returned refunds. Correct.
I ran it again. refunds. Again. refunds. Nine times out of ten it was right. The tenth time it wasn't, and I could not make the tenth time happen on purpose.
That was the actual moment. Not the wrong answer — the unreproducible wrong answer. I had spent a career on a foundation I'd never once articulated: if I run it again with the same input, I get the same thing. Two hundred lines of ugly regex gave me that for free. Six lines of beautiful agent call took it away, and took about five other things with it that I only found by tripping over them one at a time.
Here they are, roughly in the order they bit.
1. Failure stopped announcing itself
When route() was regexes, a miss looked like this:
route("my card got double charged") → 'fallback'
fallback is a real state. It's logged, it's counted, there's a dashboard tile for it, and when it spikes somebody adds a regex. The system's ignorance was a value. It was in the type. You couldn't not handle it.
When route() is an agent, a miss looks like this:
route("my card got double charged") → 'billing'
Confident. Valid. Wrong. There is no fallback because the agent will always pick something — that's what picking means. I had accidentally removed the only channel through which the system could say I don't know, and I removed it in the same commit that made the system more accurate, which is why nobody caught it in review.
The fix is not clever, but you have to decide to want it:
const { handler, confidence } = await agent.classify(text, HANDLERS)
if (confidence < 0.8) return 'fallback' // put the ignorance back
return handler
The regexes couldn't be uncertain, so they expressed uncertainty structurally, by failing. The agent can be uncertain and expresses it by not mentioning it. If you don't ask, you don't get it.
2. The cache started serving a wrong answer forever
This is the one that turned a bad day into three weeks.
There was a cache in front of the router. Of course there was — it had been there for a year, it was keyed on a hash of the input text, and it had been correct every single day of that year, because a pure function of its input can be cached by its input. That is what pure means.
const key = sha256(text)
const hit = await cache.get(key)
if (hit) return hit
const handler = await route(text)
await cache.set(key, handler, { ttl: '30d' })
Read it now, with a non-deterministic route() behind it, and it stops being a cache. It's a coin flip with a 30-day memory. The first time a phrasing came through, whatever the agent happened to say that time became the permanent answer for that phrasing. If that was the 2% roll, every user who ever phrased it that way for the next month went to the wrong queue. Consistently. Which made it look like a routing rule, not a flake, which is exactly why it took three weeks to find — I was looking for a bug in the rules and there were no rules.
Nothing in that cache changed. Nobody touched it. It didn't break. The thing it was built on top of stopped being true, and it had no way to notice.
Go find your caches, your memos, your useMemo, your idempotency keys, your dedupe-by-hash. Every one of them is a contract that says same input, same output. When you put an agent under one, you are not adding a feature. You are invalidating a contract that something else in your codebase is already relying on, silently, from a file you have not opened.
3. Cost became a function of traffic
The 200 lines cost the same at ten requests a day and ten million: nothing. CPU that rounds to zero. That's not a small property, it's the property that lets you not think about it — you can call route() in a loop, call it twice because it's easier than plumbing the result through, call it on every keystroke.
And we did. There was a place — a live-preview panel — that called route() on input change, debounced at 150ms, because it was free.
It is not free now. Nothing about that call site changed, no one edited that file, and its cost went from zero to a bill.
Worse: the cost isn't just per call, it's per token. A user pasting a long email into the box costs materially more than a user typing "refund". Your unit economics now have a variable in them that your users control and that nobody on the team is tracking, because when the code was regexes there was nothing to track.
4. p99 stopped existing
Old route(): about 0.1ms, and the interesting thing was that the number had a ceiling. Seventeen regexes against a bounded string. There is no input that makes it take a second.
New route(): p50 around 600ms, p99 around 4s, and — the part that matters — no ceiling at all. Not a slow ceiling. None. The upstream can hang. It can rate-limit you. It can 503 in a region. Your p99 is now a property of somebody else's infrastructure and you will find out about it during their incident, not yours.
So the timeout question, which has no good answer:
- Timeout at 2s and you turn a 3% slow-tail into a 3% failure rate.
- Timeout at 30s and you're holding connections open for half a minute for a routing decision.
- No timeout and one bad upstream day exhausts your connection pool and takes down endpoints that have nothing to do with routing.
The regexes never made me pick. Synchronous, bounded, done. I hadn't appreciated that route() being synchronous was load-bearing until making it async rippled through four call sites and one of them was in a hot loop.
5. The tests stopped testing anything
Old test:
expect(route('reset my password')).toBe('auth')
Deterministic, instant, free, and it fails when someone breaks routing. That is the entire job of a test.
You cannot write that test against an agent. Well — you can, and it'll pass most of the time, and it'll go red on a Tuesday for no reason, and within two sprints somebody marks it flaky and skips it. That's not a hypothetical, that is just what happens to a test that fails 2% of the time for reasons no one can act on.
So the tests mutate. They become:
expect(HANDLERS).toContain(await route('reset my password'))
Which asserts that the agent returned a handler. It does not assert that it returned the right handler. It passes if routing is completely broken as long as it's broken into a valid enum value. The suite is still green. The suite is now decorative.
What actually replaces it isn't a unit test at all — it's an eval set that runs on a schedule and reports a rate, plus an alert when the rate moves. That's a real answer and it works. But notice what it costs: it runs on a schedule, not in CI; it reports a distribution, not pass/fail; and it cannot block a merge, because you can't block a merge on a number that jitters. You have swapped a gate for a dashboard, and dashboards are things people have to remember to look at.
6. The code changed without a commit
Six weeks after the swap, accuracy on the nightly eval dropped about a point and a half and stayed there. Nothing in our repo had changed — I checked, twice, and then checked the lockfile.
The model behind the endpoint had been updated.
Sit with that. The behaviour of my production code changed, and there is no commit, no diff, no review, and no line in git log that I can point at. Every instinct I have for "what changed?" is built on the assumption that the answer is in version control. For those six lines, it isn't. It's in somebody else's release notes, if they wrote any.
That's the deepest one, and it's the one that has nothing to do with prompting better. Pin a version where the vendor lets you — and then own the fact that you now have a dependency that expires.
The line I'd write on the wall
Deleting code deletes its guarantees. Guarantees don't show up in the diff.
The pull request showed −194/+6 and a five-point accuracy win. It was, on the evidence available in the pull request, obviously correct, and I would approve it again today.
What the diff could not show was that those 194 lines had been quietly providing determinism, a bounded latency, a zero marginal cost, a fallback state, testability, and a change history — none of which anybody had asked for, all of which other parts of the system had been silently built on top of.
The regexes weren't the feature. They were the reason you could tell when the feature was wrong.
What I actually shipped in the end
Not a rollback. The agent stayed, because it is better. What changed is that it stopped being alone:
export async function route(text: string): Promise<Handler> {
// 1. Deterministic rules first — cheap, instant, and unambiguous when they hit.
const certain = RULES.find(r => r.when.test(text))
if (certain) return certain.handler
// 2. Agent for everything else, but it's allowed to say "no".
const { handler, confidence } = await agent.classify(text, HANDLERS)
if (confidence < CONFIDENCE_FLOOR) return 'fallback'
return handler
}
Roughly 30 lines of rules survived, not 200 — only the ones that are genuinely unambiguous, where a hit means something. They take about 60% of live traffic at zero cost and zero latency, and, more usefully, they are the only thing in the system capable of disagreeing with the agent. When the nightly eval shows the rules and the agent diverging on a case they both have an opinion on, that's a signal, and it arrives before a customer does.
The cache is keyed on the input and the model version, with a 24-hour TTL instead of 30 days. It's a cost optimisation now, not a correctness assumption.
And fallback is back in the type, where it was for eight months before I got clever.
The four questions I ask now
Before deleting deterministic code in favour of an agent, in this order:
- What reads this as if it were pure? Grep for the caches, memos, dedupes and idempotency keys downstream. Each one is a promise you are about to break from a file you aren't editing.
- How does this say "I don't know"? If the answer is "it can't", you have removed a state, not just a code path. Put it back before you merge, not after a customer finds it.
- What's the ceiling? On latency and on cost. If there isn't one, you now need a timeout and a budget, and both of those are product decisions with no correct answer.
- What goes red when this regresses? If nothing does — if the only monitor is a number a human reads on a dashboard — then you don't have a test, you have a hope.
None of that is an argument against agents. I'd make the same swap again; five points is five points and I am never going back to maintaining seventeen regexes by hand.
It's an argument against the thing I actually did wrong, which was to read a −194/+6 diff and believe I was looking at the whole change.
If you've had an agent replace something deterministic and watched a downstream assumption fall over — especially a cache — I'd genuinely like to hear which one. My money's on caches, but I've been wrong about this before.
Top comments (0)