My agent passed every check and still broke production in an hour. Here's the CI/CD I run now.

A while back my team merged what looked like a harmless change: a small refactor to our agent's tool-routing prompt. It passed lint. It passed unit tests. It passed our eval gate. Within the hour, tool-call accuracy in production had visibly regressed, and I was on-call rolling it back.

When I dug in, the cause was almost boring. Our eval set barely covered the dispatch-tool slice of traffic, so the new prompt regressed exactly where the gate had no cases to catch it. The gate did its job on everything it could see. It just could not see the thing that broke.

We shipped the same change again a day later, this time behind a canary that scored live traffic and rolled back on its own. The canary caught the regression early in the ramp, and on-call got paged once, after the automatic revert. That was the day I understood the actual shape of CI/CD for agents: the offline gate fails open on what it cannot see, and the canary closes the loop on what the offline gate misses. You need both. One without the other ships on hope.

These days I think of it as four checkpoints between "PR opened" and "everyone gets it":

• Fast checks on every PR, under about ninety seconds. Does the prompt template still parse, are the tool schemas valid, does a small fixed set of cases still pass, and did the token count blow up. This catches typos, broken schemas, and "the new prompt is three times longer" before anything expensive runs.
• A bigger eval at merge time, five to fifteen minutes. A few hundred cases covering normal traffic, edge cases, and past failures, scored by a judge. What I check is not an absolute number but whether the new version is any worse than main. More on why that matters below.
• A simulation pass before deploy, for the risky changes. Multi-turn conversations with a handful of personas, plus jailbreak and prompt-injection probes, plus deliberately broken tool responses to see if the agent recovers. Most of the failures that actually reach users show up on the second or third turn, not the first, and only this stage catches them.
• A canary after merge, ramped over a day. Start at 1 percent of traffic, then 5, then 25, then 100, scoring a sample of live requests the whole way. If quality drops, the guardrails start tripping, latency climbs, or cost per request jumps, it rolls back on its own before most users ever see it.

The one piece of gating advice I would attach to all of this: gate on the change, not on a fixed score. I used to say "faithfulness has to stay above some number." The problem is that the number moves on its own. The provider ships a minor model revision and every score shifts a little. The eval set gets harder as you add cases. Absolute thresholds either start crying wolf or get loosened until they catch nothing. Comparing the new version against main instead means each change can only slip by a small, bounded amount, and small drops stop quietly piling up.

The other thing I learned the slow way is that the eval set rots. The most common version of "the gate passed and prod broke anyway" is an eval set that stopped looking like production months ago. New kinds of requests show up, old ones fade, and the gate keeps checking the wrong things. So I treat it as living: I skew it toward real traffic with a slice of edge cases and a slice of past incidents, review it monthly, refresh a chunk every quarter, and end every postmortem by adding at least one new case. A failure we have already seen should never come back quietly.

And the strangest agent bug is the one where nothing in your code changed and the agent still got worse. It is almost always upstream: the provider rolled the model, or the judge you score with drifted. So I pin what I can. I use dated model snapshots instead of floating names, because the plain name will move under you while the dated one will not, and I keep the judge fixed so "the same eval" actually means the same eval. A weekly check then tells me if a score moved for no reason, which usually means the model changed under me.

A few of the traps I fell into on the way here:

• Trusting lint and unit tests to catch what are really meaning-level regressions. They will not.
• An eval set of clean, easy queries that never sees the weird 5 percent production actually sends.
• Routing a slice of traffic to a new version with nothing watching it and calling that a canary. Without live scoring it is just a slow rollout.
• Manual rollback, which adds real minutes of user-facing damage. Letting it revert on its own is the whole point.

None of this is exotic infrastructure. It is mostly the discipline of assuming the gate cannot see everything, and putting a second net where the first one has holes. If you ship agent changes, I am curious where you draw the line: what has to pass before a change reaches real users, and do you trust it to roll back on its own?