Your team shipped an AI feature. It went out on a Tuesday and it was good.
Six weeks later someone in sales says it has got worse. You ask enginee...
For further actions, you may consider blocking this person and/or reporting abuse
The "badly and early rather than perfectly and never" line is the one worth framing. We spent about two months trying to get the definition of "correct" right before we'd accept any eval results, and in the meantime the product was drifting in directions nobody had words for. The first twenty definitions we eventually shipped were wrong in several places, but they told us things that two months of intuition-based reviews had missed. The product ownership of the definition is also the right call practically; the one time engineering wrote it we ended up measuring for something the PM would never have accepted as passing.
Exactly. I think that’s the part that’s easy to underestimate: an imperfect definition can still give you information, while an undefined “correct” leaves you relying on intuition.
And that engineering example is exactly why I separate ownership of the definition from ownership of the machinery. Engineering can make the eval run perfectly, but if the thing being measured isn’t what the product actually considers acceptable, you’ve just built a very efficient way to measure the wrong thing. 😄
This matches how the failure shows up in practice. The missing owner is usually the person who can price a bad answer, not the person who can wire up the test runner. I like the split between quality criteria and plumbing. If a team has only one eval to start with, I would make it the highest-cost failure mode rather than the most common prompt.
I really like the “price a bad answer” framing. That gets to the heart of why eval ownership is a product decision.
And yes, if you only have capacity for one eval, I’d rather have one tied to an expensive failure than twenty measuring what happens most often. Frequency tells you what happens. Cost tells you what you need to protect.
That distinction is easy to miss when you start from the test runner instead of the product.
The engineer writing the check will supply one by accident"" is the sentence I would lead with, because that is not a hypothetical, it is the default outcome and it happens silently.
I would add a fifth decision to your four, and it is the one that gets made by accident most often: whether the bar is an absolute number or a delta against the last known-good run. An absolute bar (""must score above 0.85"") drifts, and it goes green on a change that quietly took you from 0.91 to 0.86. A delta bar fires on the drop itself. That looks like an engineering detail and it is not. It decides whether your eval is answering ""is this good enough"" or ""did we just make it worse"", and those two questions belong to different people and get used in different meetings.
Your six-weeks-later scenario is specifically the absolute-bar failure. Nothing changed on engineering's side, all the numbers are above the bar, and the thing got worse. A delta against a pinned baseline settles that meeting in one query, and no amount of tooling settles it without one.
Yes. I really like this addition because “absolute bar vs delta” sounds like an implementation detail until you ask what question the eval is actually supposed to answer.
“Is this good enough?” and “Did we just make it worse?” are very different questions, and I hadn’t separated those explicitly in the four decisions. Your point about the absolute bar quietly letting 0.91 → 0.86 go green is exactly the kind of failure that can hide in plain sight.
I think you’ve convinced me that baseline strategy deserves its own decision. 😊
"Very efficient way to measure the wrong thing" is exactly the failure mode we kept falling into. We'd get a green score and the PMs still felt something was off, and we didn't have language for the gap until we made product write out what "bad" meant to them in plain cases. Even the first attempt was wrong in five or six spots, but suddenly everyone was disagreeing about the same thing instead of about different things.
I feel that. That's why we need to define 'What the good answer' is, in the first place.
The "who writes down what good means" problem is where we've gotten stuck most consistently. In entity resolution specifically, "correct" depends on which downstream consumer you ask: the risk team wants high precision because a false merge has legal consequences, the analytics team wants high recall because missing a connection loses them a signal, and both think they're the primary stakeholder. We ended up getting both in a room with ten actual contested cases and forcing a written decision, which sounds obvious but nobody wanted to do it before we made it mandatory. The resulting rubric had more asterisks than a tax form but at least two people scoring the same output now land on the same answer.
We hit this at a previous job - the billing chatbot had "gotten worse" but all our metrics looked green, and we didn't figure out why until three weeks later by manually digging through failure logs. Engineering had been watching latency and refusal rate, and neither caught what the sales team was seeing. The thing I'd add to your ownership question: whoever makes the "good enough" call should really be the person who fields the complaint when something slips, not whoever had the nearest sprint to fit it in. In practice that's meant product in theory and whoever built the dashboard in reality.
That last distinction is painful because it happens so often: the dashboard is green while the people actually using the system are telling you it is getting worse.
I really like your point that the person making the “good enough” decision should also own the consequences when that threshold turns out to be wrong. Otherwise ownership gets fragmented very quickly: engineering owns the metrics, product owns the requirement, sales owns the complaint, and nobody actually owns the gap between them.
And your billing chatbot example is a good reminder that an eval can be technically correct and still be measuring the wrong thing. If the metric isn't connected to what “working” actually means for the user, a green dashboard can become a very convincing false reassurance.