Your team shipped an AI feature. It went out on a Tuesday and it was good.
Six weeks later someone in sales says it has got worse. You ask engineering. Engineering says nothing changed on their side, and they are telling the truth. You ask for numbers and there are numbers, and none of them answer the question. You are now in a meeting about whether a thing is worse, with no way to settle it, and the meeting will happen again next month.
That meeting is what AI evals exist to prevent. And the reason most teams do not have them is not technical.
What an AI eval actually is
Strip the tooling away and an eval is three things.
A set of real examples of what people ask your product. A written description of what a good answer to each one looks like. And a repeatable way to check whether the answer you got matches the description.
That is it. Everything else is plumbing. There are good frameworks for the plumbing, deepeval among them, and picking one is genuinely the easy part of this.
The hard part is the middle item. Somebody has to write down what good means. And that is not an engineering document.
The four decisions inside an eval that are product decisions
What counts as correct. An agent answers a billing question accurately but coldly, to a customer who has already complained twice. Correct or not? There is no technical answer to that. There is a product answer, and if nobody gives it, the engineer writing the check will supply one by accident.
What a failure costs. A wrong tone and a wrong refund are not the same failure, and they should not sit behind the same passing bar. Somebody has to say what each kind of mistake costs the business. That person is not the person writing the test.
Which trade you are making. Every eval bar buys one thing and sells another. Push refusal rates down and you get more helpful answers and more wrong ones. Push them up and the opposite. This is a positioning decision wearing a percentage sign.
When it is good enough to ship. The one everybody agrees is a product call, right up until it arrives as a number in a dashboard, at which point it quietly becomes whoever built the dashboard.
Four decisions. All four get made whether or not anyone decides them. That is the actual risk: not that the team skips evals, but that the team builds evals and the product judgement inside them defaults to whoever had the file open.
So who owns them
Product owns the definition. Engineering owns the machinery.
In practice that means the PM writes what good looks like, in prose, before anything is built. Not a spec of the feature. A description of an answer, specific enough that two people reading it score the same output the same way. If two of your colleagues read your definition and disagree about whether an answer passes, the definition is not done, and no amount of engineering downstream will fix it.
Then engineering makes it run every night.
This split is unglamorous and it is the whole thing. The teams that hold quality over time are not the ones with the best framework. They are the ones where a named human owns the sentence that says what correct means, and that human sits on the product side.
The honest limit
Evals will not tell you your product is good. They tell you whether it changed, and which of two versions is better at the things you decided to measure. Everything you did not think to describe is invisible to them, permanently.
Which is an argument for writing the definitions badly and early rather than perfectly and never. The first twenty are worth more than the framework, and you can write them this week without asking engineering for anything.
The broader discipline that all of this sits inside is called harness engineering. But the part that decides whether it works for you is not in the tooling. It is one person, on the product side, willing to write down what good means and put their name on it.
Top comments (1)
The "badly and early rather than perfectly and never" line is the one worth framing. We spent about two months trying to get the definition of "correct" right before we'd accept any eval results, and in the meantime the product was drifting in directions nobody had words for. The first twenty definitions we eventually shipped were wrong in several places, but they told us things that two months of intuition-based reviews had missed. The product ownership of the definition is also the right call practically; the one time engineering wrote it we ended up measuring for something the PM would never have accepted as passing.