The demo answers one question beautifully.
Someone says, “It works.”
The room nods. A slide gets a green tick. Somewhere, a requirement quietly escapes through the window.
I'm interested in the next question: works for which task, for whom, and under what conditions?
A chat box can hide several different products
An assistant that summarises a public document is different from one that reads customer records. Both are different from one that can change those records.
They might share the same interface. Their acceptance criteria should not be interchangeable.
Think of a kitchen knife and a plastic spoon. Both live in the cutlery drawer. We do not run identical safety tests because they have similar storage arrangements.
Four claims to separate
“It produced a plausible answer.”
That describes the appearance of one output.
“It answered these representative cases acceptably.”
Now we have a test set, conditions and criteria to inspect.
“It stayed inside its permissions when the request was hostile or ambiguous.”
Now we are checking behaviour that a friendly demo may never exercise.
“It continues to meet our service expectations after changes.”
Now we need monitoring and repeatable evaluation, not only a memorable launch-day conversation.
None of these statements automatically proves the next one.
An exercise for the next product meeting
Ask everyone to finish this sentence independently:
“We should not release this assistant if it ______.”
A developer might say “times out on common queries.” Product might say “cannot complete the customer's task.” Security might say “reveals another user's information.” Operations might say “fails without a usable fallback.”
That disagreement is useful. You have discovered different expectations while everyone still has time to do something about them.
It is much cheaper than discovering the same disagreement through an angry support ticket.
For evaluation background, see Anthropic's discussion of agent evals. An evaluation needs to match your task; copying somebody else's benchmark score is not a substitute for defining your own release question.
What is one AI failure your team would treat as a release blocker, even if the assistant gives excellent answers most of the time? Specific examples welcome; confidential details can stay off the internet.
Draft prepared with AI support. The meeting and assistant examples are hypothetical, not claimed firsthand events.
#AI #Discuss #Testing #Productivity
Top comments (0)