You had twenty examples
in a spreadsheet.
You changed the prompt,
ran them,
sixteen passed.
Changed it again.
Nineteen.
Moved one sentence
to the top,
and all twenty went green,
and you shipped it
on a Thursday.
Users broke it
before lunch on Friday.
Nothing about that is surprising
once you see
what the twenty examples were.
You wrote them.
They came out of the same head
that wrote the prompt,
on the same afternoon,
holding the same picture
of who was going to use this.
So they share its blind spots
exactly.
Every case you did not imagine
is missing from both,
and the set cannot tell you
about a question
you have never thought to ask.
Then there is the second thing,
which is worse
and harder to see.
You were looking at the answers
while you edited.
Twenty examples,
visible,
adjusted against,
over and over,
until the number went up.
That is not measurement.
You did not make the system better.
You made this prompt
fit these twenty,
and you have no way to tell
those two apart from where you sit.
The number rose
and nothing got safer.
So build the set
out of things you did not invent.
Real inputs, from real people.
The message in the wrong language.
The paste out of a PDF
with the line breaks in it.
The one that is three words long.
The angry one.
Keep a portion of it closed.
You do not read those
while you are tuning.
You run them once,
at the end,
and that is the only number
you are allowed to believe.
And every failure
that reaches production
becomes a permanent case.
Not a fix. A case.
It never leaves the set,
because the useful examples
are the ones nobody would invent.
Your evaluation is not a score.
It is a claim about who will use this,
and for now
that claim is just you,
on a Thursday,
agreeing with yourself.
– Serguey Asael Shinder
Top comments (0)