The standard answer to validating without interviews is: talk to customers. The useful version is narrower. You are not skipping the customer — you are changing what you collect from them. A conversation gives you an opinion you grade AFTER you have heard it. A number checked against a bar you wrote down first gives you something you cannot re-grade once you have seen it.
The problem with interviews is not the interview. It is that they are unfalsifiable in the hands of the person who wants the answer. You ask ten people whether they would pay, seven say something encouraging, and you now hold ten data points and the freedom to decide afterwards which of them counted. You will decide in the direction you were already leaning, because you wrote the questions and you are the one who has to abandon eight months of work if the answer is no.
That is not a discipline problem you fix by being more rigorous in the room. The bar moved because nothing was holding it in place. So put something in place: pre-registration. Write down what would count BEFORE the test runs — a number, a metric and a deadline — then run it and compare.
In our product that is not rhetorical. Validation plans emit machine-gradable criteria alongside the English ones, and the grader is pure code with no model call in it. It reads exactly 5 things, and that set is closed on purpose: a grader that cannot read a metric cannot grade it, and a criterion existing only as English prose is marked ungradable rather than quietly interpreted.
Three rules do the actual work, and each is a monotonicity decision rather than a feature.
A blank is not a zero. An unreported metric grades as not-reported, never as not-met. Collapsing those would make an empty form read as evidence the idea failed, and I did not measure it is not that. When every criterion comes back blank the whole outcome is ungradable and nothing moves.
The window runs one way. A criterion says reach N within D days. Reaching it faster passes; taking longer does not. A founder who needed thirty days to clear a fourteen-day bar produced a different result from the one the plan asked for, and letting it through deletes the only time-box in the loop.
And you cannot correct upward. Within one plan the report that counts is the weakest one filed: a worse number takes effect immediately, a better one leaves the earlier grade standing. The asymmetry is deliberate — a bar written in advance is worth nothing if the answer can be rewritten after you have seen how it graded.
Now the part where validation advice usually overreaches. A landing-page test that goes well is not proof the idea works. It is proof of some specific things and silence on others. In our scorecard the widest single metric can evidence 55 percent of it. The remaining 45 percent — support load, platform exposure, cash flow — a clean market test cannot reach at all.
And traffic on its own evidences 0 categories. A channel is proven when it delivers a PERSON, not when it delivers attention. That one is worth writing on the wall.
The worked plan in the piece publishes 4 criteria, of which the grader could grade 3, of which 1 was met — a partially-met outcome rather than a pass or a fail, because that is what the numbers actually support.
The template, the three rules and what a pass does not prove: https://whittleos.com/guides/validate-startup-idea-without-talking-to-customers
Top comments (0)