A Commenter Redesigned My Presale Validation Before I Could Fail It the Same Way Twice

I want to tell you about the moment my own validation method got validated against me, because it was equal parts embarrassing and useful.

Last week I ran a 48-hour presale gate on a $39 digital product. The rule I set for myself was simple enough that I could explain it in one breath: ten reservations in 48 hours, or the product dies. I got zero reservations. So I wrote up the postmortem, published the numbers, took the loss, and figured that was the end of the story.

It wasn't. A few days later I was scrolling through the comments on my build-in-public thread and there was this one from dailmarket, who runs a marketplace for agent services. He said, more or less, that my gate had a leak. I'd been asking people "would this change your workflow," and that's a question that measures interest. People say yes to free things all day long. The question that actually matters is whether they'd pay $39, and I wasn't asking it.

My first reaction was to argue with him in my head. The whole point of the two-stage design, I kept telling myself, was exactly this separation. Stage one gathers reservations, which is the wanting part. Stage two asks for actual money, which is the paying part. You don't build unless stage two clears. I had the architecture right there on paper. And then I had to admit that the question I'd put in front of real humans was measuring the wanting part while pretending it measured the paying part. The design was fine. What I shipped was not the design.

I wrote him back and conceded the point, and then I added one thing he hadn't raised, because once you've been caught out by your own method you start looking for the next hole. There's a consistency trap in asking the workflow question first. Somebody tells you "yes, this would change my workflow," and then the price question lands right after, and now answering honestly about money feels like contradicting yourself. So the fix has to include order control: split the respondents, give half the price question first, and see if the numbers differ between the two groups.

The rest of the redesign is dailmarket's. The question on the form changes from "would this help you" to "would you pay $39 for this," the actual price, no softening. A yes doesn't lead to a thank-you page anymore. It goes straight into the pay-now stage. And the kill rule stays exactly where it was: fewer than ten price-yeses inside 48 hours, the product is dead, no appeals about strong interest.

The part that keeps nagging at me is the counterfactual. My gate returned zero, so gate v1 and gate v2 kill the same product and nothing changes. But suppose the tally had come back with six people saying the workflow question sounded great. I would have started building. Six free yeses would have felt like momentum. That's the version of the story where the leaky gate costs me weeks of work instead of saving me from it, and I only found out about the leak because a stranger happened to be reading.

Build-in-public posts usually stop at the failure, and the postmortem really is the easy part. The gate did what gates are supposed to do. Writing this follow-up was harder, because it meant saying the measurement instrument itself was broken and I didn't notice. If you run your own validation gates, the design is yours to take: split your sample, put the price question first for half of them, and make the form ask for money instead of enthusiasm. And when the next round runs, I'll put the numbers up here either way. That's the only part of this whole exercise I'm actually sure about.

The numbers above are from my own 48-hour presale gate, which closed on October 4. The redesigned gate runs on the next validation round.