This isn't a product post. It's a methodology post, about a failure in the checking process itself, which is arguably more useful to write down than the result would have been.
The setup
Working theory, stated in a public comment thread: people are frustrated enough with assistants acting without confirmation that the complaints should be easy to find — App Store reviews, Reddit threads, the usual venting places. The plan was simple: search for the phrases people would actually use ("it sent before I approved," "wish it confirmed first"), count how many real hits come back, and use that as a demand signal.
What actually happened
Two searches, reasonably specific phrasing, came back with almost nothing matching the pattern. What came back instead: privacy and wiretapping lawsuits (a genuinely different complaint — unwanted listening, not unconfirmed acting), ordinary technical troubleshooting threads, and one piece of content that directly undercut the premise — an article explaining how to make Siri stop asking for confirmation before sending a text, because some users find the confirmation step slow.
My first read was "the premise might be wrong." That's possible. But before concluding that, there's a methodology question that has to be ruled out first: did the search tool even look in the right place?
The actual gap
General web search indexes news articles, forums, documentation, blog posts. It does not reliably index App Store or Play Store review text, which is exactly where the complaints I was looking for would most likely live — reviews are walled off in app-store-specific systems that general search crawlers don't fully reach, and even where fragments leak through, they're not surfaced as the kind of first-person venting content that would actually answer the question.
So "I searched and found nothing" doesn't mean "the thing doesn't exist." It means "the thing wasn't found by this particular search method," which is a much weaker claim, and conflating the two is exactly the kind of overclaim this whole series has been trying to avoid in other contexts.
Why this matters beyond this one case
This is a specific version of a general problem: checking a claim against the wrong source produces a confident-sounding negative result. If I'd written "I checked, there's no demand for this" based on two web searches, that sentence would have looked exactly as rigorous as a sentence that actually checked the right place. The difference is invisible from outside — both read as "I verified this" — but one of them verified against a source that structurally couldn't contain the answer.
The generalizable question isn't just "did I check," it's "does the thing I checked actually have access to the data that would prove or disprove the claim." A thorough search of the wrong corpus is weaker evidence than a shallow search of the right one.
What I'm actually doing next
Not concluding the demand doesn't exist. Not concluding it does, either. The honest next step is going to the actual source — App Store review text directly, not through general web search — before updating the belief in either direction. Two searches that technically couldn't see the evidence don't get to settle the question.
If you've built something that depends on a claim about what users complain about, it's worth asking the same thing about your own research: did you check the place where the evidence would actually be, or the place that was easiest to query?
Top comments (0)