DEV Community

Boris Binyaminov
Boris Binyaminov

Posted on Originally published at whittleos.com

A claim can never be promoted up the provenance ladder

The stock answer to where do I find customer problems is go read Reddit, which names a destination when the question is about a property. Destinations expire — an API closes, a forum goes private, a review site adds a login wall — and a method built on a list of sites expires with the list.

The property: somebody described it, in their own words, somewhere you can point at. Lose the third and you have a claim. Lose the second and you have a summary of a claim. Lose the first and you have your own hypothesis wearing a citation.

So the data model is a ladder of 5 provenance tiers, and the rule that makes it mean anything is that a claim can never be promoted UP it. A search snippet is real evidence that a page exists and says something; it is not evidence of what the whole page says. Anything grounded in a snippet stays capped at that tier however convincing the summary reads.

That rule is the entire product decision. Tools that collapse these tiers are not being sloppy — the collapse IS the feature, because a promoted claim reads better than a labelled one.

Why it needed a shared vocabulary rather than a convention: five hand-maintained copies once described the same tier three different ways, and one page rendered two of them at once. Now every surface imports one map.

Reachability has exactly 3 mechanisms, and each has a limit that decides what you may honestly quote from it. A search API returns results for a generated query — a snippet is not a page. Our own reader fetches a listing page and extracts entries — gated by that site's robots rules per fetch, and skipped when they say no. A documented public API, called directly.

Two consequences that cut against us, and both belong in the post rather than in a footnote.

Robots rules decide reachability and we obey them. A site that says no is skipped, and whatever it knew is absent from the run. That is a real hole in coverage, and the honest response is to say so rather than to route around it.

And the best evidence is often the least reachable. Long complaint threads behind logins, support forums that render client-side, private communities — the places people are most candid are exactly the places a fetcher cannot go. Anyone claiming complete coverage of where customers complain is describing a product that does not exist.

What one run actually read: 41 sources consulted, 24 pages opened and read in full, 56 problems recorded, of which 32 carry a link you can open — 57 percent. The other 24 are marked unverified rather than deleted, and that distinction is the method.

The gap between sources consulted and pages read is the snippet tier doing its job rather than a shortfall. And the 3 hosts that run reached are not a recommendation list — they are where one run's generated queries happened to land, for one niche, on one day.

If you are doing this by hand: keep the URL rather than the summary, record the date, mark what you inferred, and count what you could not reach. A search that returned nothing is a finding; a site that refused you is a hole whose shape is worth knowing.

The tiers, the mechanisms and the run: https://whittleos.com/guides/where-to-find-customer-problems

Top comments (0)