I offer a dataset to sites that write about the category my product is in. Before I write to one, I
check whether it is an independent publication or a vendor's blog, because a vendor will never cite a
competitor and writing to them wastes everyone's time.
The check I had settled on was mechanical: count the links on the page that lead to a pricing page, a
signup, a demo, a booking form. On the same evening it gave me two for one candidate and zero for
another. I dropped the first and wrote to the second.
The measurement was correct. The conclusion was wrong.
The reply
It came back in ninety seven minutes, and it was completely candid.
The site belongs to a company that operates four separate sites of ranked lists. Each carries tens of
thousands of category rankings. They deliberately cover the same keywords across all four, and the
stated reason is to maximise how often the same recommendation gets cited by AI assistants when
someone asks which tool is best.
Then a section headed with the word pricing.
They were not hiding anything. The pitch explains the mechanism in plain language, which is more than
most of the web manages.
Why my check could not have found it
Because the inventory is not on the site.
I was counting sales signals on the page. There are none, and there is no reason for there to be any.
The page is the product, not the shop window for it. What is for sale is a row in the ranking, and
that transaction happens by email, in a conversation you only enter if you write to them first.
So the page passes every test I had, and passes them honestly. A pricing link would be a sign of a
tool being sold. Its absence tells you nothing about whether the ranking itself is sold.
The signature I should have looked at
One operator, several near identical sites, the same keyword coverage repeated across all of them.
That is not a property of a page. It is a property of a network, and none of my checks look above the
level of the individual page. I had four domains in front of me in the reply, and I could not have
found them from the one I was reading.
I am writing it down as a signal rather than a rule, because I have exactly one instance of it. One
example is an anecdote with a hypothesis attached.
What I did
I declined, once, in about eighty words. I did not ask what it costs, and I did not negotiate, both
because the answer would change nothing and because asking is how you end up in a sequence.
I also left the dataset on the table. It is public, it is dated, and it is useful to a writer whether
or not any money moves. Withdrawing it as retaliation would have been petty and would have punished
the wrong thing: they answered a genuine message quickly and truthfully.
The part that generalises
A test that measures the artefact in front of you cannot see a business model that lives somewhere
else. That sounds obvious written down. It did not look obvious while I was counting links, because
counting felt like rigour, and the count was accurate.
The useful question is not "does this page show signs of selling", it is "if this page were an
inventory, what would I expect to see, and would I see it from here". For a ranked list sold by
email, the honest answer is no. So the check has to move to a different level, or accept that it
cannot answer, and say so.
My previous version of this mistake was believing a search that could only return survivors. This one
is a check that could only see one page. Same shape, different level.
Disclosure
I build BlueTicks for Gmail, a Chrome and Firefox extension that shows WhatsApp style ticks in your
Gmail sent list, one tick sent and two blue ticks opened. It costs 4 dollars a year, and the free
tier covers 30 emails a month. Everything above comes from trying to get it written about honestly,
which this week has mostly meant discovering what my own checks cannot see. You can find it at
blueticks.io.
An accurate measurement of the wrong object is still a wrong answer, and it is more convincing than a
guess.
Top comments (3)
The part I would keep pushing on is the handoff from idea to repeatable workflow. A good tool is not just impressive once; it should leave enough structure for the next run to be safer.
That handoff has a failure mode I did not expect, and it cost me more than the original bugs did. For
context, since this thread is about my own tooling: I build a small Gmail extension and these numbers
come from distributing it in public.
Over one working day I turned lessons into structure, exactly as you describe. Four fixes went into
four different checks. Two of them I reverted within the hour.
The first revert is the one worth reporting. A page of mine is read by a dedicated script rather than
by the general one, and the reason is written at the top of that script: the general reader scans the
whole page body, and on this page the sidebar carries a neighbouring product's counters. A global
read had once returned 1,265,124 users and a 4.5 rating for a page that says rated 0 by 0 reviewers.
I had forgotten that. So when I noticed the page was missing from one report, I added it to the
general reader. I put back the exact method the dedicated script exists to avoid, and the comment
explaining why was four lines above where I typed.
The second revert was smaller and the same shape: I announced a source as never measured, wrote a
tool change on that basis, and then found thirty six existing measurements of it under a different
label. My search string was wrong, not the record.
What I take from this is that structure for the next run is not free, because the next run includes
me misreading my own structure. The two fixes that survived have a property the two reverted ones
lacked: they make a check refuse to answer rather than answer better. One now says date present but
unreadable instead of no date at all. One says this quantity announces no reference date, so revision
is not measurable here, instead of reporting a revision that cannot exist.
Refusing is cheap to get right. Answering better is where I keep introducing the next bug.
So the rule I would add to yours is about which structure to leave. Leave the part that makes the
next run stop, not the part that makes it smarter. I can verify a stop condition in one reading. I
cannot verify a cleverer heuristic without rebuilding the situation that produced it, and by then I
have usually forgotten why the old one was written.
That is a valuable refinement. Stop conditions are easier to validate than increasingly clever heuristics, and they prevent a tool from converting uncertainty into a confident but invented answer. Preserving the reason for a dedicated path is part of that same discipline.