DEV Community

Blueticks
Blueticks

Posted on

Two of my measurements lied to me. The third one had a control

I write a blog for a very small product and I publish on three platforms. For about a week I thought I knew which of my articles search engines had found. I was wrong, and the way I was wrong is more useful than the answer.

Here is the short version. I was using a site: query to ask which of my pages an engine knew about. It told me two of my five articles on one platform were known. I wrote a whole post about that number. Then I noticed something that did not fit, and the number turned out to be worthless.

The thing that did not fit

One of my articles had been ranking second on a real content query since the morning of 31 July. I had written that down at the time, with the ranking table, because it was the first time anything I wrote had placed on a query somebody might actually type.

On 1 August at half past twelve, the site: query listed only two of my articles, and that one was not among them.

Both facts were in my own notes. I had never put them side by side. A page cannot rank second on a query without being in the index. So for more than twenty four hours I had been counting a page that was ranking as a page that did not exist.

The operator under reports. It does not lie in a dramatic way, it just quietly returns less than it knows, and nothing about the empty result tells you it is incomplete. That is the whole problem: a measurement that returns nothing looks exactly like a measurement that found nothing.

The replacement that failed the same way

The obvious fix is to stop asking the engine what it knows about a site, and start asking whether it can find a specific page. So I took an exact sentence from the body of each article and searched for it in quotes.

First check: does the engine honour quotes at all? I invented a sentence that exists nowhere, searched it, and got zero results. Good. Quotes work.

Second check: is my witness sentence actually mine? The first one I picked was a line from official documentation that my article quotes in a block, with attribution. The engine returned the documentation, not me, which is correct behaviour and useless for my purpose. Fixed by excluding quoted blocks when choosing the witness.

Third check, and this is the one that killed it: take an article I know for certain is indexed, because it ranks second on a query, and search an exact sentence from its body.

Zero results.

The page ranks. Its own sentences do not find it. Whatever that search does, it is not looking at the same index that produced the ranking. I deleted the tool. A tool that fails on a case you know to be true is not a tool that needs tuning, it is a tool that measures something else.

What actually worked, and why

The third method is duller and it works: search the exact title of the page.

A title query is close to navigational. If the page is in the index, it should come back first. And crucially, that expectation gives you something the other two methods never had.

Four of my pages come back at rank one on their own titles. That is the control. It means the method is capable of finding my pages, which means that when it does not find one, the silence carries information.

This is the rule I should have had from the start, and it is the only thing in this post I would call generally useful:

A measurement with no positive control measures nothing. Before you believe a negative result, require a case you know to be true to come back green.

The site: count had no control. I never checked that it could find anything of mine, so I had no way to read its silences. The sentence search failed its control and I only ran that check third, after building the whole thing.

The answer, once the instrument worked

Sixteen of my article pages, each queried on its own exact title, ages equalised to the same three day window:

Platform Found Of
Platform A 4 4
Platform B 2 4
Platform C 0 4

I checked the zero twice before believing it, because a clean zero is usually a bug. It is not a URL format problem: no alternative address for those pages comes back either. It is not a duplicate content effect: all four titles are unique to that platform. On one of those titles the engine cheerfully returns my article from a different platform, at rank one, and still not the page whose exact title I typed.

I am naming no platforms here, for a reason I will come back to.

The side effect I was not looking for

Two of my articles existed on two platforms, because cross posting felt free. It is not free.

Where the same title exists twice, the engine keeps one copy and drops the other. In both of my cases it kept the same platform, the one that scores four out of four above. The second copy is not a second chance at being found, it is a slot spent. Publishing the same text twice cost me a publishing slot that a different article could have used.

So: one article, one platform. I wish I had worked that out before writing twenty of them.

The limits, stated plainly

This is one engine. Everything above describes what one search engine returns for my pages. Another engine may hold the missing ones. What I can say is narrow and it is enough to decide where to publish next: on this engine, my pages on one platform are not retrievable even by their full titles, while my pages of the same age on another platform all are.

I am also not claiming the zero platform is bad at anything else. It may well be excellent for readers who are already there. It just is not a way to be found.

That is why I have not named them. A three day sample on one engine is enough to change my own publishing schedule and nowhere near enough to tell you where to publish yours. Run it on your own pages: query each one on its exact title, and check that at least one of them comes back first before you believe any of the misses.

Disclosure and the boring detail

I build BlueTicks for Gmail, an extension that shows WhatsApp style ticks in your Gmail sent list, one tick sent and two blue ticks opened. It costs 4 dollars a year and there is a free tier. It is not relevant to any of the above except as the reason I care whether my writing is findable at all. You can find it at blueticks.io.

The measurement scripts are thirty lines of Node against the browser's own protocol, with one browsing context per query so the engine does not treat the run as a burst. The interesting part was never the code. It was running the control first, which I did third, twice.

Top comments (0)