Earlier this month I published a short piece here arguing that my articles were being indexed in strict order of age, oldest first, with a delay of one to two days. I had five data points, they lined up perfectly, and I said so: the ordering by age is exact, with no exception.
I have since tested twenty four articles instead of five. The ordering is not exact. It is not even approximately exact.
This post is the correction, and the way the claim broke is more useful than the claim ever was.
How I test indexation
One query per article: the article's own exact title, in quotes. A title query is close to navigational, so a page that is in the index wins it. If the page does not come back on its own title, it is almost certainly not indexed.
The test is asymmetric and I want to be clear about that. A hit proves indexation. A miss strongly suggests absence but does not prove it. Everything below rests on hits where it can.
My script refuses to report a run in which nothing at all was found, because a broken measurement and an empty index produce identical output. That guard has earned its keep.
What twenty four articles say
Eight of twenty four came back on their own title.
Then the ordering. One article published on the 1st of the month is absent at four days. Another published on the 4th is present at one day. Same platform, same account, both public, both self canonical, neither carrying a noindex directive. I checked the older one specifically because it was the one that broke the pattern: it is a perfectly ordinary page, 994 words, nothing wrong with it.
So the delay is not a function of time. There is no waiting period after which a page appears. Age does not order the queue, and my earlier five point sample was a coincidence I dressed up as a rule.
That is the first correction.
The second thing, which I did not expect at all
Three days before this run, I had measured the same cohort of articles, on the same search engine, with the same exact title test. Here are the two readings side by side, restricted to the twelve articles that existed on both dates.
| Reading | dev.to | Hashnode | Medium |
|---|---|---|---|
| 2nd of the month | 4 of 4 | 2 of 4 | 0 of 4 |
| 5th of the month | 2 of 4 | 4 of 4 | 0 of 4 |
The picture inverted. Pages that were indexed are no longer returned on their own title, and pages that were absent now are.
I had been treating indexation as a ratchet: a page gets in, and it stays. It does not. An indexation snapshot is perishable, including the one you just took.
The objection I had to answer before believing any of this
There is an obvious rival explanation: my measurement is noisy, and two disagreeing readings say nothing about the index at all.
So I ran the whole test again, twenty minutes later, and compared it article by article rather than by totals. Totals matching would have proved nothing, because the same count can hide two swaps.
Zero verdicts changed. Twenty four out of twenty four identical.
That does not make the inversion certain. Twenty minutes tests short term stability of the instrument, not day to day stability, and a search engine could plausibly be steady within an hour and vary across days for reasons that have nothing to do with whether a page is indexed. What it does is eliminate the most likely rival explanation, which is the most any of us can do without access to the index itself.
What I actually believe now
Do not decide anything on one indexation reading. I chose a publishing platform on the strength of a single measurement, wrote it into my notes as settled, and it reversed within three days. The reading was accurate. The conclusion drawn from it was not.
Trust the result that does not move. Across both readings, Medium returned zero out of four, twice, and zero out of five when I widened the sample to every article old enough to compare. That is the only stable finding I have, and it is the only one I now act on. I am not claiming Medium does not get indexed in general, only that not one of the Medium pages I have tested has ever come back on its own title.
Write the decision rule before the next measurement. I have committed, in writing and in advance, to what a third reading will mean: if it confirms Hashnode ahead of dev.to, my default changes; if it reverses again, the two platforms are equivalent and indexation stops being the criterion; if it produces a third pattern, then this measure does not discriminate between platforms and I should stop choosing on it. Deciding afterwards is how you end up with a rule that fits whatever you happened to see. I wrote those three branches down before taking the reading, and I took the reading before this post went out. It is in the next section.
And publish the correction where you published the claim. The earlier piece is still up, still says the ordering is exact, and still gets read. It is linked at the top of this post, and I am adding a pointer to this correction on it rather than quietly leaving it alone.
The third reading, taken the night before this went out
I wrote the rule above before taking the reading. Then I took it, and I would rather report it here
than publish a promise and settle it quietly somewhere else.
The third reading covers all twenty four articles rather than the twelve that existed on both of
the earlier dates, so it is a comparison of rates and not of the same set. That makes it weaker
evidence than the table above, and I would be overselling it if I did not say so.
| Platform | Found on its own title |
|---|---|
| Hashnode | 4 of 6 |
| dev.to | 5 of 12 |
| Medium | 0 of 6 |
Under the rule I committed to, this is the first case: it confirms Hashnode ahead of dev.to, so my
default changes. Nine of twenty four overall, against eight three days earlier.
The part I find most useful is the column that has not moved. Medium has now returned zero on three
separate readings, across three different sample sizes, taken on three different days. Everything
else in this experiment has reversed at least once. That one number has not.
I will add that a decision rule written in advance is easier to honour when the answer is the one
you were half expecting. The test of it will be the reading that tells me to abandon something I
have just invested in.
The uncomfortable part
The five point sample was not sloppy. The articles really did line up by age, perfectly, with no exception, on the day I looked. Every number in that post was correct.
What was wrong was the sentence I built on top of them, which turned five observations into a law. There is no error message for that step, no test that fails, and no reviewer. It is the most reliable way I know of to be wrong while holding entirely accurate data, and the only defence I have found is to keep measuring the same thing until it either holds or does not.
Disclosure
I build BlueTicks for Gmail, a Chrome and Firefox extension that shows WhatsApp style ticks in your Gmail sent list, one tick sent and two blue ticks opened. It costs 4 dollars a year and there is a free tier. The measurements above come from tracking how its articles and listings get distributed, written up as it happens, including the parts where I got it wrong in public. You can find it at blueticks.io.
If you have written down a rule about indexation based on one look, the cheapest thing you can do this week is take a second look at the same pages.
Top comments (0)