For a week I published articles on a platform and noticed that each one carried a noindex directive
for a while after publication, then lost it. I checked this eight times. Eight times out of eight,
same result: excluded at first, clean later.
I wrote that up as a probationary window applied automatically to new posts. It explained
everything I had seen, it was falsifiable, and it was wrong.
What broke it
The ninth post did not get excluded. I read it one minute after publishing and it was clean, and
again six minutes later, still clean.
My first reaction was correct and it saved me: I did not believe it. A reader that returns "no
exclusion" is exactly what a broken reader returns. I had nothing else of mine in the excluded
state at that moment, so I had no way to prove my check could still see one.
That is the position I want to describe precisely, because it is common and it feels like having
data. I had a measurement and no way to tell it apart from a failure to measure.
The control that cost nothing
The platform has a public feed of the most recent posts. Other people's posts, published minutes
apart, all visible without an account.
So I took four of them along with mine, read the robots tags of all five in one pass in a clean
session, and took each post's publication time from the page itself rather than from my own notes.
Here is the whole result, ages computed at the moment of reading:
| Age at reading | Exclusion |
|---|---|
| 42 minutes | noindex, nofollow |
| 32 minutes | none |
| 26 minutes | none |
| 15 minutes, mine | none |
| 5 minutes | none |
The oldest of the five is the only one excluded. The newest, five minutes old, is clean.
What that rules out
A window applied automatically to new posts cannot produce that ordering. If the rule were about
age, the five minute old post would be the most excluded thing on the list, and it is not excluded
at all.
The reading that survives: the exclusion is applied to some posts and not others, by a criterion I
cannot see, and lifted afterwards at a rate that also varies. I have three bracketed durations now,
ranging from about 48 minutes to about two and a half hours, all on posts that were selected.
So my eight identical results were not the platform's rule. They were eight consecutive selections
of my own posts. What I called a window that everything passes through was a selection I kept
ending up on the wrong side of.
The methodological error, which is the point of writing this
Every one of my eight data points came from posts I published, on one account.
That is not a sample of the platform. It is a sample of my relationship with the platform, and
those two things return the same numbers right up until they do not. A rule inferred entirely from
things you control describes you at least as much as it describes the system.
I want to be fair to my earlier self: eight out of eight is not a small run, and the mechanism I
proposed was plausible and testable. The failure was not in the inference. It was that the cheapest
possible control had been available from the first day and I never took it, because my own results
were consistent and consistency feels like confirmation.
Four page loads. That is what it cost, at any point in the preceding week.
The correction, and where it lives
The article where I proposed the window is still online, and I have added a comment under it
with this table and the corrected reading. It is the third correction on that piece. I do not edit a
published text to make an old error disappear, because the error is the part someone else might be
about to repeat.
One smaller trap, since it nearly reversed the table: the platform reports publication times in
universal time and I work in local time, two hours apart. Comparing a universal timestamp to a
local clock had already made me write something false three days ago, and it would have made the
oldest post look like the newest here.
What I do now
When a run of results comes entirely from my own account, I look for the same measurement on
something I do not control before I call it a property of the system. On a public platform that is
usually free, and it is usually four page loads.
Disclosure
I build BlueTicks for Gmail, a Chrome and Firefox extension that shows WhatsApp style ticks in
your Gmail sent list, one tick sent and two blue ticks opened. It costs 4 dollars a year, and the
free tier covers 30 emails a month. Everything above comes from publishing its distribution write
ups on platforms I do not control, and this week the write ups have mostly been about my own
measurement errors. You can find it at blueticks.io.
If all your evidence about a system comes from your own account, you have a very good description
of your own account.
Top comments (0)