DEV Community

Daniel G
Daniel G

Posted on

53 of my 57 "visitors" were my own test scripts. Here is the code that fixed my count.

#analytics #webdev #marketing #sideprojects

I publish one piece a night about a one-minute habit tracker I built. Last night's piece has been live for twenty-four hours and zero people have read it. Not "few" — zero, straight from the platform's own numbers.

I am not writing this to complain about reach. I am writing it because the same blindness was sitting in my own analytics, and the fix is small enough to copy.

For thirty days my dashboard said one thing and the database said another. Every "arrival" row in my table was mine. Here is the count I read tonight, live:

30-day window: 57 arrival rows
  of which 53 were my own automated checks
  real arrivals: 4   (2 direct, 1 from an article, 1 from a Google search on a phone)
  free places claimed: 0
Enter fullscreen mode Exit fullscreen mode

Fifty-three of fifty-seven. Ninety-three per cent of my traffic was the instrumentation.

Why the usual filters do not save you

The reflex is to filter by IP, or by user agent, or to drop anything that looks like a bot. All three fail, for the same reason: your own checks are not bots, they are you, running your real browser and your real network.

  • My nightly checks load the real page in a real browser, so they fire the exact same call a reader's phone fires. There is nothing in the request that says "test" unless I put it there.
  • A user-agent rule works until the day a check uses a headless browser and a real reader uses a phone with the same engine.
  • "Ignore anything faster than 2 seconds" is a guess dressed as a rule.

The rule that actually holds: a row you cannot attribute to a name is a row you cannot count. So the check must label itself, at write time, in the data — not be filtered out later by a heuristic.

The smallest version that works

One function, one table, one name per row:

table arrivals (
  day        date,
  channel    text,     -- 'devto', 'direct', 'self-check-e2e', 'self-check-pre-guard'
  page       text,
  hits       int
)
Enter fullscreen mode Exit fullscreen mode

Every page on the site fires one POST per session per channel to a tiny endpoint. My test scripts call the same endpoint, but they pass a channel name that starts with self-check-:

// the page: one call, no cookie, nothing that follows the reader
fetch('https://omm-unlock.vercel.app/api/visit', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ channel: ch, page: location.pathname })
});
Enter fullscreen mode Exit fullscreen mode 
# the check, in the harness that walks the site
channel = f"self-check-{run_name}"   # e.g. self-check-pre-guard
Enter fullscreen mode Exit fullscreen mode

Then the readout does one subtraction, and it must show the subtraction rather than quietly doing it:

{
  "windowDays": 30,
  "total": 4,
  "testsExcluded": 53,
  "byChannel": [
    { "channel": "direct",   "arrivals": 2 },
    { "channel": "devto",    "arrivals": 1 },
    { "channel": "com.google.android.googlequicksearchbox", "arrivals": 1 }
  ],
  "testChannels": [
    { "channel": "self-check-pre-guard", "arrivals": 47 },
    { "channel": "self-check-e2e",       "arrivals": 2 }
  ]
}
Enter fullscreen mode Exit fullscreen mode

Two things in that output are doing the real work.

The excluded rows are printed, not hidden. If the harness ever fails to label itself, the excluded list is the first place I will see it — a self-check row count that suddenly explodes means a check is firing on the wrong page, and a real-channel row that crashes the total means a test forgot its name.

The Google arrival is ugly on purpose. The referrer was not google.com; it was com.google.android.googlequicksearchbox — the package name a phone hands over when someone taps a search result. A referrer parser that accepts only hostnames it recognises throws that visit away and calls it "direct". I keep unknown referrers as their own channel, so a new place that sends me a reader shows up by itself instead of vanishing into "direct". That cost me one line and found one real reader.

The channel has to survive the browse

The second bug was quieter. A reader lands from an article on page A, taps through to page B, then to the signup page. Page B's referrer is my own domain, which is not a channel. So the visitor who did the most work was filed as direct.

Fix: the channel is resolved once per tab and carried in sessionStorage, and the page's own offer links are rewritten to carry it too. No cookie, no cross-site call, nothing stored on the server about the person — one short string in the tab, gone when the tab closes.

What this does not fix

Let me be exact about what the numbers can and cannot say.

  • Page loads are not people. One person with four tabs is four loads. I have no way to know the difference, and I am not going to pretend otherwise.
  • Four arrivals is not a sample. It is four. Everything I say about why people do not claim a free place is a guess, and I will label it as one.
  • The honest reading of my own dashboard is the one nobody wants: the count was never wrong because the code was bad. It was wrong because I never asked who was standing behind each row.

The lesson worth copying

If you build your own instrumentation, spend the five minutes to make every row say whose it is. Name your tests, print what you excluded, and keep the referrers you do not recognise. The number that survives all three is small — mine is 4 — but it is the only number I can act on. A dashboard that flatters you is worse than no dashboard, because you will plan around it.

I write these one minute at a time, on a habit tracker built to photograph the minute instead of scoring the streak. The first thirty people get the app and the ebook free, no card: take one of the 30 free places.

Top comments (0)