Adding evidence of innocence made our sessions look more like bots

We build a document-analytics product. Someone shares a PDF as a link, and the owner wants to know who read it — which pages held attention, where the reader stopped, whether they came back.

The whole product rests on one number being trustworthy: how many people actually opened this. And that number is polluted by default. Corporate mail security opens every link in every inbound message. Slack, Teams, WhatsApp and LinkedIn fetch a preview card. Uptime monitors poll. If you count those as readers, you hand a salesperson a list of prospects to call who never existed.

So we score each viewing session for bot-likeness. What follows is the part that was harder than expected, including one bug where the scoring logic inverted on itself.

The shape of the scorer

Flags, each with a weight, summed and capped at 100. At or above 60, the session is marked suspicious and excluded from the owner's counts.

ua-denylist              100
no-user-agent             80
zero-engagement           60
sub-second-single-page    40
beacon-no-pages           30
render-no-pages           30
gate-email-no-pages       30

Two arms feed it: one reads the user agent, one reads behaviour — page-view rows, dwell time, the furthest page reached, and how the session ended.

Problem 1: a phone brand that matches /bot/

The obvious user-agent test is a regex of bot|crawl|spider|headless|curl|…. It works until you meet a CUBOT — a real Android phone brand whose user agent contains the substring bot. Every CUBOT owner who opened a document got classified as a crawler.

The fix is unglamorous and has to stay narrow:

const DEVICE_BRAND_FALSE_POSITIVES = /cubot/gi;
// stripped from the UA before the denylist test

The rule we wrote next to it matters more than the fix: additions here must be full brand tokens, never generic words. The tempting move after being burned once is to soften the denylist. Softening it is how you stop catching crawlers.

Problem 2: twitterbot is a crawler, Twitter for iPhone is a person

The naive denylist includes bare app names — twitter, telegram, discord. Those tokens match the in-app browsers that recipients actually open share links in. "Twitter for iPhone" is a human holding a phone.

The link-preview crawlers all identify with the -bot suffix: Twitterbot/1.0, TelegramBot (like TwitterBot), Mozilla/5.0 (compatible; Discordbot/2.0…). So the denylist carries telegrambot, discordbot, twitterbot — never the bare names.

WhatsApp is the exception that proves it needs case-by-case work rather than a rule: its crawler is WhatsApp/2.x and its in-app browser doesn't carry the token, so whatsapp stays bare.

Problem 3: the timestamp that updates without a browser

Our first behavioural test for "did a person look at this" was last_active_at > started_at, with a 1ms floor. It looked reasonable. It was wrong twice over.

That column gets bumped with no client JavaScript running at all — by a 30-minute identity-reuse path on a second plain HTTP hit, and by a public unauthenticated POST /session-activity endpoint. And of the sessions it was exonerating, thirteen had a bump under one second. The smallest was 3ms.

A blink is not a reading beat. The floor became explicit:

export const MIN_PLAUSIBLE_DWELL_MS = 10_000;

Ten seconds, and only from a source a viewer's browser actually had to run to produce.

Problem 4: the one that inverted

Here is the interesting one.

A session with zero page-view rows is suspicious — zero-engagement, weight 60, exactly at the threshold. But there are three separate pieces of evidence that such a session might still be a person:

1. The close beacon. The viewer's browser ran all the way to teardown and reported a plausible reading duration. Our own telemetry, and the only signal carrying a magnitude — time actually spent.
2. The render stamp. The PDF finished downloading and PDF.js parsed it. Also our telemetry, and proof the bytes reached the reader — but silent about dwell. It evidences delivery, not reading.
3. The gate email. Somebody typed an address into the email gate. A deliberate act no generic crawler performs — but the gate is a public endpoint and the address is attacker-suppliable.

Each got a sub-threshold weight of 30: enough to pull a session below 60 and back into the counts, not enough to clear anyone on its own.

Now read the arithmetic. A session that had both a close beacon and a render stamp would collect two flags: 30 + 30 = 60. Threshold. The session is marked suspicious.

Two independent pieces of evidence that a human read the document summed to exactly the score for having no evidence at all. More proof of innocence made the session look guiltier — not by a rounding error, but by the structure of the scoring.

The fix is to treat them as alternative evidence rather than cumulative evidence. The classifier emits exactly one flag for a zero-page session, strongest arm first:

if (input.pageViewCount === 0) {
  const closedItself =
    input.durationSource === "beacon" &&
    (input.totalDurationMs ?? 0) >= MIN_PLAUSIBLE_DWELL_MS;
  if (closedItself) return ["beacon-no-pages"];
  if (input.hasDocumentRendered) return ["render-no-pages"];
  return input.hasViewerEmail ? ["gate-email-no-pages"] : ["zero-engagement"];
}

Precedence is by evidential strength, and we wrote down why each one ranks where it does: the beacon first because it carries a magnitude; the render stamp second because it proves delivery but says nothing about reading; the gate email last because a public endpoint accepting an attacker-supplied string is the weakest thing on the list.

The generalisable lesson: in any additive risk score, exculpatory signals below the threshold are a trap. If two of them can co-occur, check what they sum to. Ours summed to exactly the number we were trying to stay under.

The absence that isn't evidence

One more rule worth stating, because it took a measurement to earn.

Our session-end beacon was measured 25% lossy. So the render stamp's presence is evidence the document reached the reader — but its absence is evidence of nothing at all. A quarter of the time it simply never arrives.

We wrote that into the source next to the field, in those words: it may exonerate and may never convict. Any signal delivered over a best-effort channel has this property, and it is easy to forget once the field exists and looks authoritative in a database column.

What it does in practice

When we turned this on and backfilled history, 14.31% of 3,046 sessions were flagged suspicious and dropped out of the owner-facing numbers.

We deliberately do not claim a filter accuracy rate, and we're not going to start. We have no labelled ground-truth set — nobody does for this — and any percentage we published would be a number about our own rules, not about reality. What we do instead is show the owner what was excluded and why, so the judgement is auditable rather than trusted.

That is also the honest limit of the whole approach. The user-agent arm catches what self-identifies. A headless browser that lies about its UA and scrolls pages on a timer is indistinguishable from a reader, and we don't pretend otherwise. What the classifier removes is the enormous, boring, automated majority — the scanners and preview fetchers that never claimed to be people in the first place.

---

The classifier is ~145 lines and has no database dependency, which is what makes it testable. It's the counting engine behind PDFTrackr — document analytics where the view counts have the automated traffic removed and itemised. Happy to answer questions about any of the above.