DEV Community

Daniel Pertu
Daniel Pertu

Posted on

Our 10 per cent trace sample still contacts Sentry on every page view, to tell it about the 90 per cent we threw away

Notifio is a desktop app, and its website is 38 pages of static text with no analytics on it. I wanted to confirm that second part properly rather than from memory, so I loaded all 38 URLs in a scripted Chromium and recorded every request, cookie and storage key.

The result was better than I expected in one way and more interesting than I expected in another.

requests to https://notifio.app                                 1,091
requests to https://o4511224369315840.ingest.us.sentry.io          37
cookies set after 38 page loads                                     0
localStorage keys                                                   0
sessionStorage keys                                                 0
Enter fullscreen mode Exit fullscreen mode

Two origins. No Google Analytics, no tag manager, no Meta pixel, no consent banner because there is nothing to consent to. Our privacy policy says we do not use any of that, and the network log agrees.

Then there is the second line. Thirty-seven requests to Sentry across 38 page loads, which is one per page view, on a site where nothing threw an error. That is not what sampling is supposed to look like.

What the request is

Our browser configuration is nine lines, and the sampling decision is one of them:

Sentry.init({
  dsn: process.env.NEXT_PUBLIC_SENTRY_DSN,
  tracesSampleRate: 0.1,
});
Enter fullscreen mode Exit fullscreen mode

Ten per cent of page loads should send a performance trace, and the other ninety per cent should send nothing. So I captured the bodies. Every one of them was the same 140 bytes:

POST /api/4511779378167808/envelope/
content-type: text/plain;charset=UTF-8

{}
{"type":"client_report"}
{"timestamp":1795084980.719,"discarded_events":[{"reason":"sample_rate","category":"transaction","quantity":1}]}
Enter fullscreen mode Exit fullscreen mode

It is not an error and it is not a trace. It is a receipt for the trace we decided not to send. The SDK keeps a tally of events it dropped and why, and flushes that tally as a client_report so the numbers in your Sentry project are honest about what is missing. sample_rate is the reason, transaction is the category, and the quantity is one, because one page view produced one dropped transaction.

The timing says the same thing. Navigating through four pages with two and a half seconds on each produced three reports, each landing within fifty milliseconds of the moment I left a page:

[   0 ms] navigate to /
[2,926 ms] navigate to /terms
[2,975 ms] POST client_report      <- the report for /
[5,522 ms] navigate to /privacy
[5,569 ms] POST client_report      <- the report for /terms
[8,110 ms] navigate to /pricing
[8,268 ms] POST client_report      <- the report for /privacy
Enter fullscreen mode Exit fullscreen mode

It is flushed on page hide, so it is the exit toll rather than the entry fee. One page view, one connection, whichever side of the sampling line the page view falls on.

So what does sampling buy

The payload, and nothing else. A kept page view sends a transaction, which is kilobytes of span data. A dropped page view sends 140 bytes. That is a real saving on Sentry's ingestion bill and on the reader's bandwidth, and it is worth having.

What it does not buy is the absence of a request. Every reader of every page on the site makes one cross-origin POST to a host in the United States, and an HTTP request carries things a body does not:

what the discarded-event report carries   IP address (from the connection)
                                          User-Agent
                                          a timestamp
                                          a count and a reason
what it does not carry                    the page URL
                                          any identifier we assigned
                                          any cookie, because there is not one
Enter fullscreen mode Exit fullscreen mode

That last row is worth being precise about. The Referer on all three requests read exactly https://notifio.app/, on pages that were not the home page, because Chrome's default referrer policy reduces a cross-origin request's referrer to the bare origin. Sentry learns that somebody loaded something on notifio.app, not which page. The URL only travels when there is a real error or a sampled trace to attach it to.

Reading our own privacy policy against the log

Section five of notifio.app/privacy says this:

Sentry (sentry.io) handles error and performance monitoring for our website. When a page you visit throws an error, or is included in a small sample of traced page loads, Sentry receives technical diagnostic data which can include your IP address, browser, and the page URL.

Measured against the log, that sentence is accurate about the payload and silent about the connection. The two cases it names are the two cases where diagnostic data is sent, and the list of what can be in it is right. What it does not tell a reader is that the other ninety per cent of page views also reach Sentry, carrying an IP address and a User-Agent to say a transaction was discarded.

I do not think that sentence was written to hide anything, because the person who wrote it did not know. I did not know either, and I wrote the nine lines of config. It is the kind of thing you only find by watching the wire rather than reading the code, which is also how I found out that the alert request our desktop app sends has nine fields while the policy names two.

What I am doing about it

Three options, and they are not equivalent:

  1. sendClientReports: false in the init call. One line, and it stops every one of these requests. The cost is that our Sentry project loses the record of what it is missing, which is a real loss when you are trying to work out whether a quiet day is a quiet day or a broken SDK.
  2. Amend the policy sentence so that it describes the connection as well as the payload. Honest, free, and it leaves the diagnostics intact.
  3. Move the whole thing behind our own domain with a tunnel route, so there is no cross-origin request to a third party at all. This is the version I like, and it is also the one that costs the most, because that route then handles traffic on every page view and has to be rate limited like the eight API routes we already have.

I am doing 2 now and 3 later. Not 1, because the error reporter is the only way I hear about a page failing for a visitor in a country I have never tested from, and it is already the single largest thing we ship to a browser without me pretending it is free.

If you run Sentry in a browser, the measurement takes five minutes: open your site with the network panel filtered to your ingest host, click through four pages, and read the bodies rather than counting the rows. The rows look like noise. The bodies tell you what your sample rate actually does.

Top comments (0)