On September 30 we shipped a cookie consent banner and called it review-safe in the deploy note. For the next 36 hours, it silently uninstalled our analytics from 98% of our visitors — and the scariest part is how reasonably every individual decision looked along the way.
This is a postmortem of a self-inflicted outage that never threw an error, never returned a 500, and never fired an alert. The outage was invisible by construction, because the thing it broke was the instrument you would use to notice it.
The setup
ToolVault is my side project: 172 developer tools (JSON, hashing, JWT, PDF, image work) that run entirely in the browser — no uploads, no accounts, no backend except one audited proxy route. Like every hobby site with real traffic, it monetizes (eventually) with ads, and ad programs want to see a consent flow. So the to-do was legitimate: ship a GDPR-style cookie banner before applying for ad review.
The implementation was, by every checklist we had at the time, clean:
- A banner with Accept all / Reject all / Essential-only choices
- Consent state persisted, reject semantics honored, page reloads on change
- AA-contrast UI, no layout shift, no route changes, no page structure changes
- Deploy note: "review-safe — no structural changes"
And one behavioral decision buried in the middle of it: analytics and ad scripts are injected only after the visitor clicks "Accept all."
That is the strictest, most privacy-conservative model — opt-in. If you were grading this change for compliance, you would give it extra credit.
The silence
Here is the number nobody on the team had internalized: about 98% of visitors never click any button on a consent banner. They scroll past it, or dismiss it mentally, or have trained themselves to ignore anything banner-shaped years ago. This is banner blindness, and it is well-documented in the UX literature — it just never appears in the compliance checklist.
With opt-in gating, "never clicks" means "never loads the script." Not "loads it later." Not "loads it with reduced granularity." The script simply does not exist for that pageview.
The next day, GA4 reported 8 users. The day before the deploy: ~115.
And the server access log, which listens to no consent banners, counted 446 unique human visitors that same day.
| Signal | Sept 30 (before) | Oct 1 (opt-in gate live) |
|---|---|---|
| GA4 active users | ~115 | 8 |
| Server log, unique humans | — | 446 |
Analytics coverage didn't dip. It fell off a cliff to ~2%, on the healthiest traffic day of the week.
Why it took 36 hours to notice
Day 1, the drop got read as a reporting artifact. GA4 shows a "processing incomplete" badge on recent days, and 8-vs-yesterday is exactly the kind of number a broken report card would explain away. The badge was real — GA4 reports genuinely do lag — and that's what made it such an effective camouflage. A lie is easy to catch; a technically true status message that stops applying at the exact wrong moment is much harder.
The deeper failure was structural: we had exactly one instrument, and the outage broke that instrument. A single-channel monitoring setup cannot detect the death of its own channel. It's the monitoring equivalent of the light in the fridge — everything looks fine from inside.
Day 2 morning, the cross-check finally happened: 446 humans in nginx vs 8 users in GA4 is not a reporting lag, it's a 98% disagreement. From there the diagnosis took minutes, because each step only had one possible next step:
-
curlthe homepage → nogtag.jsin the HTML - Search the codebase for what conditionally renders analytics → the new
ConsentScriptscomponent - Git timeline → shipped 20:03 the night before, labeled review-safe
Deploy the fix at 09:17. Realtime shows a live visitor within the minute. Total time from first blind pageview to fix: ~36 hours, of which nearly all were "nobody knew."
The fix: opt-out, and why it's still compliant
The repaired model is opt-out: analytics loads by default; an explicit click on "Reject all" (or "Essential only") stops it, immediately and persistently. Visitors keep full control — the reject path is honored more carefully than before, down to reloading the page so no already-loaded script survives the choice.
Opt-out-with-real-rejection is the model most of the industry actually runs (and what many regulators effectively expect for analytics-grade, non-personalized usage). Pure opt-in for measurement scripts sounds saintly and, combined with human banner behavior, functions as an analytics kill switch. Compliance is not a slider where "most restrictive" always means "most correct" — it's a trade-off that has to survive contact with how people actually behave.
One more lesson shipped with the fix, free of charge: our CDN caches HTML with s-maxage=3600, stale-while-revalidate=86400. That means after the fix deployed, the first visitor to each cached URL still got the old consent-gated page, and only triggered a refresh for the next one. With 172 tool pages and 140+ blog posts of long-tail traffic, stale copies of the blind page kept serving for up to 24 hours. "Fixed" and "effective" are different timestamps, and the distance between them is your cache TTL × your URL count.
The counterfactual that made us take it seriously
Here's the part that turned this from an embarrassing anecdote into a process change. At the time of the incident, the ad program hadn't approved the site yet — zero ads were running — so the revenue impact was exactly ¥0.
If the same gate had shipped two weeks later, with ads live at ~1,000 visitors/day: 98% coverage loss on ad inventory, at typical RPMs, is roughly ¥9,800/day ($1,400). And there is no alert for "ads didn't render" — you'd discover it in the weekly review, 5–7 days in, at ¥50k–70k. A "compliance improvement" that quietly costs a month of revenue.
Zero loss by luck is not zero risk. The only reason this was free is that we hadn't yet earned the ability to lose money.
Four guardrails we now run
- Analytics smoke test after every deploy (mandatory, 60 seconds). Open any tool page, open GA4 Realtime, see your own visit within 30 seconds. If you can't, you roll back before touching anything else. This single habit would have caught the outage at 20:05 instead of two days later.
- Cross-channel validation, daily. Analytics users vs. server-log unique humans. Divergence over 50% must be investigated that day, and "GA4 processing is lagging" is no longer an acceptable closing statement for a 98% gap.
- Any change that touches when scripts load is a revenue-pipe change. The deploy note for anything touching consent, analytics, or ad loading must carry one explicit line: who can still be measured, and who can't. "No structural change" describes the diff, not the behavior.
- Compliance changes get a business-impact review, not just a legal one. Coverage of measurement, coverage of ad inventory, revenue at risk — written down next to the AA contrast check. Our postmortem's counterfactual math (above) is the template.
Try it
The site that survived its own consent banner is ToolVault — 172 developer tools that process everything locally in your browser, which is also why it can run an honest default-on analytics setup: there's no upload pipeline to consent to in the first place.
The rest of the series covers the 100%-local architecture, auditing the one route that broke that promise, and what six frontier models saw (and missed) when auditing the lying code.
And the takeaway I'd tattoo on the inside of every deploy note: if a change gates when scripts load, it doesn't matter that it "only touches compliance" or "only touches UI" — it just changed your revenue pipes. Smoke-test the pipes.
Top comments (1)
the 98% figure is the thing nobody internalises. everyone assumes some reasonable fraction clicks, and it's just not true.
the CDN stale-while-revalidate detail is the one that would have bitten me. fix ships, hour-old HTML is still serving the old gating logic, and you're debugging a fix that's already live. that's a painful way to learn.
small counterweight to the opt-out conclusion: the third option is analytics that never enters the banner's jurisdiction. cookieless and IP-free, so there's no script-load gate to misconfigure and no deploy note to forget. curious whether that was on the table or if GA4 was already too baked in.