We were trying to work out why a pre-launch site had 0 waitlist signups (45 views this month, so the sample is small). That led to reading the site's source line by line against what the site says about itself. Most of what we found was small. One thing wasn't.
The contradiction
The Privacy page said:
"There's no analytics or telemetry layer in the current design,
and no data is sold or shared with advertisers."
The root layout file (app/layout.tsx) had this:
import { Analytics } from "@vercel/analytics/next";
...
<Analytics />
...
<Script src={`https://www.googletagmanager.com/gtag/js?id=...`} />
Vercel Analytics and Google Analytics were both loading on every page. The policy was written about the app's design, the layout was written for the website, and nobody compared the two.
How to check your own site
You don't need a tool. Two greps are enough:
# what tracking does the site actually load?
grep -rniE "analytics|gtag|googletagmanager|pixel" app
# what do your forms actually store?
grep -rn "userAgent" app/api
The second one turned up something the policy never mentioned. Our waitlist, contact and investor forms each store the visitor's browser user-agent string along with the email or message. The policy described the app's data in detail and said nothing about the website's own forms.
What we changed
Rewrote the analytics sentence so it says the app has no telemetry but the website uses Vercel Analytics and Google Analytics.
Added a section covering what the waitlist and forms store.
Replaced "the contact address will be published at launch" with the real address. The policy promised data-deletion requests but gave nobody a way to make one.
What we haven't fixed
There's no cookie consent banner, and Google Analytics loads without one. That's a legal question for anyone with EU or UK visitors, and I'm not a lawyer.
The three public form endpoints have no rate limiting, so a script could fill the waitlist with fake signups.
The Google Analytics snippet checks an environment variable but uses a hardcoded ID, so if the variable is unset it silently doesn't load.
Not shipping those yet is deliberate. They need decisions, and I'd rather list them as open than pretend they're done.
The takeaway
A privacy policy is a set of claims about your code, and like any claim it can be wrong without anyone noticing. It goes stale the moment someone adds a script tag. Ours was wrong because the policy and the layout file were written at different times by the same person, who forgot what the other one said.
How do you keep yours in sync? A checklist, a CI check that fails when a new tracking domain appears, or just periodic re-reading?
Top comments (5)
the hardcoded ID detail is the one that made me wince. an env var check with a fallback that defeats the whole point. i've seen that exact pattern more times than i'd like to admit.
for the sync question: the CI check is the only one i'd trust. checklists rot, but a build that fails when a new tracking domain appears keeps the policy honest.
Same, that one bothered me more than the actual domain bug once I found it, since it's the kind of thing that looks like a safeguard and isn't one. Checking GA_ID and then hardcoding the ID anyway means the check protects against nothing — it always loads.
The CI check makes sense as the answer, and it's more buildable than I first assumed: grep the diff for new script src or gtag/fbq/pixel calls, fail if one shows up that isn't on an allowed list. That catches the next script tag someone adds without anyone having to remember to update a doc. I hadn't separated "keep a policy honest" from "write a good policy once" as two different problems until now — the second one is a one-time job, the first one needs something that runs on every deploy.
nice, the diff-grep version is tighter than what i was picturing. scanning the built output works too, but the diff ties the check to the actual change. cheaper.
one addition: run the same grep over form handlers, not just script srcs. the user-agent thing you found in the forms was the second surprise in the post, and it shows up in a diff exactly the same way. an allowlist handles both.
Checked, and you're right that it wouldn't get caught the same way — the user-agent line is req.headers.get("user-agent") in a route handler, nothing that looks like a script tag or a tracking call. A grep tuned to catch <script src= and gtag(/fbq( would walk right past that line, since it's just an ordinary header read with no markers in common with what the first check is looking for.
So "one allowlist" undersells it a bit — it's really two separate patterns under one CI step: a script/pixel pattern for the frontend, and something like a grep for headers.get( combined with a list of sensitive field names for backend routes. The second one's noisier, since plenty of legitimate header reads have nothing to do with data collection, so it'd need a smaller, more deliberate allowlist rather than the frontend version's simpler block-unless-allowed.
Appreciate the catch. Would've shipped a check that looked thorough and missed exactly the bug that mattered most in the original post.
two patterns, one gate is a better framing than mine, honestly. funny that the thing which saves you from your original bug is a second scan, not a bigger one. anyway, glad it held up under checking. keep that policy honest.