A client's internal app (an inventory dashboard connected to 4 live Shopify stores) had been built fast, the way a lot of tools get built now. Then an alert came in from an outside scanning service: the repo was public.
I traced it the same hour, and the real problem wasn't the repo. It was a database rule that had survived from the prototype:
create policy "Allow all for demo" on orders
for all using (true);
Every table had one. Anyone holding the public anon key (the one that ships in every frontend bundle) could read the orders table, read 4 live Shopify admin tokens, and write to the table that decides who's an admin.
I confirmed it with nothing but the anon key, then locked it down.
How this happens to people who know better
Nobody writes using (true) on purpose for production. It happens like this:
- You prototype fast, often with an AI builder, and RLS gets in the way, so a "demo" policy goes in.
- The demo works and becomes the product.
- The repo gets flipped to public to share a preview.
- The anon key, which is public anyway, now opens everything.
Each step on its own looks harmless. Together they make a breach. AI tools have made steps 1 and 2 take minutes, which is why this keeps coming up so much in 2026.
The first 24 hours
- Repo set to private.
- Revoked all table access from
anon. Every open policy becameto authenticated using (is_staff()). - Token columns are readable by the service role only, enforced with column grants, not just policies.
- Sign-up turned off. It was open on an internal tool, so anyone could have made an account.
- A shared
checkCaller()guard went onto 18 edge functions. The anon key is a valid JWT, so "requires auth" on its own proves nothing. - Edge logs showed no third-party reads, but they only go back 24 hours. So I treated every token as stolen anyway.
The part most guides skip: rotation
Re-running Shopify OAuth does not kill old tokens. They stay valid until you rotate the app's client secret and revoke the old one. The order that actually worked:
- Find every place the secret lives:
.envfiles, Vercel env, Supabase secrets, database rows, webhook URLs. - Rotate the client secret and mint new tokens.
- Update all of those places, then redeploy.
- Revoke the old secret.
- Test from the attacker's side: old token → 401, new token → 200, unsigned webhook → 401.
We also rotated 47 webhook URL tokens. And when I redeployed, Vercel blocked two builds for a known framework CVE, so "fix the leak" turned into "patch the framework" in the same hour.
What I run now, every day at 10:00
A single script scans everything I can reach:
-
Every local repo: tracked
.envfiles, plus secrets in files and git history - GitHub: any repo that's public
-
Every Supabase project: RLS off,
using (true)policies, secret columns visible to anon,security definerfunctions callable by anon, public buckets, sign-up state, edge functions without a guard - Vercel: env vars marked public that look like secrets
-
Live sites: the HTML and JS bundles, checked against my own
.envvalues -
npm auditon production dependencies
It writes a report and fixes nothing on its own. Real findings get fixed. Intended exceptions go into an allowlist with a written reason, and nothing gets quietly ignored.
The checklist I now apply before anything ships
- Repo private by default,
.env*in.gitignorebefore the first commit - Only anon/publishable keys in frontend code
- RLS on every table, never
using (true)for anon on private data - Webhooks verified by signature with a constant-time compare
- Verify like an attacker: the anon key and no auth against tables, RPCs and functions
AI made building ten times faster. It did nothing for the security review. If you run tools like this for clients, automate the check before a stranger runs it for you.
More on how I build and secure commerce systems: fahadazhar.site
Top comments (10)
Treating every token as stolen despite clean 24-hour logs is the right call — logs that short can't prove a negative.
The part I'd steal for my own projects is the exceptions list with written reasons; without it, a daily scan just trains people to ignore the alerts it raises.
Exactly. 24 hours of clean logs only tells you nobody used the tokens in those 24 hours. And yes, without the exceptions list the report was getting noisy fast: a few findings are intended, like a public chat widget that writes anonymous sessions, and seeing those every morning would have trained me to skim. Writing the reason down also forces the question "is this really intended?" once.
That chat widget example is a good one — an intended finding that keeps surfacing is exactly how a scan loses trust.
One thing I'd add is a review date on each exception, so an 'intended for now' decision gets revisited instead of living forever.
If the reason ever stops matching what the widget actually does, that date is what catches it.
Ha, that's the gap in my setup right now. Every exception has a reason, but nothing makes me look at it again, so "intended for now" quietly turns into "intended forever". Your last line is the real point: the reason can be true the day I write it and wrong six months later. A review date on each entry is next on my list. Thanks, this thread has made the scanner better.
The using (true) demo policy is the silent killer here. The rotation part is what most guides miss—re-running OAuth doesn't kill old tokens, so rotating the client secret is mandatory. Great reminder that AI makes building fast but doesn't do the security review.
Thanks. The OAuth part surprised me too: you re-run it, get a fresh token, and assume the old one is dead. It isn't until you rotate the secret and revoke the old one. That's why the last step is testing with the old token on purpose and expecting a 401.
Testing with the old token to ensure a 401 is the only way to truly prove it's dead. Assuming a token expired just because you re-ran the OAuth flow is exactly how these leaks slip through the cracks.
Exactly. The worrying part is that nothing tells you. The old token keeps working without a sound, so if you never test it, you'll never know. Now that 401 check is a fixed step on my list whenever I rotate anything.
The attacker-side verification is the crucial closing step: “we deployed a fix” is not the same as proving old credentials fail and the intended path still works. The daily scanner is also better as a report than an auto-remediator; an explicit allowlist with an owner and expiry turns exceptions into reviewable risk instead of permanent prototype residue.
Agreed on both. The expiry point is a fair hit. My allowlist has a written reason for every exception, but no owner and no expiry date, so an exception from last month looks as valid as one from today. I'm adding a review-by date to each entry, and the scan will flag any that have expired. Thanks, that's a better design than mine.