DEV Community: ShipSafeScan

Your RLS is on and the table is still readable

ShipSafeScan — Fri, 04 Sep 2026 08:43:22 +0000

A while back I wrote up the pass I run over code an assistant wrote for me: secrets in git history, keys leaking into the client bundle, API routes with no guard, stale dependencies, wildcard CORS. Every one of those lives in your application code.

There is a layer underneath that, and it fails differently. If you are on Supabase, or any Postgres with Row Level Security, your database is reachable from the browser by design. The anon key is meant to be public. The thing standing between a stranger and your profiles table is not your login screen and not your API route. It is your RLS policies.

The failure I keep running into is not "I forgot about RLS." It is "RLS is on, the dashboard shows a green shield, and the table is still readable by anyone." Those are not the same state, and the dashboard does not really distinguish them for you.

One caveat before the commands: run these against a project you own. Pointing them at someone else's project is unauthorized access, and in most places that is a crime, not a code review.

The one test that matters

Forget reading the policy list for a minute. The question a stranger asks your database is very simple, so ask it the same way they would: with the public key, no session, from outside your app.

# Project ref and anon key are both in your client bundle already.
# This is exactly what an unauthenticated visitor can send.
curl "https://<project-ref>.supabase.co/rest/v1/profiles?select=*" \
  -H "apikey: <anon-key>" \
  -H "Authorization: Bearer <anon-key>"

Three outcomes:

[] with a 200. The table is protected and empty for this caller. Good.
A permission error mentioning row level security. Also good.
Rows. Your data is public, whatever the dashboard says.

Run it for every table that holds anything user specific. It takes about ten seconds per table and it answers the real question, which is not "did I enable a feature" but "can a stranger read this."

Then do the same for writes, because read policies and write policies are separate things and I have shipped tables where only one of them was covered:

# Can an anonymous caller insert?
curl -X POST "https://<project-ref>.supabase.co/rest/v1/profiles" \
  -H "apikey: <anon-key>" -H "Authorization: Bearer <anon-key>" \
  -H "Content-Type: application/json" \
  -d '{"display_name":"anon write test"}'

# Can an anonymous caller delete someone else's row?
curl -X DELETE "https://<project-ref>.supabase.co/rest/v1/profiles?id=eq.<some-id>" \
  -H "apikey: <anon-key>" -H "Authorization: Bearer <anon-key>"

Clean up any rows you create while testing.

Why it happens

Four patterns account for most of what I find.

1. RLS is off on that table. A table created from the SQL editor does not get RLS just because your other tables have it. When RLS is off, policies on the table are irrelevant, and the anon role reads everything. Check the flag directly instead of trusting your memory:

select c.relname as table_name, c.relrowsecurity as rls_enabled
from pg_class c
join pg_namespace n on n.oid = c.relnamespace
where n.nspname = 'public' and c.relkind = 'r'
order by c.relrowsecurity, c.relname;

Anything with rls_enabled = false is open to the anon key.

2. A policy that allows everything. This is the one that produces the green shield and the public table at the same time. When you are stuck and asking an assistant why your query returns nothing, the easiest fix to reach for is a policy that permits the read unconditionally. It works, the error goes away, and the door is now open.

-- Enabled, has a policy, protects nothing.
create policy "enable read access for all users"
  on public.profiles for select
  using (true);

List what you actually have:

select tablename, policyname, cmd, roles, qual, with_check
from pg_policies
where schemaname = 'public'
order by tablename, cmd;

Read the qual column. If it says true on a table with user data, that is your bug. The ownership version is what you usually meant:

create policy "read own rows"
  on public.profiles for select
  to authenticated
  using (auth.uid() = user_id);

3. Some commands are covered and others are not. Policies are per command. A table can have a careful select policy and nothing for update, which means the read is locked down and the row is still editable. Group the pg_policies output by table and check that every command you allow from the client has a policy behind it. Note that using decides which existing rows a statement can touch, while with check decides what a row is allowed to look like after an insert or update. Write policies generally need both.

4. The service role key is in the client. The service role key bypasses RLS completely, by design, so it belongs on a server and nowhere else. If it ends up in a component, an edge config, or anything with a public env prefix, every policy above stops mattering. In my own project the server client is a separate module with a comment at the top saying it must never be imported from client code, and the key is read from a non public env var. That comment has saved me at least once.

# Should return nothing outside server-only files
grep -rn "SERVICE_ROLE\|service_role" src/ | grep -v "server\|lib/db"

What I do now

Before anything with real users touches it:

List tables where relrowsecurity is false. Enable RLS on every one that holds user data. Enabling RLS with no policies denies access, which is the safe direction to fail.
Dump pg_policies and read every qual. Replace each true with an ownership check.
For every table, confirm there is a policy for each command the client is allowed to run, and that write policies carry with check.
Run the logged out curl against every table, read and write, and keep going until the only tables that answer are the ones meant to be public.
Grep for the service role key outside server code.

The part I want to leave you with is step 4. Policy review is reading, and reading is where you see what you intended. The unauthenticated request is the only one of these that tells you what you actually shipped.

If you want the application layer pass that goes with this, the earlier post covers secrets, bundles, route guards, and dependencies. I also put the checks I run most often into a scanner you can point at a public repo at https://shipsafescan.com

A 15-minute security pass for code your AI wrote

ShipSafeScan — Sun, 30 Aug 2026 11:37:04 +0000

When you're vibe-coding a side project, "working" and "safe to ship" feel like the same thing. They aren't. LLM-generated code is really good at making the happy path run, and really good at quietly skipping the boring parts that keep you out of trouble: auth checks, secret handling, dependency hygiene.

I found this out going back through my own repos. I'd let an assistant scaffold a few apps, shipped them, and then went back to audit what I'd actually pushed. It wasn't pretty. Nothing exotic, just the same handful of mistakes over and over. So this is a field guide to those mistakes, with the exact commands I now run before I call something done. No tools required to follow along; everything here is grep, npm, and reading your own diff.

A quick caveat on how to read this: these are patterns I keep seeing in code that LLMs generate, described qualitatively from my own repos. I'm not putting a percentage on it. I don't have a representative sample, and neither does anyone who tells you "X% of AI code is insecure" without a citation. Treat it as a checklist, not a statistic.

1. Secrets that made it into git history

The classic. You paste an API key into .env to test something, later add .env to .gitignore, and feel safe. But if the file was ever committed before you ignored it, the key is still sitting in your history forever.

Check the working tree first:

# Obvious offenders in files you can see right now
grep -rEn "(api[_-]?key|secret|token|password|BEGIN.*PRIVATE KEY)" \
  --include="*.{js,ts,jsx,tsx,py,env,json,yml,yaml}" . \
  | grep -v node_modules

Then check history, because that's where keys hide:

# Was .env ever committed at any point?
git log --all --full-history -- "*.env" ".env"

# Look for likely key shapes across all commits (example: AWS-style, generic hex)
git rev-list --all | xargs git grep -nE "AKIA[0-9A-Z]{16}|[a-f0-9]{32,}" 2>/dev/null

If you find anything: rotate the key first (assume it's burned), then worry about scrubbing history. Rotating is the part that actually protects you. History rewrites (git filter-repo) are secondary and can wait.

2. Secrets shipped to the browser

Frontend frameworks expose any env var with a public prefix to the client bundle. In Next.js that's NEXT_PUBLIC_; other frameworks have their own (VITE_, REACT_APP_, PUBLIC_). An assistant wiring up a fetch call will happily reach for whatever variable is in scope, including a server key it should never expose.

// This ships your key to every visitor's browser. Anyone can View Source.
const res = await fetch("https://api.example.com/data", {
  headers: { Authorization: `Bearer ${process.env.NEXT_PUBLIC_SERVICE_KEY}` },
});

The rule of thumb: a PUBLIC prefix means "I am fine with the whole world reading this." Publishable/anon keys are designed for that. Service-role keys, private API keys, and anything that can write or read other users' data are not.

Grep your client-side code for the prefix and eyeball every hit:

grep -rn "NEXT_PUBLIC_\|VITE_\|REACT_APP_" src/ | grep -iE "secret|service|admin|private"

Anything sensitive belongs behind a server route (an API route / server action), where the browser never sees the raw value.

3. API and admin routes with no auth guard

This is the one that bites hardest. The UI hides the "Delete everything" button behind a login screen, so it feels protected. But the button just calls /api/admin/reset, and that endpoint is a public URL. If the route handler doesn't check who's calling, anyone with the URL can call it directly with curl.

// app/api/admin/users/route.ts, looks fine, ships data to anyone
export async function GET() {
  const users = await db.user.findMany();     // no auth check anywhere
  return Response.json(users);
}

The fix is to check the session at the top of the handler, not just in the UI:

export async function GET(req: Request) {
  const session = await getSession(req);
  if (!session) return new Response("Unauthorized", { status: 401 });
  if (session.role !== "admin") return new Response("Forbidden", { status: 403 });

  const users = await db.user.findMany();
  return Response.json(users);
}

To find the gaps, list your routes and ask one question per file: "If a stranger hit this URL with no cookie, what happens?"

# Every API route in a Next.js app-router project
find . -path ./node_modules -prune -o -path "*/api/*" -name "route.*" -print

Then grep for the routes that don't mention a session/auth helper, those are your suspects:

grep -rL "getSession\|auth(\|requireUser\|verifyToken" $(find . -path "*/api/*" -name "route.*")

4. Dependencies with known vulnerabilities

LLMs are trained on a snapshot of the world, so they tend to reach for versions and packages that were popular at training time, sometimes pinned to something with a known CVE. You don't have to guess; the tooling already knows.

# Ships with npm, no install needed
npm audit

# Just the serious stuff, and a quick automated pass at safe upgrades
npm audit --audit-level=high
npm audit fix

If you want an ecosystem-agnostic view (or you're on pnpm/yarn and want a second opinion), OSV-Scanner reads your lockfile against the open OSV vulnerability database:

# https://github.com/google/osv-scanner
osv-scanner --lockfile=package-lock.json

Two habits worth building: don't blind-run npm audit fix --force (it can yank in breaking major versions), and re-run the audit on a schedule, because new CVEs get published against dependencies you already have.

5. CORS, rate limits, and errors that overshare

The quiet trio. None of these throws an error in dev, so they sail straight to prod.

CORS set to a wildcard lets any website call your API from a user's browser:

// Reflects "allow everyone", fine for a truly public read-only API,
// a problem the moment the endpoint does anything user-specific.
res.setHeader("Access-Control-Allow-Origin", "*");

Scope it to the origins you actually serve instead of *.

No rate limiting means a single script can hammer your login route, your signup, or your LLM-backed endpoint (that last one can also run up a real bill). Even a minimal per-IP limit in front of sensitive routes changes the economics for an attacker.

Errors that leak internals hand attackers a map. Returning the raw exception is convenient in dev and a gift in prod:

// Don't send this to the client in production
catch (err) {
  return Response.json({ error: err.stack }, { status: 500 });
}

Log the full error server-side; return a generic message and a request ID to the client. Grep for the tell:

grep -rn "err.stack\|error.message\|console.log(err" src/

The actual 5-minute / 15-minute pass

Here's the routine distilled. You can literally paste these in order.

5-minute triage (do this every time before you push):

# 1. Secrets in the working tree
grep -rEn "(api[_-]?key|secret|token|password)" --include="*.{js,ts,py,env}" . | grep -v node_modules
# 2. Was .env ever committed?
git log --all --oneline -- "*.env" ".env"
# 3. Public-prefixed secrets leaking to the client
grep -rn "NEXT_PUBLIC_\|VITE_\|REACT_APP_" src/ | grep -iE "secret|service|admin"
# 4. Known-vulnerable deps
npm audit --audit-level=high

15-minute pass (do this before anything goes to real users):

Walk every file under **/api/** and answer "stranger, no cookie, what happens?" for each one. Add the 401/403 guard where the answer is "they get data."
Confirm no Access-Control-Allow-Origin: * on endpoints that return user-specific data.
Put a basic rate limit in front of auth and any LLM/paid endpoint.
Grep for err.stack / raw error returns and swap them for generic messages + server-side logging.
Rotate any key you found in steps 1-3. Assume anything committed is compromised.

That's the whole thing. It's not sophisticated, and that's the point, the failures in AI-generated code are usually boring omissions, so a boring checklist catches most of them.

I ended up running this so often across my own projects that I got tired of doing it by hand, so I wrapped the repetitive parts (the secret grep, the dependency check, the "is this route guarded" pass) into a small scanner you point at a public GitHub repo: ShipSafeScan. It won't catch everything a careful read will, nothing does, but it's a fast first pass. Either way, the checklist above works on its own. Run it before you ship.