A wrong RLS policy doesn't throw an error, it just returns the wrong rows, so it ships untested and you find out when data leaks. rlsautotest generates the pgTAP tests and the seed data straight from your policies, so you can catch it. Free and open-source.
The problem: RLS fails quietly
Row-Level Security is what keeps one tenant's data from another's.
When it's wrong, it doesn't error. It returns the wrong rows.
- Too strict → empty results → your app looks broken.
- Too loose → rows the user shouldn't see → your app leaks.
Neither raises an exception. You find out from a confused user, or a security report.
And "it works in the SQL editor" proves nothing. The SQL editor bypasses RLS, so your policies were never exercised.
Why nobody tests it
Supabase's own docs admit writing pgTAP tests for RLS is "inaccessible to most web developers."
To test one policy by hand you have to:
- create a few users (owner, other user, anon, role-holder)
- insert rows owned by different users
- become each identity (set JWT claims + role)
- run SELECT / INSERT / UPDATE / DELETE as each
- assert exactly which rows each can and can't touch
Per table. Per policy.
And the seed data matters as much as the test. Assert "the owner sees their row" against an empty table and it passes while proving nothing.
Most teams take one look and skip it. So the actual security boundary ships untested.
What a real RLS test checks
From each identity's point of view:
owner → sees and changes their own row
other user / tenant → can't see it at all
anon **→ blocked
**role-holder → exactly the access the role grants
All against a row really owned by the identity under test.
And "denied" has two flavors a good test keeps apart:
row-level filtering → zero rows
missing grant → permission error
That's a lot of careful setup, for every policy.
Generate it instead
I got tired of doing this by hand, so I built rlsautotest **(open-source, Apache-2.0).
**Repo: https://github.com/unitautogen/rlsautotest#readme
Not Supabase-only. It runs on any Postgres (Neon, RDS, your own server). I've tested it most on Supabase, so the examples here are Supabase-flavored.
Point it at a disposable copy of your DB. It reads your policies from the catalog and generates the tests and the seed data:
bashpip install rlsautotest
# quick check: who can touch what, as an HTML report
rlsautotest --db-url "$DATABASE_URL" --schema public --html rls-report.html
# or emit a native pgTAP suite to commit + run in CI
rlsautotest --db-url "$DATABASE_URL" --schema public --emit supabase/
The report reads like a permissions table:
notes SELECT INSERT UPDATE DELETE
service_role ✓ ✓ ✓ ✓
authenticated, authorized ✓ ✓ ✓ ✓
authenticated, not authorized · · · ·
anon · · · ·
You're hunting for one thing: a ✓ where a · should be. An anon or unauthorized user who can act. That's a hole, and it jumps out.
Two things make it trustworthy:
Real seed data. It works backward from each policy to a row actually owned by the identity, so green means real isolation, not an empty-table pass.
No false greens. Anything it can't verify soundly, it marks instead of faking.
And --report exits non-zero on a leak or an RLS-off table, so it drops straight into CI.
The bug that passes code review
The kind of thing this catches that humans miss:
A table has two permissive UPDATE policies. Each one's WITH CHECK limits the value a row can become (one status per role). But the check for who may write sits only in USING, not in WITH CHECK.
Here's the trap:
Postgres OR-combines every permissive policy's WITH CHECK, independent of which USING matched.
So the effective check becomes the union of all of them, with the "who" gone. Any identity that can touch the row can write any value any policy allows. A role can set a status its own policy forbids.
Each policy looks correct alone. The leak only exists in the combination. rlsautotest enumerates the value space per identity and shows you exactly which forbidden value slips through.
What it won't do (on purpose)
It proves your DB enforces what your policies declare, not what you intended. A wrong-but-consistent policy is confirmed, greenly.
A command with no policy shows as blocked and isn't asserted unless you opt in.
A policy behind an opaque function is reported, not faked.
A tester that emits confident checkmarks it can't back up is worse than no tester. So when it can't prove something, it says so.
Try it
Got policies you're not 100% sure about? Point it at a throwaway copy and read the report. One command. Worst case, you confirm you're fine.
Repo **+ **docs: https://github.com/unitautogen/rlsautotest#readme
Top comments (0)