DEV Community: Vlad Tkachenko

Three times our Supabase scanner said all clear without looking

Vlad Tkachenko — Thu, 29 Oct 2026 10:40:23 +0000

On 11 August 2026 two apps I had permission to scan came back from our scanner with the same two rows: "Database access: Not applicable" and "Storage buckets: Not applicable". Both apps run on Supabase and call it from the browser, so neither row could have been right. Working out why turned up three separate defects, and each one turned a question we could not answer into a clean result.

The scanner checks an app from outside, with what any visitor has: the page, the JavaScript it loads, and the publishable key that ships in that JavaScript. A check can end three ways. It found something, it looked and found nothing, or it could not look. We print the second as "All good" or "Not applicable" and the third as "Couldn't check", and all three defects below turned the third into the second.

Every Supabase response in this post was measured that day against real projects, so some of them may have changed since.

1. The table list came back empty

The RLS check used to start by asking the PostgREST root, /rest/v1/, for its OpenAPI document, then probe each table it listed. On a project using Supabase's current key format, that root answers a publishable key like this:

401 {"message":"Secret API key required"}

The list was empty, nothing got probed, and the check returned no findings, which the scanner prints as "Not applicable". Our main check had no coverage at all on any app that ships one of the new publishable keys.

Discovery is now a union. The root document is still tried first, because a legacy project still serves it and its list is authoritative. After that come the table names the app's own bundle uses (.from("x") calls and /rest/v1/x paths), then 26 conventional names, up to 30 probes in all. Thirty came from the clock: a wave of four probes through our proxy measured about 480 ms, so 30 names take about four seconds of a 12-second check budget. For one day the cap was 20, because a stale .env on the scanning machine still held the old 5-second budget.

If none of the 30 names exists, the check now prints "Couldn't check". It knows the app has a database and has learned nothing about it.

RLS answers 200 either way

Fixing discovery uncovered a second problem in the same file. The old code counted any 200 from a table as readable by strangers. Row Level Security filters rows instead of refusing the request, so a protected table answers 200 with content-range: */0, the same status a wide-open table gets. On one of the two apps, profiles, notes, decks and flashcards all answered 200 and all said */0. Under the old rule every one of them would have been reported as "Strangers can read your private database tables", at critical severity because profiles holds people, about four tables that were in fact protected.

That would have been a false alarm, the opposite failure. On these apps it never fired, because the empty table list meant no probe reached it.

The probe asks for a count and never for rows: ?select=count with Prefer: count=exact and Range: 0-0. The verdict is the number after the slash in content-range. Above zero is a finding. Zero means the table exists and nothing came back. A missing or unreadable header means we learned nothing, and it is never a finding in either direction.

Later in August we ran it over 30,998 live apps. Of the 8,435 that named a Supabase project in the page, the check could finish on 3,680, and the figure we published is over those alone: Can anyone read your Supabase database? We checked 3,680 apps.

2. The bucket check never sent a body

Storage lists a bucket through POST /storage/v1/object/list/<bucket>. The options type of our HTTP helper had no body field, so the POST went out empty, and Supabase answers an empty POST with 400 "Body cannot be empty". It did that for every bucket name on every project, for the whole life of the check, and since a 400 is not a listable bucket, every time the check got as far as asking, it printed "Your uploaded files are kept private".

The unit test passed the entire time. Its fixture answered 200 whatever the body was, because it encoded what we believed about the endpoint.

Adding the body was only half of it. A bucket that does not exist also answers 200 [], the same as a private one, so with the body added and the old rule kept (a 200 with any array counted), the check would have flagged all seven names on its list on every project. Listable now means a 200 with a non-empty array, and the body asks for one entry at most:

{"prefix":"","limit":1}

Before committing we broke each fix on purpose to see whether the tests would notice. Dropping the request body fails exactly the four bucket-listing tests, and counting any 200 as exposed fails exactly the three count-verdict tests.

3. The page never loaded

This one sat under the other two. The scanner loads the page in headless Chromium and collects the JavaScript it really runs, and the Supabase URL and key come out of that bundle. When the load failed, the scan runner caught the error and handed every check an empty bundle without telling anyone. The secrets check then printed "No exposed API keys found" and the source-map check printed "Your original source code isn't published", about code nobody had downloaded. With no bundle there was no Supabase URL either, and the two Supabase checks took that to mean the app had no Supabase. Both stored scans behind the report had the secrets check down as clean, which is why nobody suspected the rest.

Treating every empty bundle as a failure would have been wrong too, because a page that ships no JavaScript is a real result. The runner now sets collectFailed when the load fails, and the four checks that read the bundle (secrets, source maps, RLS and buckets) call assertCollected() first. It throws, and a check that throws is printed as "Couldn't check".

The rule since

The rule we wrote down afterwards: an empty result means "we looked and found nothing", and nothing else may print as one. Every fixture involved had encoded a version of Supabase two API generations old, so when a vendor's response shape is in question we now check it against the live service before writing the code that reads it. Re-run after the fix, both apps got a real answer on both rows.

Two days later the same mistake turned up in our own statistics. The percentages we published divided every count by the apps whose scan returned, which counted every unanswered check as clean. Over the apps each check actually answered on, "95% of apps had at least one problem" became 98%.

I build Reeve, and these are the Supabase checks in its free scanner, which runs them from outside against a live URL: Reeve's security scanner.

Five things restore-testing Supabase dumps turned up

Vlad Tkachenko — Thu, 15 Oct 2026 07:36:00 +0000

We spent the first week of October restore-testing backups made with the Supabase CLI: the three files from Supabase's backup and restore guide, replayed with the psql command from the same guide into a fresh Supabase database. The goal was a GitHub Action that does that restore on every run, so we needed to know every way it fails.

Some of it fails loudly, which is fine. The parts worth writing down are the ones that finish without an error and leave you with less than you think.

The setup

These are the commands from Supabase's guide, unchanged:

supabase db dump --db-url "$DB_URL" -f roles.sql --role-only
supabase db dump --db-url "$DB_URL" -f schema.sql
supabase db dump --db-url "$DB_URL" -f data.sql --use-copy --data-only \
  -x "storage.buckets_vectors" -x "storage.vector_indexes"

psql --single-transaction --variable ON_ERROR_STOP=1 \
  --file roles.sql --file schema.sql \
  --command 'SET session_replication_role = replica' \
  --file data.sql --dbname "$TARGET_URL"

The target was a local Supabase from supabase start, on the same Postgres major version as the source. A plain Postgres container is not enough: the dump refers to the auth schema and to roles like authenticated that only a Supabase database has.

1. A cut-off dump can restore with exit code 0

This is the one that matters most, and it has nothing to do with Supabase.

On 27 September we dumped a database of two tables, 5,000 rows and 300 rows, cut the data file at three points and replayed each with --single-transaction and ON_ERROR_STOP=1, which are there to stop at the first problem and roll everything back.

Cut in the middle of a value: psql stopped with an error and wrote nothing.
Cut inside the last column of a row: exit code 0. The first table came back with 2,596 of its 5,000 rows and the second came back empty.
Cut exactly between the two tables: exit code 0, first table complete, second empty.

Each table's rows sit in one COPY block that ends with a line holding only \.. When the file runs out first, psql takes the end of the file as the end of the block and carries on.

The cheap check is the line pg_dump writes at the very end of a complete file:

grep -c 'PostgreSQL database dump complete' data.sql   # 1 is good, 0 means truncated

Only data.sql keeps it. The CLI strips comments out of the schema file (CLI 2.111), so search the data file.

2. Your grep for users is probably wrong

The second check most people write is whether the accounts made it into the dump. Our first version searched for COPY auth.users, and it would have failed on every real dump, because the CLI quotes every identifier:

grep -c 'COPY "auth"."users"' data.sql

Worth getting right, because supabase db dump without --data-only writes no rows at all and leaves auth out entirely. It restores cleanly. You find out at the login page.

3. pg_dump 17 against a Postgres 15 project

With no supabase/config.toml next to it (the usual case in a repository that only holds a backup workflow), the CLI dumps with pg_dump 17 whatever your project runs. The backup succeeds. The data file then carries this near its top:

SET transaction_timeout = 0;

Postgres 15 has no such setting, and with ON_ERROR_STOP=1 the replay stops on that line in any Postgres 15 database, including the project the file came from. We reproduced it on 4 October with pg_dump 17.11 and Postgres 15.19.

If your project is on 15, one line in the job's environment makes the CLI use the matching client:

env:
  SUPABASE_DB_MAJOR_VERSION: '15'

4. roles.sql stops twice on Postgres 17

Replaying the files of two Postgres 17 databases, one of them a hosted Supabase project, roles.sql stopped on two lines that Supabase's guide does not mention:

ERROR: "supabase_admin" is a reserved role, only superusers can modify it
ERROR: permission denied for parameter log_min_messages

The first is on a line starting ALTER ROLE "supabase_admin", the second on one starting GRANT SET ON PARAMETER. Both configure roles Supabase creates in every project for itself, so commenting them out is safe and the replay goes through. The CLI already comments out most lines about its reserved roles; these get past it.

5. pg_cron jobs do not come back

If your project schedules work with pg_cron, the restore brings back the extension and none of the jobs. The jobs are rows in cron.job, and the CLI's data dump leaves those rows out. We confirmed it in CI on 4 October: the source had a scheduled job, and the restored database had 36 tables and no cron rows.

A restored project therefore runs nothing on a schedule until you notice. Keep your cron.schedule calls in a migration or a script you can run again.

What a restore still does not prove

Counting rows tells you the data came back. Three things it cannot tell you:

That anyone can sign in. The rows in auth.users include the password hashes, but only a real sign-in against the restored copy tests them.
Your uploaded files. Storage keeps the bytes outside Postgres. The dump has the storage.objects rows and none of the files, so a restore shows every bucket full and every download broken.
Encrypted columns. Vault secrets and pgsodium columns come back as ciphertext, and the root key is never in a dump.

Doing it on every run

All of the above is what our open-source Action does after each backup. It reads the server's Postgres version, takes the three dumps, comments out the two roles.sql lines, restores into supabase db start on the runner, and compares the row count of every table with the rows the file holds. The run goes green only when every table came back:

- uses: Reeve-page/supabase-backup-action@v1
  with:
    db-url: ${{ secrets.SUPABASE_DB_URL }}

It is MIT licensed: https://github.com/Reeve-page/supabase-backup-action. Use the Session pooler connection string in the secret, since GitHub's runners reach IPv4 only and the direct host is IPv6 unless you pay for the add-on. Every run also downloads the whole database, and Supabase counts that against your egress allowance.

The full drill, with the row counts to compare and the sign-in test, is here: https://reeve.page/blog/test-your-supabase-backup

I build Reeve, and if you would rather not run this yourself, its managed Supabase backups keep the copies outside your Supabase account and restore them with one button: https://reeve.page/supabase-backups