I built the same security auditor 5 times this week — once each for Supabase, PocketBase, Appwrite, Hasura/Nhost, and Firebase. Here is what I learned.

Five days ago I shipped a Supabase security auditor. Today I shipped the fifth in the family — Firebase. Same pattern, five different backends. Here's the timeline, the patterns I keep seeing, and what's actually different about each one.

The rough timeline

• May 5 — Supabase auditor (the original). Detects RLS-disabled tables, public buckets, exposed SECURITY DEFINER functions.
• May 9 morning — PocketBase. Detects empty API rules, the @request.auth.id != "" trap, true literals.
• May 9 mid-morning — Appwrite. Detects any and users role grants, document security misconfig.
• May 9 late morning — Hasura/Nhost. Detects anonymous role with open SELECT, user role missing row filter, public introspection.
• May 9 afternoon — Firebase. Detects the infamous match /{document=**} { if true; }, expired test-mode rules, auth-without-ownership.

Each one is its own repo + npm package + MCP server + Apify actor. Pure Node.js, zero deps, MIT.

The single thing that ports across all five

Every BaaS has the SAME failure mode at the app layer: a security model that's powerful, declarative, and easy to leave too open. The exact mechanism differs — RLS policies, API rules, role permissions, GraphQL filters, Firestore rules — but the misuse pattern is identical:

"I'll lock it down later" → never does → ships → some part of the data is accessible to anyone who knows the public URL.

The auditor doesn't need to be smart. It just needs to enumerate the security primitives the BaaS provides and match them against three or four well-known anti-patterns:

1. The "open by default" trap — empty rule, if true, any role, missing RLS. (Most common.)
2. The "any logged-in user" trap — @request.auth.id != "" (PocketBase), if request.auth != null (Firebase), anonymous role grants (Hasura). All let any signed-up user, including anonymously-authed ones, touch every row.
3. The "all columns" trap — exposing sensitive columns the role doesn't need (SELECT * permissions, no allowlist).
4. The "introspection" trap — anonymous schema reads in GraphQL APIs, public document listing in Firebase REST.

If you've ever shipped on one of these, you've probably written one of these patterns and forgotten about it.

The thing that actually convinces people: active probe

Static analysis tells you "this looks exposed." Active probe fetches the data anonymously and shows you what came back. After detecting the metadata pattern, each auditor sends an unauthenticated request:

• Supabase — GET /rest/v1/<table>?select=*&limit=1 with the anon key (which you can scrape from any deployed app's JS bundle in 30 seconds).
• PocketBase — GET /api/collections/<name>/records?perPage=1 anonymously.
• Appwrite — GET /v1/databases/{db}/collections/{col}/documents?queries[]=limit(1) anonymously.
• Hasura — POST /v1/graphql with { <table>(limit:1) { __typename } } and no auth header.
• Firebase — GET /v1/projects/<pid>/databases/(default)/documents anonymously.

If documents come back, the finding is marked confirmed: true with the row count, columns, and bytes returned. This closes the "is this real or theoretical?" question every dev asks when reading a security report.

The first time I ran this on my own Supabase project, 17 of my own tables were leaking. That was the impulse behind the whole thing.

What's different per BaaS

Building this five times in a row taught me each one's quirks:

Supabase — the security model is just Postgres RLS. The auditor uses the Supabase Management API to introspect projects, then probes via the public REST endpoint. Easiest to build because Postgres tooling is excellent.

PocketBase — admin auth required for the metadata fetch. Single binary backend, so the rule format is dead simple (string expressions). Active probe is the most satisfying because empty/permissive rules instantly leak the entire collection on the first request.

Appwrite — has a unique "Document Security" toggle that changes whether collection-level perms apply per-document or globally. Audit needs to detect this state because a single broad rule with Doc Security OFF exposes every doc.

Hasura/Nhost — the most complex permission model (per-role × per-table × per-action with row-level filters and column allowlists). Audit pulls metadata via POST /v1/metadata with the admin secret. Probe via anonymous GraphQL.

Firebase — totally different shape: rules live in a firestore.rules file, not a metadata API. So the auditor is a static analyzer (parse the file, detect anti-patterns line by line). Optional probe sends an unauthenticated GET against the Firestore REST endpoint when you provide a project ID.

The "build the same thing five times" payoff

Each new ecosystem took ~30 minutes from mkdir to "Apify actor build SUCCEEDED." Same template:

• scripts/audit.js — pull metadata, run checks, optional probe
• scripts/report.js — Tailwind + Chart.js HTML report (~25KB self-contained)
• src/server.js (MCP) — three tools: audit, list_findings, preview_fix
• push.py — idempotent Apify actor create + version push + build trigger

The work compounds because the SHAPE of every BaaS security audit is the same: pull config, match patterns, optionally probe. Only the bytes change.

What's open

Code MIT, all five live on github.com/Perufitlife (<name>-security-skill for the CLI, <name>-security-mcp for the MCP server). Each has an Apify actor too.

If you've seen Firebase/Supabase/PocketBase/Appwrite/Hasura security misconfigs in the wild that aren't in my anti-pattern list, I want to hear about them — I'll add them as new checks.

And if you don't want to install five things to scan five backends — I do paid audits at https://perufitlife.github.io/supabase-security-skill/ ($99, 24h delivery, written report). One landing covers all five.