What I found when I security-scanned 10 AI-built apps (and how to check yours manually)

pam mafeng — Sun, 21 Jun 2026 17:31:11 +0000

AI coding agents — Lovable, bolt.new, Cursor, Replit — have made it possible
for anyone to ship a working app without writing a line of code themselves.
That's genuinely great. It also means a huge and growing number of live apps
were never looked at by anyone who knows what a SQL injection or a missing auth
check looks like.

The numbers on this aren't subtle:

45–88% of AI-generated code samples contain at least one real, exploitable vulnerability, depending on the study.
380,000 publicly exposed assets have been found running on vibe-coding platforms with no security review — roughly 5,000 of them holding sensitive corporate data.
Moltbook, a real app launched in 2026, had 1.5M API tokens and 35,000 user emails exposed in under three minutes. Its founder wrote zero lines of code himself.
The Tea App left private user images publicly accessible — the kind of mistake a scanner catches instantly and a busy solo builder never sees.

I wanted to know what this actually looks like in practice, so I found 10 real,
public repos generated by Lovable and bolt.new and ran them through
Semgrep — free, open-source, no account needed — using
the same rulesets a commercial scanner runs as a baseline (p/secrets and
p/owasp-top-ten). Here's what came back, in plain language instead of CVE
numbers.

Finding 1: A complete set of production secrets, committed to git

One app — an AI-tooling SaaS — had a .env and a .env.server file both
committed directly to the repo. Together they contained an OpenAI API key, a
Supabase service-role key (which bypasses every Row-Level-Security policy
in the database, full read/write on every table), the Supabase JWT secret,
the production database password, a Clerk auth secret key, and a Vercel deploy
token.

The JWT secret is the one to understand: with it, anyone can mint their own
valid login token for any user, including an admin, and Supabase's API will
accept it as real. No password, no exploit — just the file sitting in the repo.

The fix, if this were your repo: rotate every one of those credentials
today (yes, all of them, not just the obvious one), scrub the files from git
history (deleting the latest commit isn't enough — the old commit still has
it), and add .env* to .gitignore before committing again.

Finding 2: Row-Level Security disabled mid-migration, no transaction

A financial group-savings app had a migration file that runs
ALTER TABLE public.profiles DISABLE ROW LEVEL SECURITY;, performs some
unrelated column additions, and re-enables RLS near the bottom of the same
file — without wrapping any of it in BEGIN; ... COMMIT;.

This script reads like it was meant to be pasted into Supabase's SQL editor a
block at a time, which is exactly how someone without a database background
fixes things. If anything errors out between the disable and the re-enable —
a typo, a duplicate constraint — the user profiles table (names, emails, phone
numbers) is left wide open with no policy protecting it, and nothing in the UI
tells you that happened.

The fix: wrap fix-up migrations in an explicit transaction so a mid-script
failure rolls everything back together, and re-check the RLS toggle in the
Supabase dashboard after running any migration that touches it.

Finding 3: No way to verify security from the repo at all

Across the Supabase-backed apps I scanned, two out of three had no
supabase/migrations folder and no SQL anywhere in the repo. They had a fully
working Supabase client wired into the frontend, and nothing in version control
showing whether the tables behind it have an RLS policy on them or not.

This is the one that matters most for how I think about this problem: a code
scanner — including the one I'm building — genuinely can't see this. The
schema and its policies were built by hand in a dashboard and never exported
anywhere reviewable. The risk isn't a bug in the code; it's a decision that was
never written down.

How to check your own repo right now

You don't need to wait for a tool. This takes about five minutes:

pip install semgrep
semgrep --config=p/secrets --config=p/owasp-top-ten /path/to/your/repo

That catches committed secrets and the most common injection/XSS patterns. For
the Supabase-specific gap (Finding 2 and 3), there's no substitute yet for
manually opening your Supabase dashboard, going to Table Editor, and checking
that every table handling real user data has Row-Level Security turned on with
policies that actually scope to auth.uid().

What's next