AI app builders made it possible to ship a working product in a weekend. I wanted to know what that does to security, so we read the code of 1,029 public apps built with Lovable, Bolt, v0, Cursor, Claude Code and similar tools, and compared them with 188 similar apps written before ChatGPT.
The short version: 40% of the AI-built apps had at least one high or critical security problem. That number alone isn't the interesting part, though.
AI apps aren't worse overall. They're wrong in different places.
The apps from before ChatGPT actually had a slightly higher rate of serious problems (45%). But the kind of problem changed:
| Problem | AI-built (1,029) | Before ChatGPT (188) |
|---|---|---|
| Leaked keys or secrets | 5% | 13% |
| Key or password written in the code | 3% | 11% |
| Strangers can read or change data | 8% | 1% |
| Missing or broken login checks | 7% | 2% |
| Payment holes | 3% | 0% |
| Features anyone can use on your bill | 3% | 0% |
AI tools have mostly learned not to hardcode secrets. What they haven't learned is who is allowed to do what. Access control and payments are where AI-built apps go wrong far more often than hand-written ones.
Where it goes wrong
The browser talks straight to the database. Lovable, Bolt and v0 apps did worst: 62% had a high or critical problem, against 20% for full-stack Cursor and Claude Code projects with a server in between. When the browser talks to Supabase or Firebase directly, the database rules are the only lock on the door.
It's rarely "RLS off" any more. Only 1% of tables had row level security switched off; the platforms now turn it on by default. The holes moved:
- policies that let every signed-in user read every row
- database functions anyone can call
- Supabase Edge Functions that skip the login check (5% of all apps)
Payments are the riskiest group. Of the 160 apps that take payments, 50% had a serious problem, and 9% had a hole in the payment flow itself. The usual mistakes:
- the webhook that marks an order paid doesn't verify the provider's signature, so anyone can send a fake "paid" message
- the paid status, credits or plan are written from the browser, so a user can give themselves the paid plan
- secret payment keys sit in a variable that is shipped to the browser
AI features are open wallets. 441 apps call an AI model themselves. 6% had an AI or email feature anyone could use, on the owner's bill.
And the classic: the .env file. 15% of apps had their .env secrets file committed to the repository, usually because the project was pushed before a .gitignore existed.
How to check your own app (free)
You don't need any tool for the most important test:
- The two-account test. Create users A and B. As A, create a private record. As B, request A's record by its ID, then try to update and delete it. Then try it logged out. Test the actual API or Supabase request, not just whether the UI hides the button.
- Ask what a stranger can do with your public key. For every table and database function: what can someone with your anon key read and change? What can any signed-in user? Both answers should be "only their own rows" unless the data is meant to be public.
- Run Supabase's Security Advisor (Database → Advisors) and fix what it flags.
-
Search your frontend for secrets. Anything in a
VITE_orNEXT_PUBLIC_variable is public. Stripe secret keys, service-role keys and AI API keys must live only on the server. - Check your Stripe webhook verifies the signature before it marks anything paid.
Method and limits
These numbers come from Redproof's automated checks on public GitHub repositories (November 2026), so treat them as estimates. Static analysis reads code; it can't see rules you changed directly in the Supabase dashboard, and some findings need a person to confirm them. No repository is named. The full method, including the one noisy check we left out, is on the method page.
Disclosure: I built Redproof, the checker used for this study. It runs the same checks on your code and the launch score is free, but everything in the checklist above works without it.
Top comments (0)