I scanned 62 live AI-built apps from Show HN. Here's what they miss before launch

Mansoor Mirza — Tue, 20 Oct 2026 04:25:11 +0000

I build ShipSafe, a launch-readiness scanner for apps made with Lovable, Bolt, v0, Replit and friends. Before I told anyone it was useful, I wanted real data, so I pointed it at other people's launches.

Method (and what I did not do). I pulled every Show HN post that linked to a *.lovable.app, *.replit.app or *.bolt.host URL (101 apps) and ran each through the same passive checks the free scan uses: one page load, response headers, robots.txt, sitemap.xml, the HTML and the public JS bundle. No logins, no form submissions, no probing endpoints, no exploitation. I'm not naming any app; this is aggregate only.

The headline numbers

101 apps posted to Show HN; 100 answered.
38 of 100 were already dead (HTTP 404 on the home page). Replit was hit hardest: 32 of 44 Replit links were gone, vs 6 of 55 Lovable links. Free hosting that sleeps or expires is a real launch risk.
The 62 live apps averaged 73.5/100 (median 76, range 45–92). Only 1 scored 90+; 14 scored under 70.
8.5 findings per app on average.
Leaked secrets: 0. I specifically looked for API keys and tokens in the HTML and JS bundles (OpenAI, Stripe secret keys, AWS, service-role keys, etc.) and found none. Good news, and honestly not what I expected. The problems were elsewhere.

Finding	Live apps affected
Missing security headers (CSP, HSTS, X-Frame-Options…)	62 / 62 (100%)
No error monitoring	59 (95%)
No canonical URL	57 (92%)
No sitemap.xml	54 (87%)
No analytics	54 (87%)
No privacy policy / terms / contact link	35 (56%)
No favicon	33 (53%)
Supabase in use: Row Level Security must be verified	32 (52%)
No robots.txt	16 (26%)
Incomplete Open Graph tags	14 (23%)
Server reveals its tech stack	11 (18%)
Oversized JS bundle	9 (15%)

A fairness note: on *.lovable.app and *.bolt.host the platform controls the response headers, so 47 of the 62 header findings are not the builder's fault. They go away (or become yours to fix) once you move to a custom domain. That's exactly when people forget them.

The 5 problems worth fixing before you launch

1. Supabase without verified Row Level Security (52%)

Half the live apps ship a Supabase client in the browser. That's normal: the anon key is meant to be public. But it means your RLS policies are the only thing between a stranger and your tables. A passive scan can't (and shouldn't) test them, so treat this as a checklist item, not a verdict. Tables created by the AI without policies, or with using (true), are the classic vibe-coded data leak.

Fix: In Supabase → Table Editor, every table should show "RLS enabled", and every policy should scope rows to auth.uid().

AI prompt: "List every Supabase table this app uses. For each, enable Row Level Security and write policies so users can only select, insert, update and delete rows where user_id = auth.uid(). Show me the SQL, and flag any table that is intentionally public. Never use the service-role key in client code."

Then test it: open your app in a private window, logged out, and try to read data from the browser console with the anon key. You should get nothing back.

2. Missing security headers (100%)

No Content-Security-Policy, Strict-Transport-Security, X-Frame-Options/frame-ancestors, X-Content-Type-Options or Referrer-Policy. Without them your app can be framed for clickjacking and any injected script runs freely.

Fix: On a custom domain behind Vercel/Netlify/Cloudflare, add them in config.

AI prompt: "Add these response headers to every route: Strict-Transport-Security: max-age=63072000; includeSubDomains; preload, X-Content-Type-Options: nosniff, Referrer-Policy: strict-origin-when-cross-origin, X-Frame-Options: DENY, Permissions-Policy: camera=(), microphone=(), geolocation=(), and a Content-Security-Policy that only allows the domains this app actually loads scripts, styles, images and API calls from. Put them in vercel.json (or netlify.toml / _headers) and list every third-party domain you included in the CSP."

Start CSP in Content-Security-Policy-Report-Only mode if you're nervous about breaking things.

3. No privacy policy, terms or contact (56%)

More than half of live apps collect emails or logins but have no privacy policy, no terms and no way to reach a human. That's a GDPR problem the day an EU user signs up, it's a blocker for Stripe/Gumroad review and app stores, and visitors notice.

AI prompt: "Create /privacy and /terms pages for this app. It collects: [email, name, usage data]. It uses: [Supabase, Stripe, OpenAI, analytics tool]. Write plain-English pages that say what's collected, why, which processors receive it, how long it's kept, and how to request deletion at [your email]. Link both pages and a contact email in the footer of every page."

Have a human read it. It's a starting point, not legal advice.

4. Flying blind: no error monitoring (95%) and no analytics (87%)

Almost nobody had Sentry (or similar) wired up, so when the launch traffic hits and something breaks, the founder finds out from an angry comment. And 87% had no analytics, so they can't tell whether Show HN sent 20 visitors or 2,000.

AI prompt: "Add Sentry error monitoring to this app for both the frontend and any server/edge functions, with source maps uploaded on build and the DSN read from an environment variable. Also add [Plausible / PostHog / Vercel Analytics] page-view tracking. Don't log personal data in error reports."

5. Invisible to search and social: no sitemap (87%), canonical (92%), robots.txt, OG tags

Single-page apps from AI builders often ship one index.html with no sitemap.xml, no canonical link and half-filled Open Graph tags, so links shared on X, Slack or LinkedIn show a blank card. Two apps still had the default title "Lovable App".

AI prompt: "Make this app shareable and indexable: set a unique
and meta description per page, add a canonical link tag, full Open Graph and Twitter card tags with a 1200×630 image, a favicon, a robots.txt that allows crawling and points to the sitemap, and a sitemap.xml listing every public route."</p> </blockquote> <h2> <a name="bonus-keep-your-app-alive" href="#bonus-keep-your-app-alive" class="anchor"> </a> Bonus: keep your app alive </h2> <p>38% of the apps in this study were dead weeks or months after their Show HN. If you launch on a free tier, put it on a custom domain, check what happens when the free plan sleeps or expires, and add a free uptime monitor.</p> <h2> <a name="what-surprised-me" href="#what-surprised-me" class="anchor"> </a> What surprised me </h2> <p>I went in expecting leaked API keys everywhere. I found none. The AI builders have gotten much better at keeping secrets out of the frontend. What they <em>don't</em> do is the boring launch checklist: headers, legal pages, monitoring, SEO basics and, most importantly, checking the database permissions behind that public Supabase key.</p> <h2> <a name="check-your-own-app" href="#check-your-own-app" class="anchor"> </a> Check your own app </h2> <p>If you want to run the same checks on your app before you post it, the scan at <a href="https://shipitsafe.dev">shipitsafe.dev</a> is free and takes about 30 seconds. It gives you a score and the list of issues. There's an optional $9 report with step-by-step fixes and AI prompts for each one, but the free scan alone covers everything in this post.</p> <p>Happy (safe) shipping. Questions about the method are welcome in the comments.</p>

DEV Community: Mansoor Mirza