DEV Community

hao li
hao li

Posted on

Your AI Shipped the App. Did It Leave the Keys in the Repo?

#opensource #python #showdev

Your AI Shipped the App. Did It Leave the Keys in the Repo?

Last month, security firm UpGuard told TechCrunch it had found around 16,000 Supabase-hosted databases exposing personal data — names, addresses, phone numbers, even passwords — queryable by anyone on the open web. Not hacked. Just... left open.

The mechanism behind almost all of it: AI coding tools write SQL migrations directly, and in that path Postgres row-level security defaults to off. Supabase's dashboard turns RLS on when you click through the Table Editor, but AI tools don't click through dashboards. A founder types a prompt, the AI ships a working app in an afternoon, and nobody ever flips the switch that keeps one user from reading every other user's rows.

That is the vibe-coding security gap in one sentence: the app works, so nobody checks the configuration.

What I built

vibe-shipcheck — a zero-dependency Python CLI that runs the pre-publish security checklist vibe-coded apps never get:

pip install vibe-shipcheck
vibe-shipcheck ./my-app
Enter fullscreen mode Exit fullscreen mode 
vibe-shipcheck 0.1.0 — pre-publish security check
Scanning: /home/hao/my-app

ERRORS:
  [env-committed] .env — .env file is tracked by git — its secrets ship with the repo
    fix: Add it to .gitignore, run `git rm --cached .env`, rotate every key it held.
  [rls-disabled] supabase/migrations/001_init.sql — migration creates table(s) profiles without ENABLE ROW LEVEL SECURITY — readable by anyone with the anon key
    fix: Add `ALTER TABLE <table> ENABLE ROW LEVEL SECURITY;` plus policies for every table, or enable RLS in the dashboard.

WARNINGS:
  [supabase-url-exposed] src/lib/supabase.ts:3 — Supabase project URL in source
    fix: The URL itself is public, but anyone with it + your anon key can query every table whose RLS is off.
  [cors-wildcard] api/main.py:12 — wildcard CORS origin — any website can call this API
    fix: Restrict origins to your own domains before publishing.

2 error(s), 2 warning(s)
BLOCKED — fix the errors above before publishing.
Enter fullscreen mode Exit fullscreen mode

What it checks:

  • Committed .env files — tracked by git, untracked-but-unignored, or unverifiable. The classic git add . leak.
  • Hardcoded secrets — AWS, GitHub, Stripe, OpenAI-style keys, PEM blocks, plus generic password= / api_key= literals. Proper env references (os.environ[...], process.env...) and placeholders are never flagged.
  • Supabase exposure — project URLs in source, and it actually decodes JWTs to tell the public anon key from service_role (which bypasses all RLS and must never be in code).
  • Missing RLS — scans your migration SQL; any CREATE TABLE without ENABLE ROW LEVEL SECURITY is an error. This is the exact UpGuard failure mode.
  • Wildcard CORS and debug/dev flags left on.

Every finding ships with a one-sentence fix line — the audience is someone who prompted an app into existence, not someone who reads CVE feeds.

Exit code 1 on errors, so it drops straight into CI or a deploy script. --strict fails on warnings too, --json gives machine-readable output.

Why not just use Trivy / Gitleaks?

Those tools scan for vulnerabilities in code — CVEs, secret patterns in history. That's a different moment. vibe-shipcheck owns the pre-publish checklist: the ten seconds between "the AI says it's done" and "it's live on the internet." Nobody hand-writing an app forgets RLS on every table; AI tools do it by default. The failure mode is configuration, not code, and it needs a checker that thinks in terms of shipping, not CVEs.

Use both: Gitleaks in CI for deep secret history, vibe-shipcheck as the gate right before you hit publish.

Try it

pip install vibe-shipcheck and run it against your own vibe-coded projects — I'd genuinely like to hear what it finds. If it catches a real leak before you ship, tell me in the comments.

tags: python, security, supabase, vibecoding
canonical: https://github.com/hahahahahahahahah6/vibe-shipcheck

Top comments (0)