Tomer goldstein

Posted on Apr 2 • Originally published at ship-safe.co

Your Vibe Coding Security Scanner Is Missing the Worst Bugs. Here's Why.

#security #ai #webdev #beginners

there are like 8+ security scanners for vibe-coded apps now. a year ago there were zero. we went from "just ship it and pray" to having actual tools. love to see it.

but here's the thing nobody's talking about: these tools don't all scan the same stuff. and the difference between them decides whether you catch the bugs that actually get you hacked or just the surface-level ones.

I built one of these scanners (ShipSafe) and I've been testing the others too. the split is simple: does it scan your deployed URL or your actual source code? that one question changes everything.

the two flavors

URL scanners hit your live site from the outside. they catch missing headers, exposed endpoints, leaked keys in your JS bundle, SSL issues. real stuff. useful.

repo scanners read your actual code. they look at auth logic, database queries, how secrets are handled on the server. different game entirely.

most people assume these overlap more than they do. they don't. like, at all.

three bugs URL scanners literally cannot find

these are from real AI-generated codebases. I keep running into them.

the backwards auth

export function middleware(req) {
  const token = req.cookies.get("session");
  if (token) {
    return NextResponse.redirect("/login");
  }
  return NextResponse.next();
}

missing !. logged-in users get kicked, anonymous users get in. found this in 31% of Cursor apps I scanned.

why a URL scanner is blind to it: you hit the page without auth, get a 200 back. scanner says "looks good." it has zero way of knowing the logic is flipped — that's a source code problem. the app looks normal from the outside. it's just completely open.

IDOR with no ownership check

export async function GET(req, { params }) {
  const invoice = await db.invoice.findUnique({
    where: { id: params.id },
  });
  return Response.json(invoice);
}

change the ID, get someone else's data. 43% of apps had this.

URL scanner hits it, gets a 200 with JSON, says "valid response." it can't tell the difference between your invoice and someone else's invoice — both look the same from the outside. you have to read the Prisma query to see there's no userId filter. that's in the source.

admin API with no backend check

// React hides the button. cool I guess.
if (user.role !== "admin") return null;

// API deletes users. no check. anyone can call this.
export async function DELETE(req, { params }) {
  await db.user.delete({ where: { id: params.id } });
  return Response.json({ success: true });
}

scanner visits the page, doesn't see admin buttons (they're conditionally rendered), reports clean. meanwhile curl -X DELETE /api/users/123 works for literally anyone. the endpoint isn't even linked in the DOM so the scanner never discovers it.

being real tho — repo scanners miss stuff too

gotta keep it honest:

runtime config - your code is fine but someone set the wrong env var on Vercel. repo scanner can't see that
infra issues — permissive CORS at the CDN level, open ports on your VPS. not in the code
database config — Supabase RLS policies live in Postgres, not your repo
deploy drift — you fixed it locally but forgot to push. the deployed version is still cooked

so yeah neither approach is complete on its own.

the actual move

run both. I'm not just saying that bc I built a repo scanner — I use URL tools on my own stuff too.

layer	tool	catches
source code	ShipSafe, ChakraView, Aikido	auth logic, IDOR, secrets in server code
live site	VibeCheck, amihackable.dev	exposed endpoints, headers, runtime config
dependencies	`npm audit`, Snyk	known CVEs in packages
database	Supabase dashboard	RLS policy issues

the thing is - the vulns that actually lead to data breaches in AI code are almost always logic bugs. backwards conditions, missing ownership checks, unprotected endpoints. that stuff lives in the source. it's invisible from the outside. a URL scanner literally cannot catch a missing ! in your middleware. it just can't.

quick gut check for your current setup: can your scanner find a missing ! in middleware? can it tell that a Prisma query doesn't filter by userId?

if the answer is no, you've got a blind spot. and it's the layer where the critical stuff lives.

ShipSafe scans your GitHub repo for auth bugs, IDOR, hardcoded secrets, and OWASP Top 10 vulns. free plan, no credit card.

DEV Community