A 15-minute security pass for code your AI wrote

ShipSafeScan — Sun, 30 Aug 2026 11:37:04 +0000

When you're vibe-coding a side project, "working" and "safe to ship" feel like the same thing. They aren't. LLM-generated code is really good at making the happy path run, and really good at quietly skipping the boring parts that keep you out of trouble: auth checks, secret handling, dependency hygiene.

I found this out going back through my own repos. I'd let an assistant scaffold a few apps, shipped them, and then went back to audit what I'd actually pushed. It wasn't pretty. Nothing exotic, just the same handful of mistakes over and over. So this is a field guide to those mistakes, with the exact commands I now run before I call something done. No tools required to follow along; everything here is grep, npm, and reading your own diff.

A quick caveat on how to read this: these are patterns I keep seeing in code that LLMs generate, described qualitatively from my own repos. I'm not putting a percentage on it. I don't have a representative sample, and neither does anyone who tells you "X% of AI code is insecure" without a citation. Treat it as a checklist, not a statistic.

1. Secrets that made it into git history

The classic. You paste an API key into .env to test something, later add .env to .gitignore, and feel safe. But if the file was ever committed before you ignored it, the key is still sitting in your history forever.

Check the working tree first:

# Obvious offenders in files you can see right now
grep -rEn "(api[_-]?key|secret|token|password|BEGIN.*PRIVATE KEY)" \
  --include="*.{js,ts,jsx,tsx,py,env,json,yml,yaml}" . \
  | grep -v node_modules

Then check history, because that's where keys hide:

# Was .env ever committed at any point?
git log --all --full-history -- "*.env" ".env"

# Look for likely key shapes across all commits (example: AWS-style, generic hex)
git rev-list --all | xargs git grep -nE "AKIA[0-9A-Z]{16}|[a-f0-9]{32,}" 2>/dev/null

If you find anything: rotate the key first (assume it's burned), then worry about scrubbing history. Rotating is the part that actually protects you. History rewrites (git filter-repo) are secondary and can wait.

2. Secrets shipped to the browser

Frontend frameworks expose any env var with a public prefix to the client bundle. In Next.js that's NEXT_PUBLIC_; other frameworks have their own (VITE_, REACT_APP_, PUBLIC_). An assistant wiring up a fetch call will happily reach for whatever variable is in scope, including a server key it should never expose.

// This ships your key to every visitor's browser. Anyone can View Source.
const res = await fetch("https://api.example.com/data", {
  headers: { Authorization: `Bearer ${process.env.NEXT_PUBLIC_SERVICE_KEY}` },
});

The rule of thumb: a PUBLIC prefix means "I am fine with the whole world reading this." Publishable/anon keys are designed for that. Service-role keys, private API keys, and anything that can write or read other users' data are not.

Grep your client-side code for the prefix and eyeball every hit:

grep -rn "NEXT_PUBLIC_\|VITE_\|REACT_APP_" src/ | grep -iE "secret|service|admin|private"

Anything sensitive belongs behind a server route (an API route / server action), where the browser never sees the raw value.

3. API and admin routes with no auth guard

This is the one that bites hardest. The UI hides the "Delete everything" button behind a login screen, so it feels protected. But the button just calls /api/admin/reset, and that endpoint is a public URL. If the route handler doesn't check who's calling, anyone with the URL can call it directly with curl.

// app/api/admin/users/route.ts, looks fine, ships data to anyone
export async function GET() {
  const users = await db.user.findMany();     // no auth check anywhere
  return Response.json(users);
}

The fix is to check the session at the top of the handler, not just in the UI:

export async function GET(req: Request) {
  const session = await getSession(req);
  if (!session) return new Response("Unauthorized", { status: 401 });
  if (session.role !== "admin") return new Response("Forbidden", { status: 403 });

  const users = await db.user.findMany();
  return Response.json(users);
}

To find the gaps, list your routes and ask one question per file: "If a stranger hit this URL with no cookie, what happens?"

# Every API route in a Next.js app-router project
find . -path ./node_modules -prune -o -path "*/api/*" -name "route.*" -print

Then grep for the routes that don't mention a session/auth helper, those are your suspects:

grep -rL "getSession\|auth(\|requireUser\|verifyToken" $(find . -path "*/api/*" -name "route.*")

4. Dependencies with known vulnerabilities

LLMs are trained on a snapshot of the world, so they tend to reach for versions and packages that were popular at training time, sometimes pinned to something with a known CVE. You don't have to guess; the tooling already knows.

# Ships with npm, no install needed
npm audit

# Just the serious stuff, and a quick automated pass at safe upgrades
npm audit --audit-level=high
npm audit fix

If you want an ecosystem-agnostic view (or you're on pnpm/yarn and want a second opinion), OSV-Scanner reads your lockfile against the open OSV vulnerability database:

# https://github.com/google/osv-scanner
osv-scanner --lockfile=package-lock.json

Two habits worth building: don't blind-run npm audit fix --force (it can yank in breaking major versions), and re-run the audit on a schedule, because new CVEs get published against dependencies you already have.

5. CORS, rate limits, and errors that overshare

The quiet trio. None of these throws an error in dev, so they sail straight to prod.

CORS set to a wildcard lets any website call your API from a user's browser:

// Reflects "allow everyone", fine for a truly public read-only API,
// a problem the moment the endpoint does anything user-specific.
res.setHeader("Access-Control-Allow-Origin", "*");

Scope it to the origins you actually serve instead of *.

No rate limiting means a single script can hammer your login route, your signup, or your LLM-backed endpoint (that last one can also run up a real bill). Even a minimal per-IP limit in front of sensitive routes changes the economics for an attacker.

Errors that leak internals hand attackers a map. Returning the raw exception is convenient in dev and a gift in prod:

// Don't send this to the client in production
catch (err) {
  return Response.json({ error: err.stack }, { status: 500 });
}

Log the full error server-side; return a generic message and a request ID to the client. Grep for the tell:

grep -rn "err.stack\|error.message\|console.log(err" src/

The actual 5-minute / 15-minute pass

Here's the routine distilled. You can literally paste these in order.

5-minute triage (do this every time before you push):

# 1. Secrets in the working tree
grep -rEn "(api[_-]?key|secret|token|password)" --include="*.{js,ts,py,env}" . | grep -v node_modules
# 2. Was .env ever committed?
git log --all --oneline -- "*.env" ".env"
# 3. Public-prefixed secrets leaking to the client
grep -rn "NEXT_PUBLIC_\|VITE_\|REACT_APP_" src/ | grep -iE "secret|service|admin"
# 4. Known-vulnerable deps
npm audit --audit-level=high

15-minute pass (do this before anything goes to real users):

Walk every file under **/api/** and answer "stranger, no cookie, what happens?" for each one. Add the 401/403 guard where the answer is "they get data."
Confirm no Access-Control-Allow-Origin: * on endpoints that return user-specific data.
Put a basic rate limit in front of auth and any LLM/paid endpoint.
Grep for err.stack / raw error returns and swap them for generic messages + server-side logging.
Rotate any key you found in steps 1-3. Assume anything committed is compromised.

That's the whole thing. It's not sophisticated, and that's the point, the failures in AI-generated code are usually boring omissions, so a boring checklist catches most of them.

I ended up running this so often across my own projects that I got tired of doing it by hand, so I wrapped the repetitive parts (the secret grep, the dependency check, the "is this route guarded" pass) into a small scanner you point at a public GitHub repo: ShipSafeScan. It won't catch everything a careful read will, nothing does, but it's a fast first pass. Either way, the checklist above works on its own. Run it before you ship.