7 Vulnerability Patterns I Found in AI-Generated Code (and How to Catch Them)

jitendrarout — Fri, 11 Sep 2026 03:03:00 +0000

7 Vulnerability Patterns I Found in AI-Generated Code (and How to Catch Them)

If you've used GitHub Copilot, Claude Code, or any AI coding assistant for more than a few weeks, you've probably shipped at least one of the bugs in this post without realizing it. Not because the AI is bad at coding — these tools are remarkably good — but because certain classes of mistake show up disproportionately in AI-generated code, for reasons that have nothing to do with capability and everything to do with what a code sample optimizes for.

I wanted to find out whether that pattern was real or just a feeling, so I built ai-vuln-scan, a static analysis tool tuned specifically to these patterns, and used it to look closely at what actually goes wrong. Here's what I found, and the open-source tool that came out of it.

The core idea: AI-generated bugs aren't random

Traditional static analysis tools look for bugs in general. What I was after was narrower: which specific mistakes are more likely to show up in AI-assisted code than in code a human wrote from scratch?

The pattern I kept noticing was this: AI assistants are optimizing, in a sense, for "a plausible, runnable example" — and a plausible runnable example doesn't need a real secrets manager, doesn't need parameterized queries to demonstrate the concept, and doesn't need the hardened production config. So it generates the version that works, not necessarily the version that's safe, unless the prompt specifically asks for the safe version.

That's not a knock on the models. It's a predictable consequence of what "helpful code sample" optimizes for versus what "production-ready code" requires. Which means the fix isn't "better prompting" (though that helps) — it's catching the gap systematically, the same way we catch any other predictable class of bug.

Seven patterns worth knowing

I documented these as a public, versioned taxonomy — the AI Vulnerability Pattern Catalog — specifically so the reasoning behind each one is checkable and extendable by anyone else who's noticed the same thing. Here are the seven, briefly:

1. Hardcoded secrets in placeholder form. A plausible-looking example API key or connection string gets generated to make the sample runnable, and gets copy-pasted into real code without ever being swapped for an environment variable.

2. Unparameterized query construction. `SELECT * FROM users WHERE id = ${userId}` reads naturally as "the way you'd explain a query with a variable in it." Parameterization is the correct approach, but it's an extra, less narratively obvious step.

3. Shell commands built by string interpolation. Same root cause as #2 — exec("cmd " + arg) is the intuitive-looking version;
execFile() with an argument array is correct but less often what gets generated by default.

4. Permissive default configuration. Wildcard CORS, disabled TLS
verification, debug mode left on — these "just work" in a demo and
remove setup friction the assistant doesn't have the context to resolve (it doesn't know your real allowed origins or have your real cert).

5. Weak cryptographic primitives in a security context. MD5 and
Math.random() are often the first hashing/randomness functions that
come to mind for a generic "hash this" or "generate a random string"
request — the security-context distinction isn't always surfaced unless specifically prompted.

6. Inconsistent authorization across near-identical routes. This one is, I think, the most distinctly AI-flavored bug on the list. When you ask an assistant to "add another route like the others," it regenerates the pattern rather than copy-pasting the existing block — and regeneration is where a step like auth middleware can quietly drop out, especially across separate prompts or edits. A human copy-pasting an existing route is more likely to preserve the whole block by construction; an AI regenerating it from a description is not.

7. Verbose error responses leaking internals. Returning err.stack directly in an HTTP response is the fastest way to make error handling "work" and visible during development — the split between server-side logging and a generic client message is a production concern that's easy to omit from a first-pass generation.

Seeing it in practice

Here's a realistic example — an Express route handler, the kind you'd
get by asking an assistant for "an endpoint that looks up a user's
orders":

app.get('/api/users/:id/orders', (req, res) => {
  const userId = req.params.id;
  res.json({ userId });
});

Nothing looks wrong at a glance. But if this route sits in a file where every other route includes an authMiddleware call and this one doesn't, that's exactly pattern #6 — and it's the kind of thing that's easy to miss in review because each individual route reads fine in isolation.

Running ai-vuln-scan against a small sample file with a handful of
these patterns deliberately included caught all of them — 12 findings
across hardcoded secrets, unparameterized queries, a missing-auth route, and a weak-randomness token generator — with zero false positives on the same routes rewritten safely.

Why build a dedicated tool instead of using existing linters

Generic security linters (ESLint security plugins, Bandit, Semgrep) catch some of this - but generically, not with the AI-specific framing. Two things make a dedicated tool worth having:

The taxonomy itself is useful independent of the tool. Naming and documenting why each pattern shows up disproportionately in AI-generated code is a different, more specific claim than "this is a bug that can happen," and it's citable/extendable on its own.
Detection can be tuned to the actual distribution. Rule #6 (inconsistent auth across near-identical routes) isn't something a general-purpose linter is likely to check for at all — it requires knowing that this specific class of drift is common in AI-regenerated code specifically.

What's next

The tool currently covers JavaScript/TypeScript and Python with regex/structural detection — solid for the patterns above, but a real
AST-based taint-tracking engine would catch more, especially multi-line or aliased variants. That's the next milestone, along with expanding language coverage and publishing a labeled dataset of AI-generated vulnerable code samples for anyone else working on this problem.

If you've noticed other patterns that seem to show up disproportionately in AI-assisted code, I'd genuinely like to hear about them — the pattern catalog is open to contributions, and documenting a new pattern doesn't require having a detection rule for it yet.

Try it: git clone https://github.com/jitendrarout/ai-vuln-scan and run npm run scan:examples to see it catch all seven patterns against the bundled sample files.

Jitendra Rout is a Principal Software Engineer working on AI systems. This post is part of ongoing work on transparent, auditable AI tooling.

DEV Community: jitendrarout

7 Vulnerability Patterns I Found in AI-Generated Code (and How to Catch Them)

7 Vulnerability Patterns I Found in AI-Generated Code (and How to Catch Them)

The core idea: AI-generated bugs aren't random

Seven patterns worth knowing

Seeing it in practice

Why build a dedicated tool instead of using existing linters

What's next