Every week, I review code that AI assistants wrote. And every week, I find the same security holes.
This isn't theoretical. I've been building a security scanner specifically for AI-generated code, and after analyzing hundreds of code samples from ChatGPT, Claude, Copilot, and other AI tools, the patterns are disturbingly consistent.
Here's what I keep finding — and why traditional security tools miss most of it.
The Core Problem: AI Optimizes for "Works," Not "Safe"
AI code assistants are trained to produce functional code. When you ask for a login system, you get a login system. It works. It compiles. It passes basic tests.
But "works" and "secure" are different things.
I ran the same prompt — "build a user authentication system in Node.js" — through three different AI assistants. Every single one produced code with at least two critical vulnerabilities. The most common? Hardcoded secrets and missing input validation.
// What AI typically generates
const JWT_SECRET = "super-secret-key-123";
app.post("/login", (req, res) => {
const { username, password } = req.body;
// No input sanitization
const user = db.query(`SELECT * FROM users WHERE username = '${username}'`);
// SQL injection waiting to happen
});
This isn't a cherry-picked example. This is the median quality of AI-generated auth code.
5 Vulnerability Patterns AI Keeps Repeating
After scanning hundreds of AI-generated code samples, these are the top patterns by frequency:
1. Hardcoded Secrets (Found in ~70% of samples)
AI loves putting API keys, database passwords, and JWT secrets directly in source code. It doesn't know about .env files unless you specifically ask.
Impact: One git push and your credentials are public. GitHub's secret scanning catches some of these, but not application-level secrets like database connection strings or internal API keys.
2. Missing Input Validation (Found in ~65% of samples)
AI generates the happy path. User input goes straight into database queries, shell commands, or file operations without sanitization.
Impact: SQL injection, command injection, path traversal — the entire OWASP Top 10 shows up because AI skips validation.
3. Silent Error Handling (Found in ~50% of samples)
try {
await processPayment(order);
} catch (e) {
// handle error
}
That comment isn't handling anything. The payment fails silently. Logs show nothing. The user gets charged but the order never processes. I see this pattern constantly in AI-generated code.
Impact: Security failures go undetected. Attackers exploit unhandled edge cases.
4. Overprivileged Operations (Found in ~30% of samples)
AI doesn't think about the principle of least privilege. It generates code that runs with admin permissions, accesses all files, and opens unnecessary network connections.
Impact: If the application is compromised, the attacker inherits all those excessive permissions.
5. Outdated or Vulnerable Dependencies (Found in ~25% of samples)
AI's training data has a cutoff. It recommends packages that have known CVEs, deprecated APIs, or even packages that don't exist anymore (opening the door to typosquatting attacks).
Impact: You inherit vulnerabilities from dependencies you didn't choose — the AI chose them for you.
Why Traditional Security Tools Miss These
Here's what surprised me the most: Snyk, SonarQube, and semgrep catch less than half of these patterns.
Why? Because they're designed for human-written code. They look for known CVE patterns in dependencies, common coding mistakes, and configuration issues.
AI-generated code creates a different class of problems:
- Plausible but insecure patterns — The code looks correct. It follows conventions. But the security logic is subtly wrong.
- Cross-concern vulnerabilities — Input validation missing in one file, combined with shell execution in another. No single-file scanner catches the composite risk.
- AI-specific anti-patterns — Hardcoded secrets that look like example values, debug code left in production, TODO comments masking missing security features.
What Actually Works
After months of building and testing, here's what I've found effective:
1. Scan Before You Commit
Don't trust AI output. Treat every AI-generated code block like an untrusted pull request from a junior developer who's never heard of OWASP.
2. Use Deterministic Analysis, Not More AI
I initially tried using LLMs to analyze LLM output. The results were... inconsistent. Running the same scan 5 times gave 5 different severity ratings. That's not a security tool — that's a coin flip.
Static analysis with pattern matching gives you:
- 100% reproducible results — Same code, same findings, every time
- Zero API costs — No tokens burned on analysis
- Millisecond speed — Scan before every commit without friction
3. Check for AI-Specific Patterns
Standard security checklists miss AI-specific issues. You need to specifically check for:
- Hardcoded values that look like "example" data but are actually used in production
- TODO/FIXME comments that mask missing security features
- Empty catch blocks and silent error handling
- Unnecessary network calls and file system access
4. Make Scanning Frictionless
If security scanning takes more than 30 seconds, developers skip it. The scanner needs to be fast enough to run on every paste, every commit, every PR.
The Uncomfortable Truth
AI code assistants are incredibly productive tools. I use them daily. But they're optimizing for the wrong metric.
Speed of generation ≠ Quality of output.
Every hour saved by AI-generated code costs minutes of security review. And if you skip that review, you're building on a foundation of vulnerabilities that will cost you far more later.
The solution isn't to stop using AI. The solution is to verify everything it produces, automatically, before it reaches production.
Try It Yourself
I built CodeHeal to solve exactly this problem — a security scanner designed specifically for AI-generated code. 14 vulnerability categories, 93 detection rules, deterministic results, zero API costs.
Paste your AI-generated code and see what it finds. No signup required for your first scan.
Top comments (0)