I've mass-reviewed code from teams using Cursor, Copilot, and ChatGPT. After hundreds of PRs, I started noticing patterns — specific code smells that humans
rarely produce, but AI generates constantly.
Here are the top 5, with real examples and how to catch them automatically.
- The Hallucinated Import
AI models confidently import packages that don't exist. Not typos — completely fabricated module names that sound plausible.
import { validateSchema } from 'express-validator-utils';
// This package does not exist on npm
from fastapi.middleware.logging import RequestLogger
# This module doesn't exist in FastAPI
Why does this happen? LLMs predict the most likely next token. A package named express-validator-utils is statistically plausible. The model doesn't check
npm. It just guesses.
How common: I've seen this in ~15% of AI-heavy PRs. It always passes code review because the name looks right.
- The Copy-Paste Clone
Ask AI to "add a similar endpoint" and it will duplicate 30 lines with one field changed. Humans refactor. AI copies.
app.get('/users', async (req, res) => {
const page = parseInt(req.query.page) || 1;
const limit = parseInt(req.query.limit) || 20;
const offset = (page - 1) * limit;
const results = await db.query(
'SELECT * FROM users LIMIT ? OFFSET ?', [limit, offset]
);
const total = await db.query('SELECT COUNT(*) as count FROM users');
res.json({ data: results, page, limit, total: total[0].count });
});
// Spot the difference...
app.get('/products', async (req, res) => {
const page = parseInt(req.query.page) || 1;
const limit = parseInt(req.query.limit) || 20;
const offset = (page - 1) * limit;
const results = await db.query(
'SELECT * FROM products LIMIT ? OFFSET ?', [limit, offset]
);
const total = await db.query('SELECT COUNT(*) as count FROM products');
res.json({ data: results, page, limit, total: total[0].count });
});
Only the table name changed. A human would extract a paginate() helper. AI doesn't feel the pain of duplication — it has infinite patience for boilerplate.
- The Empty Catch Block
AI's favorite anti-pattern. Every try/catch gets an empty catch:
try {
const data = await fetchUser(id);
return data;
} catch (e) {
}
try:
config = json.loads(raw)
except Exception:
pass
Errors vanish. Bugs become invisible. Your app fails silently at 3 AM and the logs show nothing.
AI does this because in training data, error handling is often abbreviated. The model learns that catch blocks are "boilerplate to fill in" and fills them
with nothing.
- The Stale API
AI training data has a cutoff. Models suggest APIs that were deprecated years ago:
const buf = new Buffer('hello'); // deprecated since Node 6
const parsed = url.parse(req.url); // deprecated since Node 11
app.use(bodyParser.json()); // built into Express since 4.16
from distutils.core import setup # removed in Python 3.12
loop = asyncio.get_event_loop() # deprecated in Python 3.10
These still work (usually), so they pass tests. But they're ticking time bombs.
- The Overengineered Singleton
Ask AI for a simple utility and you get an enterprise-grade AbstractSingletonProxyFactoryBean:
class DatabaseConnection {
private static instance: DatabaseConnection;
private constructor() {}
static getInstance(): DatabaseConnection {
if (!DatabaseConnection.instance) {
DatabaseConnection.instance = new DatabaseConnection();
}
return DatabaseConnection.instance;
}
}
In a file that's imported once. By one function. AI loves design patterns — it's seen thousands of them in training data. It applies them whether they're
needed or not.
How to Catch These Automatically
I built vibe-check — an open-source CLI that detects these patterns. No AI, no API keys, no cloud. Just pattern matching tuned for AI-generated code.
npx @bzprchny/vibe-check .
Output:
✖ error Hallucinated import: "express-validator-utils" not found
src/middleware.ts:3
⚠ warn Stale API: new Buffer() → use Buffer.from()
src/utils.ts:14
⚠ warn Copy-paste clone detected (18 similar lines)
src/routes/users.ts:10 ↔ src/routes/products.ts:10
● info Empty catch block swallows errors silently
src/db.ts:42
Vibe Score: 61/100 — Needs Work
4 issues · 127 files · 340ms
It also has --fix to auto-repair simple issues (Buffer.from, console.log removal, empty catch fills).
GitHub Action
Add it to your CI in 30 seconds:
- uses: BZPRCHNY/vibe-check@v0.1.0 with: preset: strict fail-on: error
Every PR gets a Vibe Score in the job summary.
What it checks (22 rules)
┌─────────────────┬─────────────────────────────────────────────────┐
│ Category │ Rules │
├─────────────────┼─────────────────────────────────────────────────┤
│ Hallucinations │ Fake imports, non-existent packages │
├─────────────────┼─────────────────────────────────────────────────┤
│ Copypasta │ Cross-file clone detection │
├─────────────────┼─────────────────────────────────────────────────┤
│ Dead code │ Console.log leftovers, empty catches │
├─────────────────┼─────────────────────────────────────────────────┤
│ Stale APIs │ Deprecated Node/Python/Express/React APIs │
├─────────────────┼─────────────────────────────────────────────────┤
│ Overengineering │ Unnecessary singletons, abstractions, factories │
├─────────────────┼─────────────────────────────────────────────────┤
│ Security │ Hardcoded secrets, missing env validation │
├─────────────────┼─────────────────────────────────────────────────┤
│ Dependencies │ Missing packages, phantom dependencies │
└─────────────────┴─────────────────────────────────────────────────┘
Configuration
Drop a .vibecheckrc in your repo:
{
"preset": "strict",
"disable": ["console-leftover"],
"exclude": ["legacy/**"]
}
The Bigger Picture
AI coding tools are incredible. I use them daily. But they have blind spots:
- No runtime context — AI doesn't know your Node version or installed packages
- No project memory — it can't feel the pain of duplicated code across files
- Training data lag — it suggests what was popular, not what's current
- Pattern addiction — it applies design patterns reflexively, not intentionally
Linting for these patterns is the equivalent of spell-check for AI code. You wouldn't ship a document without spell-check. Don't ship AI code without a vibe
check.
GitHub: https://github.com/BZPRCHNY/vibe-check
npm: npm i -g @bzprchny/vibe-check
Try it now: npx @bzprchny/vibe-check .
Top comments (0)