The worst AI code isn't the code that crashes. Code that crashes, you notice.
The worst one looks right, passes a quick glance, and the AI says "Done! ✅". Then it breaks in production two weeks later.
You're not alone in this. In Stack Overflow's 2025 developer survey, the most common complaint about AI tools was "AI solutions that are almost right, but not quite" (66% of developers). In the 2026 survey, nearly half said they trust AI output only when they can check it easily.
So here's my list of how "almost right" sneaks in, with a check for each that takes about 30 seconds. I use AI coding tools every day, and I keep running into every one of these.
1. "Fixed!" … but nothing was run
The AI edits the code, explains the fix nicely and stops. No test ran. Sometimes a test command did run, but it found 0 tests and still exited green.
30-second check: ask "Which command did you run, and what was the exit code?" Then look at the output for the number of tests that actually ran. 0 passed is not a pass.
2. The tests pass because the test changed
The fastest way to make a red test green is to change the test, not the code. Look out for:
-
it.skip(/xit(/@pytest.mark.skip -
expect(total).toBe(90)quietly becomingexpect(total).toBeDefined() - a snapshot "updated" to match the new (wrong) output
30-second check: git diff -- '*test*' '*spec*' before you accept anything. If a test file changed and you didn't ask for that, ask why.
3. A function or package that doesn't exist
It reads perfectly: import { parseSafe } from "zod/utils". That function is not there. Or it suggests npm install of a package with a very believable name. Attackers now register those made-up names on purpose, so that slip can turn into a real risk.
30-second check: open the real docs or the npm page. A brand-new package with a handful of downloads is a red flag.
4. Only the happy path works
The demo input works. An empty list, null, a timezone, a 0, a very long string or a double click doesn't.
30-second check: ask the AI, "Which 3 inputs would break this?" Then run those 3. It is surprisingly honest when you ask that directly.
5. Errors get swallowed
The code stops crashing because the crash is hidden:
try {
return await getOrders(userId);
} catch {
return []; // the user just sees "no orders"
}
Nothing fails, nothing logs, and the bug becomes a support ticket.
30-second check: search the diff for catch, || [], ?? null and except: pass. Each one should log, rethrow or have a reason.
6. The right answer for the wrong version
The code is correct for a version you don't use: the Next.js pages router in an app-router project, an old React Router API, a removed config option. It compiles in the AI's head, not in your project.
30-second check: compare the imports with the versions in your package.json (or lock file), and paste your version into the prompt next time.
7. Security that "works"
The login works, so it looks done. But the JWT is decoded instead of verified, the secret is hard-coded, or an API route never checks who is calling it.
30-second check: for every new route, ask "Who is allowed to call this, and where is that checked?" Search the diff for strings that look like keys.
The habit that catches most of these
Before accepting any AI change, run one command:
git diff --stat
Did it touch files you never mentioned? Did test files change? Is the diff 10× bigger than the task? That one look catches half of this list.
Your turn 👇
Which one has bitten you most recently? And what's the 8th way that's missing from this list? I'll add the best ones (with credit) to a follow-up post.
Small note: I got tired of doing checks 1 and 2 by hand, so I built a free Claude Code plugin, proof-of-done, that does them automatically, and a free security-audit skill for number 7.
Happy Diwali in advance 🪔: may your builds be green and your tests honest.
Everything I've built (all free and open source)
which-ai-to-use: picks the best AI and model for each task, and hands your work to another AI when you hit a usage limit
- Try it:
npx which-ai-to-use - npm · VS Code Marketplace · Open VSX (Cursor, Windsurf) · GitHub · Website
proof-of-done: a Claude Code plugin that won't let the AI say "done" until the tests pass
- Install:
/plugin marketplace add WST2002/claude-code-skillsthen/plugin install proof-of-done@claude-code-skills - GitHub · Featured on the HOL blog
security-audit: a Claude Code skill that finds security holes before attackers do
All my Claude Code skills: github.com/WST2002/claude-code-skills
Watch the demos on YouTube: @wst2002rk · proof-of-done Short · which-ai-to-use Short · security-audit Short
Read more: Switch AIs mid-task without losing context · Your AI said "fixed" but never ran the tests · My AI finds security holes before attackers do
Top comments (0)