Most AI-generated code I audit isn't wrong. That's the problem. It's plausible. It compiles, it runs, it passes the happy path — and it fails in a handful of predictable, repeating patterns that you start spotting once you know the shapes.
This is a field guide for those shapes. Written for developers doing code review on an MVP they didn't write, and for the non-technical founders who might be about to hand one over.
The thesis
LLMs are optimized to produce outputs that resemble correct code, not to produce correct code. The failure modes of an AI-assisted MVP are therefore specific and recognizable — not random bugs, but recurring tells. If you know what to look for, you can triage a codebase in an afternoon.
The assumption this catches out
Most founders (and more than a few engineers) operate on:
If the tests pass and the happy path works, the AI got it right.
LLM code is adversarially disappointing to this heuristic. It's trained to match the distribution of correct code, which means it reproduces the surface features — shape, naming, structure — far more reliably than the load-bearing details. The obvious tells compile. The subtle ones ship.
The 8 tells
1. Auth that's almost right
Look for: endpoints without an auth middleware, missing CSRF on state-changing routes, hand-rolled password hashing (crypto.createHash('sha256').update(password).digest('hex') instead of a proper KDF like bcrypt or argon2), session cookies without HttpOnly / Secure / SameSite, or a password-reset flow that doesn't invalidate prior sessions.
Why it happens: the AI wrote each piece correctly in isolation; nothing enforces that the pieces work as a system.
// classic tell: login writes a session, but /api/internal/* doesn't check it
app.post('/api/login', handleLogin);
app.get('/api/internal/users', getAllUsers); // no middleware
2. Over-specific helpers that pretend to be general
Functions like getUserOrdersByStatusAndDateRangeAndRegion. Fifteen arguments, three of them optional, all with defaults that only make sense for the one caller. Six months later you'll find its near-identical cousin, listOrdersForUserFilteredByStatus, written when nobody realized the first one existed.
Fix by auditing for duplication with a name-similarity pass, not by reading line-by-line. A 15-line grep can usually surface the worst offenders.
3. try/except as a worldview
try:
charge_customer(user, amount)
except Exception:
pass # or: logger.info("failed")
Silent failure is the most expensive pattern in an AI-generated codebase because it hides the other bugs. Payments that didn't go through. Emails that weren't sent. Rows that didn't get written. The product appears healthy right up until a customer notices.
Audit rule: any bare except, any catch (e) {}, any error that only goes to stdout is a review item.
4. Data models that encode today's assumptions forever
One user per account. One product per order. One currency. One language. One timezone. The AI wrote the schema from the prompt it was given; the prompt described the product as the founder saw it on day one.
Tell: foreign keys that should be many-to-many are modeled as single columns. Join tables don't exist. Status fields are booleans instead of enums. Money is stored as float.
The cost isn't correctness — it's that every "one more thing" a customer asks for a year later is a migration.
5. Integrations without the boring parts
Stripe is integrated. Webhooks arrive. What's missing: signature verification, idempotency keys, retry handling, dead-letter queues for failed events, rate-limit handling.
The tell is the absence of tests around failure modes, not the presence of buggy code. Ask: what happens if this webhook arrives twice? If the answer is "I don't know," that's the audit finding.
6. N+1 queries the ORM was supposed to prevent
LLMs love writing for user in users: user.orders without realizing that's a query per iteration. Works fine with 10 users. Falls over at 10,000.
Tell: no query logging in development, no slow-query alerts in production, no select_related / include / populate anywhere in the code. This one's easy to find once you turn on the logs for ten minutes of real traffic.
7. Config and secrets entangled
.env committed to git, or worse, constants hard-coded in the source. Separate dev/prod configs that don't actually differ. Secrets rotated by editing a file on a server somewhere.
Don't laugh — this is maybe the single most common finding on AI-generated MVPs. LLMs default to inline constants because that's what their training data shows.
8. Tests that test the mock
When tests exist at all in an AI-generated MVP, they often assert against the mock, not against behavior. The test passes when the code calls the mock the way the test expects — regardless of whether that behavior is correct.
// tests that this function calls its dependency — not that it does the right thing
expect(sendEmail).toHaveBeenCalledWith(user.email, 'welcome');
Fine for isolating logic. Dangerous as your only safety net, because refactoring moves the mock calls around and the tests break for the wrong reasons — so they get deleted.
The possible solution: a repeatable triage pass
Don't rewrite. Triage.
- Hour 1 — inventory. List every endpoint, every external integration, every place data is persisted. Most of the risk lives in a small part of this list.
- Hour 2 — hit the 8 tells. Grep for the patterns above. Capture findings in a doc with severity tags. Don't fix anything yet.
- Hour 3 — triage with the founder. Sort into security-now, correctness-soon, performance-eventually, leave-alone. Business priorities drive this, not code aesthetics.
- Then, and only then, fix the "now" tier. No new features during this window. This is the discipline most founders skip.
Full rewrite isn't wrong because it's technically bad. It's wrong because it throws away the one thing the MVP earned: evidence of what users actually want.
Summary
If you're auditing an AI-assisted MVP: the bugs aren't hiding in clever places. They're hiding in exactly eight familiar shapes. Know the shapes, grep for them, triage, and ship a short fix list.
If you're the founder who built it: you don't need to become technical. You need someone who knows the tells to spend half a day with your codebase before the next enterprise demo, security questionnaire, or engineering hire — whichever comes first.
AI is a great way to get to signal. It is not, yet, a great way to stay at signal.
I'm a developer who runs **The AI Mechanic* (theaimechanic.dev) — a practice focused on auditing and stabilizing MVPs built by non-technical founders and AI-assisted teams. There is a shorter, founder-facing version of this argument at theaimechanic.dev/blog/invisible-bugs-in-ai-generated-code.*
Top comments (0)