Originally published on the Dromeas blog.
The pull request looks clean, the linter's green, the tests pass — and the code still does the wrong thing the moment someone hands it an input nobody thought about. Not a crash, not a lint warning. Just the wrong answer, quietly.
I've been digging into why that keeps happening even as AI writes more and more of our code, and it turns out there's decent data on it now — not just vibes. So let's go through what the numbers actually say, why the tools most of us already run can't really catch this category of bug, and what I think actually works.
What the 2026 data says
First: people don't fully trust this code, and it's specifically about correctness, not whether it runs. Sonar's 2026 State of Code Developer Survey found 96% of developers don't fully trust that AI-generated code is functionally correct, and 61% agree that "AI often produces code that looks correct but isn't reliable." Only 48% say they always verify AI-assisted code before committing it. Worth sitting with that for a second: the complaint isn't "it doesn't compile." It's "it runs, it looks fine, and I don't actually know if it's right."
Second, when you break down what kind of bugs show up, it lines up with that. A 2026 empirical study ("Debt Behind the AI Boom") looked across five widely used AI coding tools — Copilot, Claude, Cursor, Gemini, Devin — and found code smells made up 89.3% of flagged issues (the stuff any linter catches fine). Correctness issues were a smaller slice, 6.0% — but the single most common one was "undefined variable or reference," almost 24,000 instances, which the researchers describe as code that "may look locally correct, but still fails to stay consistent with the surrounding context." That's a pretty good one-line description of the whole problem. More than 15% of commits from every single tool they studied introduced at least one issue, and 22.7% of the AI-introduced issues they tracked were still sitting in the repo, unnoticed, at the time of the study — some for nine months or longer.
Third, when this reaches production, it's not staying theoretical. New Relic's 2026 State of AI Coding report found 82% of organizations had at least one major production failure caused by AI code in the past six months, and 78% report a measurable spike in incidents tied to AI code overall — with AI-generated code introducing roughly 1.7x more critical runtime issues than human-reviewed code. CloudBees' 2026 State of Code Abundance report found something similar from a different angle: 81% of enterprise leaders report increased production issues tied to AI code, and their own summary of it stuck with me — "writing code is no longer the primary bottleneck, governing it is." Same conclusion, three separate 2026 surveys.
And the backdrop makes all of this harder to catch by hand. GitClear's 2026 research tracked eight quality signals across 623 million code changes from 2023 to 2026, and the trend lines aren't great: duplicated code blocks up 81% since 2023 (highest on record), copy-paste share of changed lines up from 9.4% to 15.7%, actual refactoring down about 70% over the same stretch. GitClear's own read on it: block duplication is the single risk signal most tied to defects and propagated bugs in the research they looked at. So more code is shipping, less of it is getting consolidated or re-examined, and reviewers have less bandwidth per line than ever.
Put simply: it's not that AI writes "bad" code. It clears the bars we're good at checking automatically — style, structure, the obviously dangerous patterns — and it's specifically weaker on whether the logic holds up for a real input, in a way that's now showing up as real production failures, not just review comments. That's worth being precise about, because it tells you exactly why the tools most teams already run don't solve it.
What these bugs actually look like
"Behavioral bug" is the term I keep reaching for, but it's worth knowing it's not the only name people use for this — you'll see the same basic idea called logic bugs or logic errors (the plain-English default), semantic bugs (more of an academic/compiler-literature term — the code is syntactically fine but means the wrong thing), correctness bugs or correctness issues (the term the arxiv paper above uses), business logic bugs or business logic vulnerabilities (the AppSec framing), and functional bugs (QA/testing terminology). Some people just call them silent bugs or silent failures, which honestly might be the most useful name of the bunch — it points at the actual property that makes them dangerous: nothing crashes, nothing throws, nothing shows red in CI.
In practice, most of what falls under that umbrella breaks down into a handful of recurring shapes:
Boolean logic bugs — an inverted condition, the wrong operator, a guard clause that lets through exactly the case it was written to block. Classic version: a permission check written with || where it needed &&, so access gets granted if any one condition is met instead of requiring all of them.
Reference correctness bugs — stale closures, the wrong variable captured, the wrong object mutated once two calls overlap. Classic version: a loop that captures its loop variable by reference in a callback, so every callback ends up pointing at the last value instead of the one that was current when it was created.
Boundary and indexing bugs — the everyday off-by-one: loop bounds, first/last-element handling, pagination math. Classic version: a "load more" offset computed as page * pageSize instead of (page - 1) * pageSize.
Normalization-symmetry bugs — a value gets normalized on write but not on read (case, whitespace, encoding), so a lookup silently misses. Classic version: emails lowercased at signup but not at login.
Nullability bugs — a new code path where a value can legitimately be absent, and nothing downstream accounts for it.
Shared-state concurrency bugs — races between concurrent writes, a rollback to a stale captured value, a cache write that clobbers something fresher.
None of these are exotic. Every engineer has shipped at least one of each at some point. What's changed is the volume and the plausibility — code that reads as confidently correct, generated fast enough that nobody's tracing each new input by hand anymore.
Why static analysis can't really get at this
Tools like SonarQube, Semgrep, and CodeQL work by pattern matching — an AST shape, a taint path from untrusted input to a dangerous sink, something that matches a known CWE. Genuinely useful, and it's why these tools are worth running. But it only works if the bug matches a pattern someone already wrote a rule for.
Most behavioral bugs don't work that way. Gecko Security put this well in a piece on why static analysis struggles with business logic: a lot of these bugs are about something being missing — a missing auth check, a missing validation step — rather than something dangerous being present that a rule can flag. Taint analysis can tell you untrusted data reaches a sensitive spot. It can't tell you whether the authorization logic guarding that spot is actually correct. And a lot of the bugs that cause real incidents span multiple files — their example is a real Cal.com auth-bypass bug that needed three separate issues chained together across different files, each looking fine on its own.
So that's the real limitation: pattern matching can tell you something looks like a shape it's seen before. It can't tell you that a specific input, actually walked through the logic, produces a different result than it should. That requires tracing behavior against intent — and it happens to be exactly where the data above says AI-generated code is weakest.
It's also why I think "detect that AI wrote this, then review it harder" — which a few vendors have shipped, SonarQube's AI Code Assurance being one — is sorting on the wrong axis. It's routing on who wrote the code, when the thing that actually matters is what the code does. The question was never "who wrote this" — it's "does this do the right thing for a real input," and that needs tracing, not detection.
Where this leaves Bug Tracing
This is basically the problem we built Bug Tracing to solve inside Dromeas — trace real inputs through the actual changed code and check what comes out, instead of scanning for patterns. The six categories above are exactly what it scores every change against, and it only reports a finding if it can name the concrete input that triggers it — no reproducible trace, no report.
And because "review a diff" and "audit a whole system" are genuinely different jobs, it runs at whichever scope your team actually works at: on the pull request itself (changed lines plus their blast radius), on a branch or release after merge for teams doing trunk-based development, or as an on-demand sweep across a whole repository when you just want a second opinion on a system nobody's looked at closely in a while.
If this is a problem you're running into, happy to show you how it works — the feature page is at dromeas.ai/bug-tracing.
Top comments (0)