DEV Community

Nikolay Chernev
Nikolay Chernev

Posted on

Your AI Wrote 3 Tests. 3 Passed. The Code Was Still Broken.

#testing #ai #qa #programming

Your AI Wrote 3 Tests. 3 Passed. The Code Was Still Broken.

I ran this exact scenario last week: an AI assistant writes a small test suite against a function with a seeded bug. All 3 tests pass. Green checkmarks, done, ship it. Then I ran an independent verification suite against the same code — written to check the spec, not to agree with whatever the implementation already does. It failed, and it caught all 4 bugs I'd planted, including the one the AI's own tests walked right past.

That gap — "the tests pass" vs. "the code is correct" — is the single biggest trust problem in AI-assisted testing right now, and it's not a vibe. Stack Overflow's 2025 Developer Survey put real numbers on it: 84% of developers now use or plan to use AI tools, up from 76% in 2024, but trust in the accuracy of what those tools produce kept falling — Stack Overflow's own analysis puts it at 29%, down from 40% the year before, and 46% of developers say they actively distrust AI output (InfoWorld, LeadDev). Adoption is up. Trust is down. That's not a contradiction, it's a warning sign.

Why "all tests passed" doesn't mean what you think

Here's the mechanism, and it's structural, not a one-off glitch. When the same model writes the implementation and the tests, both inherit the same blind spots. If the code has a bug — say, a counter that increments regardless of whether a request actually succeeded — a model asked to "write tests for this function" will often write an assertion that checks the behavior the code actually has, not the behavior the spec requires. The test and the bug agree with each other. Nothing goes red, because nothing was ever checking against an independent standard.

This is distinct from flakiness, though the two get confused. Flaky tests fail intermittently on correct code. What I'm describing is the opposite and arguably worse: tests that pass reliably on incorrect code. Google's own engineering data shows roughly 16% of tests exhibit some flakiness, and flaky failures account for around 84% of red builds — a huge, well-studied problem. But a test suite that passes consistently on broken code doesn't even give you a red build to investigate. There's nothing alerting you. That's the harder failure mode to catch, because every signal you'd normally rely on says "fine."

A concrete way to tell the difference

The fix isn't "don't use AI to write tests." It's: don't let the same pass of AI-authored assertions be the only check. Verify the verifier.

  1. Separate the test-writer from the test-grader. Don't ask the same prompt/context that explains the implementation to also validate it. Write (or generate) an independent suite from the spec/requirements alone, without showing it the implementation.
  2. Run the AI's suite against a known-broken version of the code. If it still passes 3/3, the suite isn't testing behavior — it's testing "does the code run."
  3. Check what fraction of seeded bugs survive. This is exactly what mutation testing measures (coverage tells you code executed; mutation score tells you whether anything was actually checked) — but you don't need a mutation-testing framework to get the intuition. Plant 3-4 real bugs by hand and see which ones your AI-written suite catches.
  4. Treat a pass from AI-authored tests as a claim, not a verdict. "Don't trust it — run it" against an independent check before you believe it.

I built a small, free, runnable lab that makes this concrete instead of theoretical: a deliberately broken implementation, an AI-generated test suite that passes 3/3 against it, and an independent verification suite that fails and catches all 4 seeded bugs. You can run both in under a minute and watch the gap yourself:

Free lab (GitHub): https://github.com/chernevnikolay86-wq/ai-test-verification-lab

node run-lab.js weak              # AI's own tests: PASS 3/3 on broken code
node run-lab.js verify-candidate  # independent suite: FAILS, catches all 4 bugs
Enter fullscreen mode Exit fullscreen mode

If this distinction — "the model failed" vs. "there's a real bug, and the model's own tests just didn't catch it" — is a question you're currently answering by gut feel, that's the exact problem the lab is built to make visible. It's also chapter one of a longer field guide I wrote on verification-first AI testing for QA/SDET engineers, with reproducible examples instead of slide-deck claims:

No guarantees, no "best practices" fluff — just a repeatable way to check whether you can trust what the AI just handed you.

Top comments (0)