Can you trust a green test written by AI? I measured it.

I've spent eleven years building test automation for enterprise systems. The whole job rests on one quiet assumption: a passing test is evidence. Green means safe. We ship on green.

Lately that assumption has been getting shakier, because more and more of the test itself is being produced by AI:

• AI generates the test oracle (the assertion that decides pass/fail).
• AI heals broken UI locators when the page changes (self-healing frameworks).
• AI adjudicates whether a run actually passed or was "just flaky" (LLM-as-judge).

Each of these can hand you a GREEN result that isn't actually sound:

• An AI-written assertion can be too weak to catch a real bug — it checks the type but not the value, so the test passes even when the output is wrong.
• A self-healing locator can "repair" itself onto the wrong element and sail through, silently testing nothing.
• An LLM judge can wave a genuine failure through as a flake.

In all three cases the dashboard stays green while the safety net quietly grows holes. I wanted to stop hand-waving about this and actually measure it. So I built a small, open research prototype called TrustLab.

The idea: the False-Confidence Gap

The core concept is simple:

False-Confidence Gap = how often tests appear to pass − how often they are actually sound.

To measure it honestly you need ground truth for "sound." I use mutation testing: an engine injects real faults into a program (flip a + to a -, a < to a <=, and so on), and a verdict only counts as sound if it actually catches those faults. No opinions, no self-report — a test either kills the mutant or it doesn't.

I ran this across the three modalities above, on a set of small subject programs, repeated over 15 random seeds with held-out evaluation.

What came out

• AI-generated oracles: ~62% gap. This is the scary one. A huge share of "passing" came from assertions that don't actually catch injected faults.
• LLM-as-judge: ~41% gap.
• AI self-healing: ~11% gap — lower, but with silent mis-heals hiding in the tail (the worst kind, because nothing looks wrong).

A detail I didn't expect: for AI-generated oracles, the danger tracks code structure. Branch-heavy logic like a grading function (99.7% gap) or a triangle classifier (93.6%) was far more dangerous than a simple BMI bucket (5.2%). The more branches your code has, the more room a weak AI oracle has to look green while missing faults.

Can we do anything about it?

Measuring the problem is only half of it. The other half: can we predict which green verdicts to believe, using only signals we can actually observe at runtime (not the ground truth, which in real life we don't have)?

I trained a small, calibrated trust metric on observable features and compared it to the naive policy everyone uses today — "it's green, trust it."

If you keep the 70% most-trusted green verdicts and set aside the least-trusted 30%, the silent false-pass rate drops hard:

• Self-healing: 11% → 0.5%
• LLM-judge: 40% → 19%

The naive "trust any green" baseline can't separate sound from unsound at all (ROC-AUC 0.50 by construction). The trust metric reaches ROC-AUC up to ~0.95, with calibration (Expected Calibration Error) reported so the scores actually mean something. Effect sizes are "large" (Vargha–Delaney A12 0.76–0.95), Mann–Whitney p < 0.001.

Being honest about what's real

I think credibility matters more than hype, so I'm explicit about this in the repo too:

• The oracle results are real: a genuine AST mutation engine injects faults, and runtime coverage decides whether a verdict is sound.
• The self-healing and adjudication results are empirically-shaped models of documented tool and LLM behaviour, placed behind an adapter interface. That means you can drop a real Playwright self-healer or a real LLM judge into the same seam and re-run the exact experiments without touching the metric. (That's literally one of the open issues on the repo — contributions welcome.)

I'd rather you know exactly where the solid ground is.

Why it matters for the rest of us

If your pipeline uses AI to generate assertions, heal locators, or judge flakiness, "green" is no longer self-evidently trustworthy. Two practical takeaways:

1. Weak oracles are the biggest risk, and they're worst exactly where your code is most complex. AI-generated assertions deserve review in proportion to branch complexity.
2. A trust score beats a raw green/red. Even a simple calibrated metric over observable signals can tell you which passes to re-examine — turning a binary into a prioritised queue.

Try it / break it

Everything is open and reproducible — pure Python standard library core, 19 passing tests, the benchmark dataset, and the experiment outputs:

👉 github.com/rollamadugu/trustlab

I'd genuinely value feedback from people who build test oracles, self-healing frameworks, or LLM-evaluation pipelines — especially if you can poke holes in the methodology. There are a couple of "good first issues" open if you want to plug in a real tool. A fuller write-up (preprint) is in preparation.

---

I'm Krishnasai Rollamadugu, a test automation architect (11+ years) researching trust and measurement in AI-driven software testing. If this resonates, follow along — this is the first step of a longer research direction.