Green tests are lying to you.

There's a reflex every engineer has, and I had it for a decade: the test suite goes green and my shoulders drop. Green means safe. Green means ship. Green means someone checked.

That reflex is now wrong, and it took 30 days of letting AI write 100% of my code to see exactly how wrong.

Here's the sentence I want you to sit with: a passing test proves your code agrees with your test. It has never proved your code is correct. Those were the same thing for years — but only by accident, and the accident just ended.

This isn't a hype post and it isn't a doom post. It's about why the most trusted signal in your CI pipeline quietly stopped meaning what you think it means.

What green used to mean

For most of software history, a passing test carried a hidden second signal you never had to think about: a human, at some point, wrote that test on purpose.

Even when the same person wrote the code and the test, there was a gap in time and attention between the two acts. You wrote the function believing one thing, then you sat down later — different mindset, slightly more skeptical, sometimes after lunch, sometimes after a bug — and tried to pin down what "correct" meant. The test was a second pass by a second version of you. Not a perfect check. But a real one, because the person writing the assertion was at least trying to catch the person who wrote the code.

Green meant: two attempts, made at different times, agreed. That's a weak form of independence, but it's genuine, and it's the entire reason the green checkmark ever earned your trust.

What green means now

When AI writes the code, it also writes the test. And it writes both in the same pass, from the same context window, holding the same mental model of what the code is supposed to do.

There is no second version. There is no gap in time or attention. The thing that decided what "correct" means is the exact same thing that decided what the code does — in the same breath, from the same prior.

So when the test passes, it's telling you something real. It's just not the thing you think:

The test doesn't confirm the code is correct. It confirms the code matches the assumption the model already had when it wrote the code.

If that assumption was right, green means right. If that assumption was wrong, the model writes wrong code, then writes a test that asserts the wrong behavior is the expected behavior — and the test passes, bright green, confirming the bug as a feature.

The test isn't checking the code. It's a mirror held up to the code's own assumptions. And a mirror never disagrees with you.

The exam nobody proctored

The cleanest way I can put it: the model is a student who writes its own exam, takes its own exam, and grades its own exam — all in one sitting, from one understanding of the material.

Of course it passes. It would be shocking if it didn't. But "the student passed the exam it wrote for itself" tells you the student is consistent. It tells you nothing about whether the student understands the subject. Consistency and correctness feel identical right up until the moment the student's understanding is wrong — and then they diverge completely, silently, with a 100% score stapled on top.

Green tests measure "did I build what I intended?" They have never once measured "was my intention correct?" We got away with conflating those two questions for years because a human's intention was checked by other humans — in review, in production, in the 2am page. Hand both the intention and the verification to the same model and the conflation isn't a shortcut anymore. It's the whole failure.

The bug that ran green all the way to production

Let me make this concrete, because it's not hypothetical — it cost a version of me real trust.

During my 30-day experiment, the AI wrote a Stripe webhook handler. It acknowledged the event — returned 200 to Stripe — and then persisted the row to the database. Ack first, write second.

The tests were green. All of them. The model wrote a test that fired a webhook, asserted a 200 came back, and asserted the happy-path row got written. Textbook. Beautiful coverage numbers.

But look at what the test couldn't assert, because the model that wrote it didn't believe there was anything to assert: what happens if the database blips in the gap between the ack and the write? In production, that gap is real. One hiccup and you've told Stripe "got it, we're done" while your own database never heard about the payment. The result is a paying customer with access to nothing and no record they ever paid. Stripe considers it delivered. You have no idea it happened.

The test didn't miss this because it was a bad test. It missed it because it was written by the same mind that thought ack-before-persist was clean. You cannot write an assertion against a failure mode you don't believe exists. The green checkmark wasn't a lie about the code. It was an honest report of a wrong belief.

Let me be fair to the tests

I'm not telling you to delete your test suite. That would be the doom-post version of this, and it's just as wrong as the hype.

Tests written this way are still genuinely good at one thing: catching the code from drifting away from itself later. Regression is a real category of bug — you change one thing, something three files over quietly breaks — and a green-agrees-with-green suite catches exactly that, cheaply, forever. That's worth a lot. I run more tests now than I ever did, not fewer.

What I stopped doing is treating green as a verdict. Green is a consistency check. It proves today's code still agrees with the day it was written. It cannot, structurally, tell you whether the day it was written was already wrong. Those are different jobs, and the green checkmark only ever did one of them. We just never had to notice, because a human used to do the other.

What makes green mean something again

The fix is the same structural move I keep landing on, and it's not "write better tests." It's who writes the test.

A test only checks an assumption when it's written by something that doesn't already hold that assumption. So you buy that separation deliberately, in three places:

1. A different author for the assertion than for the code. Not "write tests" bolted onto the same agent in the same pass — a separate reviewer whose entire job is to write the test the author would never think to write, because the author can't see the state that would break it. The test that asserts the row exists after a simulated DB failure between ack and write. That test only gets written by something looking for the failure, not confirming the success.

2. A different frame. Stop generating tests that ask "does the happy path work?" It always does. Generate the ones that ask "produce the input that loses a customer money, then assert we don't." A test hunting a specific failure is worth ten that bless the demo. Same model, completely different output — because the objective, not the model, is what's doing the checking.

3. A human on the merge who's been burned. Some failure modes don't live in any prompt because they live in someone's scar tissue — the memory of the 2am reconciliation, the paged-at-3am dual-write nightmare. That person reads ack-before-persist and goes pale in five minutes. No coverage number produces that flinch. It has to be lived.

Stack those and green starts meaning something again, because now a bug has to survive a test it didn't get to write, a frame built to break it, and a human who's paid for it before. Any one of those alone, and the checkmark is decoration.

The structure, not the prediction

Here's what I actually believe after 30 days of watching green tests pass over landmines:

• A test written in the same pass as the code checks consistency, never correctness.
• To check correctness, the assertion has to come from something that doesn't share the code's assumptions.
• And a human owns the merge, because the assumption most likely to be wrong is the one nobody — no model, no test — thought to question.

That's not a claim about which model wins. It's a structure, and it holds whether the author is today's model or GPT-9, because it doesn't depend on the model being good. It depends on the check failing differently than the thing it checks.

It's also, not coincidentally, exactly how we build xenition: an agent that does the work, a different agent whose job is to write the test that breaks it, and a person who owns the merge. I didn't arrive at that from a thesis. I arrived at it from watching a suite go fully green over a webhook that quietly loses paying customers.

"All tests passing" is the beginning of review. It has never been the end of it. We just had a human quietly doing the rest, and we called it green.

---

Three questions I'd genuinely like answered in the comments:

1. Does your AI-generated test suite get written by the same agent, in the same pass, as the code it tests? Have you ever measured what a separate adversarial test-writer catches that it doesn't?
2. What's a bug you've shipped with 100% green tests — the one where the test and the code were wrong in the exact same way?
3. If tests only prove consistency now, what's the signal you actually trust before you merge — and where does it come from?