AI-Generated Frontend Tests: Fake Green vs Real Coverage

I asked an agent to “add tests for this React form.” Ten minutes later CI was green, coverage jumped 12 points, and I felt productive. Then I broke the submit handler on purpose. Every test still passed.

That’s fake green: the suite looks healthy, the badge is bright, and the product is still unprotected.

AI is great at producing tests. It’s mediocre at choosing what those tests should prove. If you don’t constrain it, you get a wall of renders, snapshots, and happy-path clicks that never touch the bug you actually ship.

What fake green usually looks like

I keep seeing the same patterns from agent-written suites:

• Render-only tests — screen.getByText("Save") after mount. Proves the string exists, not that save works.
• Implementation spies — assert useState was called, or that an internal helper ran. Refactors break the test; bugs don’t.
• Snapshot soup — one giant DOM dump per story. Any class rename fails CI; none of it catches a wrong API payload.
• Mocked everything — fetch, router, analytics, even the button’s onClick. The only thing left under test is the mock setup.
• Coverage theater — lines execute because the test imports the module and calls every export with empty props. Branch that handles null user? Never hit.

Coverage went up. Confidence went down. That’s the trap.

What I ask for instead

I stopped saying “add tests.” I paste a short contract into the prompt:

Write tests that fail if these user-visible behaviors break. Do not assert on implementation details. Prefer one integration-ish test per critical path over five shallow renders.

Then I list the behaviors explicitly:

1. Submit with empty email shows the inline error and does not call the API
2. Valid submit posts { email, plan } to /api/signup and lands on /welcome
3. Server 409 shows the “already registered” message without clearing the form
4. Keyboard: Tab order reaches email → plan → submit; Enter submits when focused on email

If the agent can’t fail a test by breaking one of those lines, the test is decoration.

A quick litmus test before I merge

I run three checks on every AI-authored PR that “improves coverage”:

1. Mutate and watch — flip a condition, rename a field in the payload, remove an await. At least one test must go red. If none do, delete or rewrite.
2. Read the asserts — if every expect is toBeInTheDocument() or toMatchSnapshot(), send it back.
3. Count the mocks — more mocks than asserts is a smell. Prefer a small MSW handler over stubbing fifteen internals.

I also glance at the coverage report for branches, not lines. Fake green loves line coverage. Real confidence lives in the if you almost never take.

How I split work with the agent

Agents are fine at scaffolding: test file layout, MSW handlers, Testing Library queries once I name the roles. Humans still own:

• Which paths matter (checkout, auth, billing — not every Tooltip)
• What “done” means (the four behaviors above, not “80% coverage”)
• What not to mock (the component under test, the form library’s real validation if that’s the point)

My default prompt ending:

Do not add tests for private helpers. Do not increase coverage by importing unused modules. Every new test must map to one bullet in the behavior list.

That sentence alone cut a lot of snapshot noise.

Coverage as a signal, not a goal

I still look at coverage — as a map of untested branches, not a score to maximize. When an agent offers +15% coverage with zero new failing mutations, I treat that as a warning, not a win.

Real coverage for frontend is: can a user break the money path without a red test? If yes, the green build is lying.

Next time an agent hands you a shiny suite, break one real behavior on purpose before you merge. If CI stays green, you didn’t get tests — you got a costume.