The AI PR that raised coverage with tests that never fail

I keep seeing this in agent-written PRs.

The ticket asked for a real fix. The agent also “added tests.” Coverage went up. CI is green. The summary sounds responsible. Then you read the suite and every assertion is on the mock: toHaveBeenCalled, toHaveBeenCalledWith, a stubbed return value. Nothing fails if the production function returns the wrong shape, skips the branch, or stops calling the collaborator that still matters.

You did not ask for a prettier coverage number. You asked for a merge you can trust.

The 60-second check I run now

When an agent PR adds or rewrites tests:

1. Temporarily break the production change (or comment out the real branch). Do any new tests go red for the right reason?
2. Prefer at least one assertion on a real return value, error, or side effect — not only that a mock was invoked.
3. If the only new lines are mock.* expectations, I bounce it or ask for one behavior assertion before approve.

This is one of the stops on a short bar I use before I merge agent PRs. Free one-pager if you want the rest: https://chopragunji.gumroad.com/l/zpnmdn

If you'd rather someone else do a hard pass on one scary AI PR, I still have founding slots at $49 (normally $99): https://chopragunji.gumroad.com/l/byoyi/FOUNDING

Have you merged an agent PR whose “new tests” only watched the mocks?