Mutation Testing: Why Code Coverage Isn't Enough
How the Stryker tool reveals the true strength of your test suite — using a real business rule with clear limits.
A green "100% code coverage" badge feels great. But here's the uncomfortable question QA should ask: coverage says a line ran — does it say the behaviour was verified? If someone changed that line, would your tests notice?
Mutation testing answers exactly that. In this article I show it with Stryker + Jest, applied to a grain-service-fee business rule (a nod to agribusiness domain rules).
Full, runnable source on my GitHub.
What is a mutant?
Stryker makes a tiny change to your production code — a mutant — and re-runs the tests:
- tests fail → mutant killed ✅ (your test detected the change)
- tests pass → mutant survived ❌ (that line is executed, but not truly verified)
mutation score = killed mutants / total mutants
The killer example: 100% coverage vs ~86% mutation score
Two modules both report 100% line, statement and branch coverage. Only mutation testing tells them apart:
| Module | Coverage | Mutation score (weak suite) |
|---|---|---|
| Leap-year check | 100% | 100% (strong) |
| Taxa-service-fee rule | 100% | 86% — 5 mutants survive 🚨 |
The fee rule is a classic off-by-one factory:
if (value <= 1000) return 0; // isento (exempt)
const rate = type === 'soja' ? 0.02 : 0.015;
return round(value * rate);
The 5 surviving mutants are business bugs waiting to ship
| Surviving mutant | Why it survives |
|---|---|
value <= 1000 → value < 1000
|
the exact value 1000 is never tested |
value < 0 → value <= 0
|
the value 0 is never tested |
typeof value !== 'number' → false
|
no test passes a non-numeric value |
error message changed to ''
|
tests check toThrow(Type) but not the message |
These are off-by-one boundary bugs — precisely the defects that reach production wearing a green coverage badge.
The fix: boundary + message tests
Adding tests on the exact boundaries (0, 1000, non-numeric input) and asserting error messages kills every mutant → 100% mutation score.
Configuring the quality gate
Stryker can fail the build just like a failing test:
"thresholds": { "high": 95, "low": 80, "break": 95 }
With break: 95, a weak suite exits 1; a strong one exits 0. Wire this into CI and you turn coverage into an effectiveness gate.
Cost & levers
Each mutant re-runs the suite — slow. Mitigations: coverageAnalysis: "perTest" (only tests touching the line run), --incremental, and mutate only changed files in CI.
Equivalent mutants (the honest caveat)
A mutation that can't change observable behaviour (e.g. a pure logging call) is un-killable — that's why 100% isn't always the right goal. Use Stryker's thresholds and // Stryker disable comments.
Takeaways
- Coverage ≠ quality. Mutation testing measures your assertions, not your execution.
- Surviving mutants are a to-do list: missing test cases, weak assertions, or dead code to delete.
- Best where logic is dense in boundaries: pricing, validation, permissions, business rules.
The demo (strong vs weak suite, the SOLUTION=1 toggle, and the CI quality gate) is on my GitHub — a great conversation starter for senior QA roles.
Jessica Sales — QA Engineer. When I say my tests are good, mutation testing is what lets me prove it.
Top comments (0)