This is a really useful catch. You’re right, mutmut can give a perfect score here while missing the semantic mutation that actually matters. That makes the “plausible wrong implementation” check more important than the mutation score itself. Would you treat mutation tools as a supplement, with semantic mutants derived from the contract?
Solution architect, 16 years in software, last 3 on AI automation. I write about making automated work verifiable: evaluation harnesses, audit trails, and results published even when the answer is no.
Supplement, and specifically as a carrier rather than a generator. I ran mutmut 3.7.0 on the corrected implementation against both suites: 5 mutants, 5/5 killed with the two checks, and 5/5 killed again once test_empty is added, so the score is flat across exactly the check this article is about. It cannot rank the two suites any better than it could rank the two implementations. The contract mutant is what separates them: statuses is None becomes not statuses, which passes the two-check suite and fails test_empty in the three-check suite, so the catalogue reads 5/6 against 6/6 and every bit of the ranking power comes from the one mutant you have to derive from the requirement by hand. Boundary: CPython 3.14.6, mutmut default operators, this fixture only.
That comparison makes the limitation very concrete: adding the missing requirement check improves the suite, while the default mutation score stays unchanged. The useful unit to preserve is the named contract violation and the assertion that rejects it. Thanks for running both suites and stating the environment; that makes the result much easier to interpret without generalizing beyond this fixture.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
This is a really useful catch. You’re right, mutmut can give a perfect score here while missing the semantic mutation that actually matters. That makes the “plausible wrong implementation” check more important than the mutation score itself. Would you treat mutation tools as a supplement, with semantic mutants derived from the contract?
Supplement, and specifically as a carrier rather than a generator. I ran mutmut 3.7.0 on the corrected implementation against both suites: 5 mutants, 5/5 killed with the two checks, and 5/5 killed again once
test_emptyis added, so the score is flat across exactly the check this article is about. It cannot rank the two suites any better than it could rank the two implementations. The contract mutant is what separates them:statuses is Nonebecomesnot statuses, which passes the two-check suite and failstest_emptyin the three-check suite, so the catalogue reads 5/6 against 6/6 and every bit of the ranking power comes from the one mutant you have to derive from the requirement by hand. Boundary: CPython 3.14.6, mutmut default operators, this fixture only.That comparison makes the limitation very concrete: adding the missing requirement check improves the suite, while the default mutation score stays unchanged. The useful unit to preserve is the named contract violation and the assertion that rejects it. Thanks for running both suites and stating the environment; that makes the result much easier to interpret without generalizing beyond this fixture.