DEV Community

SDET Code
SDET Code

Posted on

What is Mutation Testing? A Practical Guide for QA Engineers

#testing #python #pytest #qa

Line coverage is a liar.

Your tests can cover 100% of your code and still miss critical bugs. Coverage tells you which lines ran -- not which bugs your tests actually catch.

Mutation testing fixes this gap. It answers a harder question: "If I introduce a bug into this code, will my tests detect it?"

How Mutation Testing Works

  1. Start with correct code -- the "golden" implementation
  2. Generate mutants -- AI or tools create variants with subtle bugs (off-by-one errors, wrong operators, missing null checks)
  3. Run your tests against each mutant
  4. Score -- if your test fails on a mutant, that mutant is "killed." Your kill ratio = killed / total mutants

A Simple Example

Given a function that calculates shipping cost:

def calculate_shipping(weight, distance):
    base = 5.0
    if weight > 10:
        base += weight * 0.5
    if distance > 100:
        base += distance * 0.1
    return round(base, 2)
Enter fullscreen mode Exit fullscreen mode

A mutant might change weight > 10 to weight >= 10 or weight > 11. If your tests don't cover the boundary at exactly weight=10, the mutant survives -- meaning your tests have a blind spot.

Why This Matters for QA Engineers

Code coverage tells you: "This line executed during testing."

Mutation testing tells you: "Your tests can actually detect when this line is wrong."

That's a fundamentally different -- and more useful -- measurement.

As QA engineers, our job isn't to execute code. It's to find defects. Mutation testing directly measures how good we are at that.

Three things mutation testing forces you to do better:

  • Think about boundary values -- zero, negative, maximum, off-by-one
  • Write specific assertions -- not just "it doesn't crash" but "it returns exactly this value"
  • Cover edge cases systematically -- every surviving mutant reveals a gap in your test strategy

See It In Action

Here's a quick demo of solving a mutation testing challenge -- finding a real bug in e-commerce pricing code:

Try It Yourself

I built SDET Code as a platform to practice mutation testing. Each challenge gives you Python code with hidden bugs (mutants), and you write pytest tests to catch them.

What's live:

  • 339 challenges across 6 real-world domains (fintech, commerce, SaaS, platform, content, common)
  • AI Coach with personalized feedback and skill gap analysis
  • Runs 100% in the browser via WebAssembly (Pyodide) -- no setup
  • Free tier with daily challenges

It's the kind of practice platform I wished existed when I was preparing for SDET interviews.

This is Part 1 of the "Mutation Testing for QA Engineers" series. Next up: How to write pytest tests that actually catch bugs.

What's your experience with mutation testing? Have you used tools like mutmut or cosmic-ray? I'd love to hear how QA teams are measuring test quality beyond coverage.

Top comments (0)