I built a single compile Mutation testing lib for Kotlin which runs inside your normal test suite

Andreas Schnapp — Tue, 03 Mar 2026 07:44:00 +0000

Introduction to mutflow
Testing existing code
Correct it to pass mutation tests
Final Conclusions

Introduction to mutflow

Hey together,

I built a K2 Compiler Plugin based mutation testing library for kotlin. It is quite different from most of the other mutation libraries or frameworks.

It does only include mutations for the class which is under test and also only the code you have reached within your test class. It works best if you have a good test coverage already but you want to be sure that the behavior is fully tested.

It's really easy and fast to include to your project and to integrate with your current tooling.

In fact it just use a JUnit extension feature which runs your tests for one test class several times (one time per tested mutation).

To enable mutflow you just have to put it in your gradle plugins:

plugins {
    kotlin("jvm") version "2.3.0"
    id("io.github.anschnapp.mutflow") version "0.8.0"
}

Maybe it's easiest to see by a little example.

Testing existing code

Let's say i have this little calculator class:

// this annotation mark this class to get configurable mutation points
// inside the test build (dual compile approach production keeps clean)
@MutationTarget
class RangeValidator {
    fun isInRange(x: Int, min: Int, max: Int): Boolean {
        return x >= min && x <= max
    }
}

And this straight forward test:

@MutFlowTest
class RangeValidatorTest {
    private val validator = RangeValidator()

    @Test
    fun `isInRange returns true for value in range`() {
        val result = MutFlow.underTest { validator.isInRange(
            x = 5,
            min = 0,
            max = 10
        ) }
        assertTrue(result)
    }

    @Test
    fun `isInRange returns false for value below range`() {
        val result = MutFlow.underTest { validator.isInRange(
            -1,
            0,
            10) }
        assertFalse(result)
    }
}

This test looks fine in general, it has full code coverage.
And testing 2 different cases with meaningful assumptions.

But do we test all relevant behavior here?

Let's see what happens when we now run our tests.

Here we see the results of the test run in my current IDE (this works without any IDE plugin)

It shows that all tests are run multiple times in a row.

First there is the baseline test. This tests run against the normal implementation as is. Because tests had pass before introduction to mutflow the baseline is also green (nothing changed here)

Then there is one run of all tests per mutation which is tried by the library.

The first mutation has changed the <= to a < expression.

Now we need a little mind twist here. The test series which are run per mutation are green if at least one test don't pass.
The idea is that if someone would do a significant change to a correct program. Then the program should not be longer correct and the tests should fail. If they don't fail, then most properly the test does miss to test the behavior fully.

The <= to < mutation fail means that we don't have tested the border cases close enough. For the is in range we should test the exact border cases (which is also quite intuitive for experienced developers). In the log output of the tests you will be pointed on the line where this mutation appears.

Correcting the code for passing mutation tests

So let's change the test for testing as close to the borders as possible.

@MutFlowTest
class RangeValidatorTest {
    private val validator = RangeValidator()

    @Test
    fun `isInRange returns true for value in range, left border`() {
        val result = MutFlow.underTest { validator.isInRange(
            x = 0,
            min = 0,
            max = 10
        ) }
        assertTrue(result)
    }
    @Test
    fun `isInRange returns true for value in range, right border`() {
        val result = MutFlow.underTest { validator.isInRange(
            x = 10,
            min = 0,
            max = 10
        ) }
        assertTrue(result)
    }

    @Test
    fun `isInRange returns false for value below range left border`() {
        val result = MutFlow.underTest { validator.isInRange(
            -1,
            0,
            10) }
        assertFalse(result)
    }

    @Test
    fun `isInRange returns false for value below range right border`() {
        val result = MutFlow.underTest { validator.isInRange(
            11,
            0,
            10) }
        assertFalse(result)
    }
}

Now we run again the test and see if it worked:

Yes as expected our clean border testing works. Now we don't have only tests which covers each code line but which seems also be robust enough for our mutations.

Final Conclusions

Of course this was a trivial example and the real world is a lot of more complex and nuance.

There are already quite some little features and helpers in the library to cope with reality:

easy ways of marking lines or function be ignored regarding mutations (only useful inside mutation target class)
global way to deactivate mutation testing even if project and tests are configured for it
limit amount of mutations per test. which has a stable enough mechanism where mutation are stable as long as implementation does not change. so it's fine to use this in a pipeline.
timing out if mutation lead to infinite loops (then you get a hint to mark this line for ignoring)

Also this project is young, for now i have very carefully selected the mutations which i have seen as most valuable.

In general i think the set of mutation should be kept small. So that it's a good trade off regarding finding useful things, having good performance and don't have many false positives.

Built with the help of an AI assistant — all design decisions carefully choose.

Feedback most welcome

But of course feedback and real world usage would be required for making this tool as useful as it could be. Therefore early birds and feedback is most welcome.

anschnapp / mutflow

Mutation testing inside your Kotlin tests. Compile once, catch gaps.

Mutation testing inside your Kotlin tests. Compile once, catch gaps

Early Stage: mutflow is young and still evolving. Its dual-compilation approach is built to keep production builds clean — mutations only exist in test compilation. The project hasn't seen broad adoption yet, so bug reports and feedback are very welcome!

What is this?

mutflow brings mutation testing to Kotlin with minimal overhead. Instead of the traditional approach (compile and run each mutant separately), mutflow:

Compiles once — All mutation variants are injected at compile time as conditional branches
Discovers dynamically — Mutation points are found during baseline test execution
Runs all mutations — Every discovered mutation is tested by default, no configuration needed

Why?

Traditional mutation testing is powerful but expensive. Most teams skip it…

View on GitHub

DEV Community: Andreas Schnapp