Mutation testing tells you how good your tests really are. Instead of only checking code coverage, it changes your compiled bytecode in small ways (creates “mutants”) and reruns your tests. If your tests fail, the mutant is “killed.” If they still pass, the mutant “survived,” which means your tests did not detect a meaningful behavioral change.
This tutorial walks Java developers through mutation testing in a Quarkus project using PIT (a popular mutation testing engine). You’ll learn why it matters, how to set it up, and how to read the results. We’ll use a tiny service class with a deliberate edge-case to keep things concrete. If you want to learn more about Quarkus tests in general, make sure to check out the excellent testing guide.
Why mutation testing matters in enterprise Java
Line and branch coverage tell you what code your tests executed. They do not tell you whether those tests would fail if the code were subtly wrong.
Enterprises depend on business rules that hide in small conditionals and boundary checks. These are exactly the places where off-by-one errors and “default path” bugs live. Mutation testing flips comparison operators, removes conditionals, nudges constants, and more, then checks whether your test suite notices.
Key benefits:
Reveals false confidence from high coverage but weak assertions.
Forces tests to assert behavior, not just call methods.
Focuses developer attention on risky code paths and boundary cases.
What we’ll build
We’ll create a fresh Quarkus project with:
A small
DiscountServicethat applies tiered discounts.A JUnit 5 test that misses an edge case.
PIT configured via Maven.
A quick iteration to “kill” a surviving mutant by improving the test.
We’ll keep REST endpoints out of the test path to keep the signal clear. You can absolutely use mutation testing with Quarkus tests; start with core services first.
Prerequisites
Java 17 or newer
Maven 3.8+
A terminal
All commands run from the project root.
Project bootstrap
Create a minimal Quarkus app:
mvn io.quarkus.platform:quarkus-maven-plugin:create \
-DprojectGroupId=org.acme \
-DprojectArtifactId=quarkus-mutation-demo \
-DclassName="org.acme.GreetingResource" \
-Dpath="/hello"
cd quarkus-mutation-demo
We won’t use the generated REST resource in tests, but it proves this is a Quarkus project.
Add PIT to Maven
Open pom.xml and ensure you have a standard Quarkus setup. Then add the PIT Maven plugin. The configuration below targets the org.acme package, uses the stronger default mutator set, and runs JUnit 5 tests. It also disables timestamped report folders to keep the report path stable.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.acme</groupId>
<artifactId>quarkus-mutation-demo</artifactId>
<version>1.0.0-SNAPSHOT</version>
<name>quarkus-mutation-demo</name>
<properties>
<!-- Quarkus: use your team’s pinned version here -->
<!-- PIT: pin on purpose; update when you choose -->
<pitest.version>1.16.6</pitest.version>
<pitest.junit5.version>1.2.1</pitest.junit5.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>${quarkus.platform.group-id}</groupId>
<artifactId>${quarkus.platform.artifact-id}</artifactId>
<version>${quarkus.platform.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<!-- Quarkus test stack (JUnit 5) -->
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-junit5</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.rest-assured</groupId>
<artifactId>rest-assured</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<!-- Keep your regular test lifecycle intact -->
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.2.5</version>
<configuration>
<!-- if you're running >Java 17 -->
<argLine>--add-opens java.base/java.lang=ALL-UNNAMED</argLine>
<!-- PIT runs unit tests; keep ITs out of its way -->
<includes>
<include>**/*Test.java</include>
</includes>
</configuration>
</plugin>
<!-- Optional: integration tests named *IT.java -->
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>3.2.5</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<includes>
<include>**/*IT.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
<!-- PIT mutation testing -->
<plugin>
<groupId>org.pitest</groupId>
<artifactId>pitest-maven</artifactId>
<version>${pitest.version}</version>
<dependencies>
<dependency>
<groupId>org.pitest</groupId>
<artifactId>pitest-junit5-plugin</artifactId>
<version>${pitest.junit5.version}</version>
</dependency>
</dependencies>
<configuration>
<!-- Mutate only your app packages -->
<targetClasses>
<param>org.acme.*</param>
</targetClasses>
<!-- And the tests that cover them -->
<targetTests>
<param>org.acme.*Test</param>
</targetTests>
<!-- Stronger default set of mutators -->
<mutators>
<mutator>STRONGER</mutator>
</mutators>
<!-- Make HTML path stable -->
<timestampedReports>false</timestampedReports>
<!-- Speed/robustness knobs you can tune later -->
<threads>4</threads>
<timeoutConstant>4000</timeoutConstant>
<outputFormats>
<param>HTML</param>
<param>XML</param>
</outputFormats>
</configuration>
</plugin>
</plugins>
</build>
</project>
Why the key lines matter:
pitest-junit5-plugin: enables PIT to run JUnit 5 tests.targetClasses/targetTests: avoid mutating test utilities or external code.STRONGER: applies a broad set of sensible mutations. You can start withDEFAULTSand increase later.timestampedReports=false: makes the report path predictable for CI.
Core implementation
Create a small service with an intentional edge case: boundary handling for “loyalty points.”
src/main/java/org/acme/DiscountService.java
package org.acme;
package org.acme;
import java.math.BigDecimal;
import java.math.RoundingMode;
/**
* Applies a discount based on loyalty points:
* - < 100 points: 0%
* - 100..499 points: 5%
* - 500..999 points: 10%
* - >= 1000 points: 15%
*
* Business rule: discount applies to subtotal only. Never returns negative
* numbers.
*/
public class DiscountService {
public BigDecimal applyDiscount(BigDecimal subtotal, int loyaltyPoints) {
if (subtotal == null)
throw new IllegalArgumentException("subtotal is required");
if (subtotal.signum() < 0)
throw new IllegalArgumentException("subtotal must be >= 0");
BigDecimal rate = discountRate(loyaltyPoints);
BigDecimal discounted = subtotal.multiply(BigDecimal.ONE.subtract(rate));
// Round to cents using banker’s rounding
return discounted.setScale(2, RoundingMode.HALF_EVEN);
}
BigDecimal discountRate(int loyaltyPoints) {
if (loyaltyPoints < 100)
return BigDecimal.ZERO;
if (loyaltyPoints < 500)
return bd("0.05");
if (loyaltyPoints < 1000)
return bd("0.10");
return bd("0.15");
}
private static BigDecimal bd(String s) {
return new BigDecimal(s);
}
}
Now write a good but incomplete test that misses a boundary:
src/test/java/org/acme/DiscountServiceTest.java
package org.acme;
package org.acme;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
import java.math.BigDecimal;
import org.junit.jupiter.api.Test;
class DiscountServiceTest {
private final DiscountService svc = new DiscountService();
@Test
void zeroDiscountUnder100Points() {
BigDecimal out = svc.applyDiscount(new BigDecimal("100.00"), 50);
assertEquals(new BigDecimal("100.00"), out);
}
@Test
void fivePercentAt200Points() {
BigDecimal out = svc.applyDiscount(new BigDecimal("200.00"), 200);
assertEquals(new BigDecimal("190.00"), out);
}
@Test
void tenPercentAt700Points() {
BigDecimal out = svc.applyDiscount(new BigDecimal("40.00"), 700);
assertEquals(new BigDecimal("36.00"), out);
}
@Test
void fifteenPercentAt1200Points() {
BigDecimal out = svc.applyDiscount(new BigDecimal("100.00"), 1200);
assertEquals(new BigDecimal("85.00"), out);
}
@Test
void validateInputs() {
assertThrows(IllegalArgumentException.class, () -> svc.applyDiscount(null, 0));
assertThrows(IllegalArgumentException.class, () -> svc.applyDiscount(new BigDecimal("-0.01"), 0));
}
// MISSING: explicit boundary tests at 100, 500, 1000
}
We never check exactly 100, 500, or 1000 points. This gap will matter.
Run tests and mutation tests
First, run your regular tests:
mvn -q test
Now run PIT:
mvn -q org.pitest:pitest-maven:mutationCoverage
Expected output:
PIT compiles and instruments your classes.
It creates a report under
target/pit-reports/index.html.The console shows a summary including “mutation score.”
Open the HTML report in a browser:
What you see:
A project-level mutation score (e.g., 75–95% depending on your machine).
Per-class breakdown. Click
org.acme.DiscountService.A source view with line-level annotations. Survived mutants are highlighted.
Common early result:
- One or more mutants around the
discountRatethresholds survive. For example, PIT may flip a comparison from< 100to<= 100or from< 500to<= 500. Because we never assert the exact boundaries, tests still pass. That’s the point.
Interpreting the results
PIT groups mutations by outcome:
Killed : Tests failed when code was mutated. Good. Your tests are sensitive.
Survived : Tests still passed. Bad. A behavior change went unnoticed.
No coverage : Mutated line never ran. Improve your test’s execution path.
Timed out : Mutated code likely created an infinite loop or slow path. Investigate.
Memory error / run error : Fix flaky tests or tune PIT’s settings.
Equivalent mutant : Behavior didn’t change in a way tests can detect. These happen and are usually rare with STRONGER mutators.
What to change:
Target survived first. They usually indicate weak assertions or missing edge cases.
Add boundary checks for comparisons and tiered rules.
Strengthen assertions. Avoid “it didn’t throw” style unless that’s the behavior.
Kill the surviving mutants
Add missing boundary tests:
src/test/java/org/acme/DiscountServiceBoundaryTest.java
package org.acme;
import static org.junit.jupiter.api.Assertions.assertEquals;
import java.math.BigDecimal;
import org.junit.jupiter.api.Test;
class DiscountServiceBoundaryTest {
private final DiscountService svc = new DiscountService();
@Test
void exactly100PointsGetsFivePercent() {
BigDecimal out = svc.applyDiscount(new BigDecimal("100.00"), 100);
assertEquals(new BigDecimal("95.00"), out);
}
@Test
void exactly500PointsGetsTenPercent() {
BigDecimal out = svc.applyDiscount(new BigDecimal("100.00"), 500);
assertEquals(new BigDecimal("90.00"), out);
}
@Test
void exactly1000PointsGetsFifteenPercent() {
BigDecimal out = svc.applyDiscount(new BigDecimal("100.00"), 1000);
assertEquals(new BigDecimal("85.00"), out);
}
}
Re-run PIT:
mvn -q org.pitest:pitest-maven:mutationCoverage
Open target/pit-reports/index.html again.
The mutants that previously survived around < vs <= should now be killed , and your overall mutation score improves.
Production notes, performance, and CI
Scope it. Start with core business logic packages. Configure
targetClassesnarrowly to keep runs fast.Speed. Increase
<threads>in the plugin, but watch CPU contention in CI. Mutation testing is CPU heavy.Flakiness. Flaky tests become very visible. Fix them before you trust mutation scores.
Build time. Don’t run PIT on every push. Add a CI job that runs:
Native builds. PIT works at the bytecode level of JVM tests. You don’t need a Quarkus native image here.
Quarkus tests. You can run PIT with Quarkus unit tests. For heavyweight integration tests (
*IT), keep them out of PIT by naming and includes/excludes, or move business logic behind thin adapters and test that logic as plain unit tests.Quality gates. Enforce a minimum mutation score for critical modules. Keep it realistic. For example, 70–80% for complex codebases is a solid start.
What-ifs and variations
Mutator sets : Start with
DEFAULTS, then trySTRONGER. You can also specify explicit mutators likeCONDITIONALS_BOUNDARY,MATH,INCREMENTS.Selective packages : Mutate only
org.acme.xxx.*first. Add more packages over time.Test styles : PIT works with plain JUnit 5, parameterized tests, and property-based tests alike.
Build profiles : Create a Maven profile
mutationthat binds the PIT goal, so CI can runmvn -Pmutation org.pitest:pitest-maven:mutationCoverage.
Where to go next
You’ve only looked at the basics. There’s room for improvement. Here are some ideas where to look next:
Apply PIT to the packages that contain your business rules first.
Add a lightweight CI job to publish the HTML report as a build artifact.
Use mutation testing sparingly on code that wraps frameworks and I/O. Focus on your logic.
In the end, better tests are the ones that actually catch bugs, and mutation testing is the discipline that keeps those tests honest.
Top comments (0)