Why Maven Build Cache Still Re-Runs Your Entire Test Suite on One-Line Edits

You install maven-build-cache-extension, your unchanged modules start restoring from cache in seconds, and PR builds feel fast. Then you edit one line in a utility class inside your biggest module and the build runs every test in that module again.

That is not a bug in the cache. It is a granularity mismatch, and it is worth understanding precisely, because the two tools solve different halves of the problem and work well together.

What the build cache does

The extension fingerprints a module's inputs (sources, dependencies, plugin configuration) into a checksum. If it has seen that checksum before, it restores the module's outputs and skips the build. If it has not, it runs the module's whole lifecycle, Surefire included.

So the cache's smallest unit is the module. Anything inside it that changes, even a single character in one class, produces a new checksum for the module. There is no "mostly the same" in a checksum.

For a repository with many small modules that is a great trade. For a repository with one large module (a monolith, a core or common module, a service with its tests in the same artifact), it means a one-line edit costs the module's entire test suite.

Measuring it

I built a small fixture rather than assert this. It is deliberately simple and synthetic, so read the numbers as an illustration of the mechanism, not a benchmark:

• One Maven module, JDK 25 toolchain, Maven 3.9.12, maven-build-cache-extension 1.2.0, Surefire 3.5.2, JUnit 5.10.2.
• Six small utility classes in src/main.
• 60 JUnit 5 test classes, 2 tests each: 120 tests. Each test class exercises exactly one utility class, so every utility has 10 test classes (20 tests) that depend on it.

Steps and results:

Step	What changed	Cache extension	Tests run
Base branch build	nothing	miss (first build)	120
Rebuild, same inputs	nothing	hit, module restored	0
Branch A	one line in StringUtil	miss	120

The unchanged rebuild is the cache doing its job. Branch A is the problem: StringUtil is exercised by 20 of the 120 tests, and all 120 ran.

The other half: which tests does this change actually reach?

Narrowing inside the module needs information the cache does not have: which tests touch which production classes. blastradius gets it from the JVM instead of guessing from names or packages:

1. Track. On the base branch it runs the suite once with a java.lang.instrument agent attached. Each time a class is loaded while a test runs, it records the pair (test, class). That gives a ground-truth map from every test to the production classes it loaded.
2. Diff. On a change branch it diffs the commit against the Git merge base to find the changed classes.
3. Select. A test is selected if one of its recorded classes changed, if the test is new or modified, if an executed class reaches a changed class through one or two recorded static invocation edges, or if a non-source change forces the conservative full-suite fallback.
4. Gate. The selection becomes a standard Surefire/Failsafe -Dtest= filter.

The same fixture, with blastradius added alongside the cache extension:

Step	What changed	Cache extension	Tests run
Base branch build (tracking)	nothing	miss	120
Branch B	one line in MathUtil	miss	20 of 120
Rebuild branch B, same inputs	nothing	hit, module restored	0

The cache still misses, because the module's checksum changed. That is correct, and blastradius does not try to change it. What changes is what the missed build runs: only the 20 tests, from the 10 classes whose recorded dependencies include MathUtil, instead of all 120 (83.3% skipped in this fixture).

The two are complementary:

• The cache removes whole modules that did not change.
• blastradius removes tests inside the modules that did.

Is this just a toy result?

The fixture is evenly partitioned on purpose, so a clean 20/120 is what you would expect. Real suites are messier: shared fixtures, base classes and widely used utilities make some changes reach many tests. That is why I would not quote the fixture number, and why the historical replay matters more.

blastradius has been replayed over 800 consecutive commits on four open-source projects (apache/shenyu, apache/httpcomponents-client, jhy/jsoup, apache/commons-io). Across those replays, the tool selected 966,368 of 2,501,105 test executions: 61.4% fewer, with the per-project range running from 45.0% (jsoup) to 71.0% (shenyu). A module-level cache has to treat each of those commits that touches a module's sources as a full rerun of that module. The replay figures are counts of test executions, not wall-clock CI time, and the full methodology, exclusions and caveats are in the first post in this series and in HISTORICAL_REPLAY_ANALYSIS.md.

Setup notes for using both

Two things I hit while building the fixture, so you do not have to:

1. The base-branch (tracking) build must actually run the tests. Blastradius records dependencies by running the suite in a forked Maven build. If the cache extension restores that build from cache, no tests execute, nothing is recorded, and blastradius reports tracking failed; ... falling back to the full suite. That fallback is safe (you get a full run), but you get no index. The fix that worked for me was to disable the cache for the tracking build with MAVEN_OPTS="-Dmaven.build.cache.enabled=false". Passing the flag only on the outer command line did not reach the forked build.
2. Persist the .blastradius/ directory between runs, the same way you persist the cache. On a change branch, blastradius needs the saved index from the base branch; without one it runs the full suite.

Example plugin configuration, as used in the fixture:

<plugin>
  <groupId>io.github.baokhang83.blastradius</groupId>
  <artifactId>blastradius-maven-plugin</artifactId>
  <version>0.4.0</version>
  <executions>
    <execution>
      <phase>process-test-classes</phase>
      <goals><goal>select</goal></goals>
    </execution>
  </executions>
  <configuration><baseRef>main</baseRef></configuration>
</plugin>

Where this does not help

• Modules that are already small. If your tests are split across many modules and each module's tests are fast, the cache is already close to optimal.
• Changes that reach everything. A change to a class loaded by every test selects nearly every test, and a build-file or resource change triggers the full-suite fallback by design.
• Code reached through reflection or string-dispatched APIs. Selection is sound by default, not guaranteed. Selector engines, expression languages and query parsers are the known hard case (jsoup showed this in the replay and drove a fix), so keep a regular full-suite run, daily is the recommendation, as a backstop.

FAQ

Why does Maven Build Cache re-run all tests after a one-line change?

The extension caches at module granularity. Any change to a module's inputs produces a new checksum, so the module is rebuilt in full, including all of its Surefire tests. The cache has no record of which tests depend on which classes.

Can the Maven Build Cache Extension skip individual tests?

Not by itself: it restores or rebuilds whole modules. To run only the tests a change can affect, you need per-test dependency information, which is what test impact analysis tools such as blastradius record.

How does blastradius know which tests a class change affects?

On the base branch a Java agent records every class each test loads at runtime. On a change branch, tests whose recorded classes include a changed class are selected, along with new or modified tests and a bounded set of tests that reach the changed class through one or two static invocation edges.

Do I have to choose between the cache and blastradius?

No. The cache skips unchanged modules; blastradius narrows tests inside changed ones. One setup note: run the base-branch tracking build with the cache disabled, otherwise the restored build runs no tests and nothing is recorded.

Is it safe to skip tests?

Selection is sound by default but not a guarantee. Reflection, generated code and string-dispatched APIs can escape it. Run the full suite on a regular cadence as a safety net; the recommendation is daily.

Does 83% fewer tests mean 83% faster builds?

No. The fixture counts tests run, not time, and it is synthetic. Real savings depend on which tests are slow, fixed costs such as compilation and packaging, and how widely your changes reach.

What do I need to try it?

Maven, JDK 21 or later, a Git repository and JUnit 5. The plugin is on Maven Central as io.github.baokhang83.blastradius:blastradius-maven-plugin. The fixture in this post is easy to recreate: six utility classes, 60 two-test classes with one per utility, and the two plugins configured as shown.

Disclosure: I am the author of blastradius.