DEV Community: Karl Heinz Marbaise

Maven 4 - Part I - Easier Versions

Karl Heinz Marbaise — Mon, 01 Apr 2024 11:34:14 +0000

Overview

This is the first article in this series about Apache Maven 4. Currently Apache Maven 4 is in alpha state (alpha-13). You can already download it and of course use it (I recommend to test things to see, if something strange happens. If you find problems, please report them) but I would not recommend to use it in production in the current stage.

Let us start with a basic example of a Maven POM file which looks similar like this (For the sake of clarity no dependencies defined):

<project
    xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

  <modelVersion>4.0.0</modelVersion>

  <parent>
    <groupId>com.soebes.smpp</groupId>
    <artifactId>smpp</artifactId>
    <version>6.0.5</version>
    <relativePath/>
  </parent>

  <groupId>com.soebes.examples.maven4</groupId>
  <artifactId>basic</artifactId>
  <version>1.0.0-SNAPSHOT</version>
  <name>Basic Example</name>
  ..
</project>

Let us take a deeper look into this. So we see a parent, which is used to inherit configuration setup for plugins etc. to prevent repetition for each project. The next parts are the usual definition of the groupId, artifactId and the version (this combination is often abbreviated with GAV). So we would like to build that setup with Maven 4. So first let us check, if we have installed the correct Maven version. The output should look similar to this:

$> mvn --version
Apache Maven 4.0.0-alpha-13 (0a6a5617fe5ef65c44f05903491e170d92cf37fc)
Maven home: /projects/tools/maven
Java version: 22, vendor: Oracle Corporation, runtime: /projects/.sdkman/candidates/java/22-open
Default locale: en_DE, platform encoding: UTF-8
OS name: "mac os x", version: "14.0", arch: "aarch64", family: "mac"

Ok, let us try to build that example project via:

mvn clean verify

and that will produce the following (a bit lengthy) output:

[INFO] Unable to find the root directory. Create a .mvn directory in the root directory or add the root="true" attribute on the root project's model to identify it.
[INFO] Scanning for projects...
[INFO] 
[INFO] -------------------------------------------< com.soebes.examples.maven4:basic >-------------------------------------------
[INFO] Building Basic Example 1.0.0-SNAPSHOT
[INFO]   from pom.xml
[INFO] ---------------------------------------------------------[ jar ]----------------------------------------------------------
[INFO] 
[INFO] --- clean:3.3.2:clean (default-clean) @ basic ---
[INFO] Deleting /projects/basic/target
[INFO] 
[INFO] --- enforcer:3.4.1:enforce (enforce-maven) @ basic ---
[INFO] Rule 0: org.apache.maven.enforcer.rules.RequireSameVersions passed
[INFO] Rule 1: org.apache.maven.enforcer.rules.version.RequireMavenVersion passed
[INFO] Rule 2: org.apache.maven.enforcer.rules.dependency.BannedDependencies passed
[INFO] Rule 3: org.apache.maven.enforcer.rules.RequireNoRepositories passed
[INFO] Rule 4: org.apache.maven.enforcer.rules.RequirePluginVersions passed
[INFO] Rule 5: org.apache.maven.enforcer.rules.property.RequireProperty passed
[INFO] Rule 6: org.apache.maven.enforcer.rules.property.RequireProperty passed
[INFO] Rule 7: org.apache.maven.enforcer.rules.property.RequireProperty passed
[INFO] 
[INFO] --- jacoco:0.8.11:prepare-agent (default) @ basic ---
[INFO] argLine set to -javaagent:/.m2/repository/org/jacoco/org.jacoco.agent/0.8.11/org.jacoco.agent-0.8.11-runtime.jar=destfile=/projects/basic/target/jacoco.exec
[INFO] 
[INFO] --- resources:3.3.1:resources (default-resources) @ basic ---
[INFO] skip non existing resourceDirectory /projects/basic/src/main/resources
[INFO] skip non existing resourceDirectory /projects/basic/src/main/resources-filtered
[INFO] 
[INFO] --- compiler:3.12.1:compile (default-compile) @ basic ---
[INFO] Recompiling the module because of changed source code.
[INFO] Compiling 1 source file with javac [debug release 11] to target/classes
[INFO] 
[INFO] --- resources:3.3.1:testResources (default-testResources) @ basic ---
[INFO] skip non existing resourceDirectory /projects/basic/src/test/resources
[INFO] skip non existing resourceDirectory /projects/basic/src/test/resources-filtered
[INFO] 
[INFO] --- compiler:3.12.1:testCompile (default-testCompile) @ basic ---
[INFO] No sources to compile
[INFO] 
[INFO] --- surefire:3.2.3:test (default-test) @ basic ---
[INFO] No tests to run.
[INFO] 
[INFO] --- jar:3.3.0:jar (default-jar) @ basic ---
[INFO] Building jar: /projects/basic/target/basic-1.0-SNAPSHOT.jar
[INFO] 
[INFO] --- site:3.12.1:attach-descriptor (attach-descriptor) @ basic ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO] 
[INFO] --- jacoco:0.8.11:report (default) @ basic ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] Copying com.soebes.examples.maven4:basic:pom:1.0-SNAPSHOT to project local repository
[INFO] Copying com.soebes.examples.maven4:basic:jar:1.0-SNAPSHOT to project local repository
[INFO] Copying com.soebes.examples.maven4:basic:pom:consumer:1.0-SNAPSHOT to project local repository
[INFO] --------------------------------------------------------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] --------------------------------------------------------------------------------------------------------------------------
[INFO] Total time:  1.199 s
[INFO] Finished at: 2024-03-26T23:20:12+01:00
[INFO] --------------------------------------------------------------------------------------------------------------------------

If you have already have used Maven 3.X before, you might have spotted some differences. The first line shows:

[INFO] Unable to find the root directory. Create a .mvn directory in the root directory or add the root="true" attribute on the root project's model to identify it.
.

We will ignore that for now, because we will discuss that later in detail. Let's talk about versions.

SNAPSHOT vs. Release Version

In the initial example, you have seen, that we have used a literal version 1.0.0-SNAPSHOT, which is very common in Maven projects. The usual process is to start with a SNAPSHOT-version which is implied by the postfix -SNAPSHOT. This is the indicator, that it is not final yet or in other words, it will change. But, of course, there is a time to make that version final (immutable). Being more accurate, finalize the state of the software (artifact), which is indicated by the version. So we change the version to 1.0.0 (without the postfix -SNAPSHOT), which is now called a release version.
This release version will be published in whatever manner (often in central repository, or in an internal repository). The next development cycle starts by changing the version from 1.0.0 to something like 1.1.0-SNAPSHOT. Now the Circle starts from the beginning. So changing the version in the pom.xml everytime is something, which some people don't like and yes it's a bit cumbersome. That version change can also be handled by using the Maven Release Plugin, which is also not really liked by some people.

Easier Revisions

Starting with Maven 4, you can define a version property simply in the pom.xml like the following:

...
  <groupId>com.soebes.examples.maven4</groupId>
  <artifactId>basic</artifactId>
  <version>${revision}</version>
  <name>Basic Example</name>
...

For brevity reasons, only the relevant parts are being shown. If you build that via: mvn clean you will see the following:

..
[INFO] 
[INFO] -------------------------------------------< com.soebes.examples.maven4:basic >-------------------------------------------
[INFO] Building Basic Example ${revision}
[INFO]   from pom.xml
[INFO] ---------------------------------------------------------[ jar ]----------------------------------------------------------
[INFO] 
[INFO] --- clean:3.3.2:clean (default-clean) @ basic ---
[INFO] Deleting /projects/basic-revision/target
[INFO] --------------------------------------------------------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] --------------------------------------------------------------------------------------------------------------------------
[INFO] Total time:  0.242 s
[INFO] Finished at: 2024-03-26T23:22:12+01:00
[INFO] --------------------------------------------------------------------------------------------------------------------------

Command Line Parameter

So it's a bit weird that you see Building Basic Example ${revision}. That means, this project does not have a version at all. The version is ${revision} which I assume is not, what you wanted nor expected. How can we define a particular version for our build (or for the resulting artifact)? This can be achieved by using a property via the command line like this:

$ mvn clean -Drevision=1.2.0-SNAPSHOT

The output during building now changes into this:

...
[INFO] 
[INFO] -------------------------------------------< com.soebes.examples.maven4:basic >-------------------------------------------
[INFO] Building Basic Example 1.2.0-SNAPSHOT
[INFO]   from pom.xml
[INFO] ---------------------------------------------------------[ jar ]----------------------------------------------------------
[INFO] 
[INFO] --- clean:3.3.2:clean (default-clean) @ basic ---
[INFO] Deleting /projects/basic-revision/target
[INFO] --------------------------------------------------------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] --------------------------------------------------------------------------------------------------------------------------
[INFO] Total time:  0.243 s
[INFO] Finished at: 2024-03-26T23:21:12+01:00
[INFO] --------------------------------------------------------------------------------------------------------------------------

The output Building Basic Example 1.2.0-SNAPSHOT now indicates, that the given version via property is being used as the version for the Maven project or more accurate for the resulting artifacts.

POM Property

This means, we can define any version we like, by just adding that option on command line every time we execute Maven. Hold on a second. Everytime? Yes everytime hm. Isn't there a way to avoid that? Yes, there is one. We can define the revision property in the pom.xml. That looks like this:

  ..
  <groupId>com.soebes.examples.maven4</groupId>
  <artifactId>basic</artifactId>
  <version>${revision}</version>
  <name>Basic Example</name>

  <properties>
    <revision>1.0.0-SNAPSHOT</revision>
  </properties>
..

Ok, now you can simply build like before without giving the revision everytime on command as an option:

$> mvn clean
..
[INFO] Scanning for projects...
[INFO] 
[INFO] -------------------------------------------< com.soebes.examples.maven4:basic >-------------------------------------------
[INFO] Building Basic Example 1.0.0-SNAPSHOT
[INFO]   from pom.xml
[INFO] ---------------------------------------------------------[ jar ]----------------------------------------------------------
[INFO] 
[INFO] --- clean:3.3.2:clean (default-clean) @ basic ---
[INFO] Deleting /projects/basic-revision-pom/target
[INFO] --------------------------------------------------------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] --------------------------------------------------------------------------------------------------------------------------
[INFO] Total time:  0.238 s
[INFO] Finished at: 2024-03-26T12:15:42+01:00
[INFO] --------------------------------------------------------------------------------------------------------------------------

One could argue: Haven't we reached the same level as in the beginning, while defining the version literally in the pom? On the first glance it might look like this, but we have reached a different level of flexibility. Ok, let's try to build with a different version? Do we have to change the pom.xml file? No, there is no need for that anymore. You simply give a different version via command line mvn clean -Drevision=1.2.0-SNAPSHOT:

..
[INFO] 
[INFO] -------------------------------------------< com.soebes.examples.maven4:basic >-------------------------------------------
[INFO] Building Basic Example 1.2.0-SNAPSHOT
[INFO]   from pom.xml
[INFO] ---------------------------------------------------------[ jar ]----------------------------------------------------------
[INFO] 
[INFO] --- clean:3.3.2:clean (default-clean) @ basic ---
[INFO] Deleting /projects/basic-revision-pom/target
[INFO] --------------------------------------------------------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] --------------------------------------------------------------------------------------------------------------------------
[INFO] Total time:  0.239 s
[INFO] Finished at: 2024-03-26T12:21:16+01:00
[INFO] --------------------------------------------------------------------------------------------------------------------------

That means, we can now overwrite the version within the pom.xml easily by giving any version we like, via the property on the command line. This is very helpful within an CI/CD tools (for example Jenkins, Circle CI, Github Actions or alike). That also prevents the manual change of the pom.xml everytime, you would like to change version.

Creating a Release

Also creating a release is very easy. Just use mvn deploy -Drevision=1.2.0. Ok, you have to have done the required setup to publish a release, but that's a different story.

The output looks similar to the following (removed the beginning of the output, because it's already shown in previous examples):

...
[INFO] 
[INFO] --- install:3.1.1:install (default-install) @ basic ---
[INFO] Deferring install for com.soebes.examples.maven4:basic:1.2.0 at end
[INFO] Installing /projects/basic-distro/target/basic-1.2.0.jar to /.m2/repository/com/soebes/examples/maven4/basic/1.2.0/basic-1.2.0.jar
[INFO] Installing /projects/basic-distro/pom.xml to /.m2/repository/com/soebes/examples/maven4/basic/1.2.0/basic-1.2.0-build.pom
[INFO] Installing /projects/basic-distro/target/consumer-4964754963249724515.pom to /.m2/repository/com/soebes/examples/maven4/basic/1.2.0/basic-1.2.0.pom
[INFO] 
[INFO] --- deploy:3.1.1:deploy (default-deploy) @ basic ---
Uploading to releases: http://localhost:8081/nexus/content/repositories/releases/com/soebes/examples/maven4/basic/1.2.0/basic-1.2.0.jar
Uploaded to releases: http://localhost:8081/nexus/content/repositories/releases/com/soebes/examples/maven4/basic/1.2.0/basic-1.2.0.jar (3.1 kB at 33 kB/s)
Uploading to releases: http://localhost:8081/nexus/content/repositories/releases/com/soebes/examples/maven4/basic/1.2.0/basic-1.2.0-build.pom
Uploading to releases: http://localhost:8081/nexus/content/repositories/releases/com/soebes/examples/maven4/basic/1.2.0/basic-1.2.0.pom
Uploaded to releases: http://localhost:8081/nexus/content/repositories/releases/com/soebes/examples/maven4/basic/1.2.0/basic-1.2.0-build.pom (969 B at 88 kB/s)
Uploaded to releases: http://localhost:8081/nexus/content/repositories/releases/com/soebes/examples/maven4/basic/1.2.0/basic-1.2.0.pom (2.2 kB at 199 kB/s)
Downloading from releases: http://localhost:8081/nexus/content/repositories/releases/com/soebes/examples/maven4/basic/maven-metadata.xml
Uploading to releases: http://localhost:8081/nexus/content/repositories/releases/com/soebes/examples/maven4/basic/maven-metadata.xml
Uploaded to releases: http://localhost:8081/nexus/content/repositories/releases/com/soebes/examples/maven4/basic/maven-metadata.xml (530 B at 48 kB/s)
[INFO] Copying com.soebes.examples.maven4:basic:pom:1.2.0 to project local repository
[INFO] Copying com.soebes.examples.maven4:basic:jar:1.2.0 to project local repository
[INFO] Copying com.soebes.examples.maven4:basic:pom:consumer:1.2.0 to project local repository
[INFO] --------------------------------------------------------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] --------------------------------------------------------------------------------------------------------------------------
[INFO] Total time:  1.367 s
[INFO] Finished at: 2024-03-26T23:50:23+01:00
[INFO] --------------------------------------------------------------------------------------------------------------------------

Multi Module Builds

Ok, now lets make a step further to a more complex setup, a multi-module-build. This kind of build comprises of a structure like this:

.
|-- pom.xml (1)
|-- domain
|   |-- pom.xml
|   `-- src
|-- service
|   |-- pom.xml
|   `-- src
|-- service-client
|   |-- pom.xml
|   `-- src
`-- webgui
|-- pom.xml
`-- src

A multi-module build is characterized by the fact, that the child modules (for example domain or service) always have a relation to the parent module (parent marked with (1)). The parent pom looks like this:

<project...>

  <modelVersion>4.0.0</modelVersion>

  <parent>
    <groupId>com.soebes.smpp</groupId>
    <artifactId>smpp</artifactId>
    <version>6.0.5</version>
    <relativePath/>
  </parent>

  <groupId>com.soebes.examples.j2ee</groupId>
  <artifactId>jee-parent</artifactId>
  <version>3.1.4-SNAPSHOT</version>
  <packaging>pom</packaging>
  ...
  <modules>
    ..
    <module>webgui</module>
    <module>domain</module>
    <module>service</module>
    <module>service-client</module>
    ..
  </modules>

</project>

It is important to mention, that the list of modules referenced via <modules>...</modules> will create the relationship to the child modules. A child pom looks like this:

<project....>

  <modelVersion>4.0.0</modelVersion>

  <parent>
    <groupId>com.soebes.examples.j2ee</groupId>
    <artifactId>jee-parent</artifactId>
    <version>3.1.4-SNAPSHOT</version>
  </parent>

  <artifactId>domain</artifactId>
  ...
</project>

It is crucial, that the specification parent refers exactly to the parent (referenced with (1)) in the multi-module build. Such a multi-module build may consist of several hundred or even thousands of child modules. This is then often divided into several sub-levels (similar to a directory tree). Theoretically, there is no limit here (only memory etc.). If you now imagine, that you have to change the version number, this leads as described at the beginning, that all pom.xml files have to be changed. However, this can be avoided by using a ${revision} property approach and simplified dramatically.

This results, that the parent POM receives a corresponding property and the child modules as well. Here the parent pom:

<project ...>

  <modelVersion>4.0.0</modelVersion>

  <parent>
    <groupId>com.soebes.smpp</groupId>
    <artifactId>smpp</artifactId>
    <version>6.0.5</version>
    <relativePath/>
  </parent>

  <groupId>com.soebes.examples.j2ee</groupId>
  <artifactId>jee-parent</artifactId>
  <version>${revision}</version>
  <packaging>pom</packaging>

  <properties>
    <revision>1.0.0-SNAPSHOT</revision>
  </properties>

  ...
  <modules>
    ..
    <module>webgui</module>
    <module>domain</module>
    <module>service</module>
    <module>service-client</module>
    ..
  </modules>
</project>

And exemplified by a child module (domain):

<project ...>

  <modelVersion>4.0.0</modelVersion>

  <parent>
    <groupId>com.soebes.examples.j2ee</groupId>
    <artifactId>jee-parent</artifactId>
    <version>${revision}</version>
  </parent>
  ..
  <artifactId>domain</artifactId>
  ...
</project>

Different Properties

Some will ask themself, could we use other properties than the used revision in the previous examples? Technically there are three properties available revision (already mentioned), sha1 and changelist. The usage and rules are the same as for revision. So you can create a combination like this <version>${revision}${sha1}${changelist}</version>. My recommendation is, use only a single one and that is the revision property. In the end it's your decision, which way you go with that. But to make things clear, you can NOT use other properties except the mentioned revision, sha1 and changelist.

Maven Configuration File

In contradiction to the previous approach, you can define the ${revision} in a different manner. You have to create a directory .mvn (in the root of your project) and put a file named maven.config into that directory, which contains the following: -Drevision=1.0.0-SNAPSHOT.

.
|-- .mvn
|-- pom.xml
|-- domain
|   |-- pom.xml
|   `-- src
|-- service
|   |-- pom.xml
|   `-- src
|-- service-client
|   |-- pom.xml
|   `-- src
`-- webgui
|-- pom.xml
`-- src

This is the equivalent of the previously mentioned command line approach. All things in maven.config will be added to the command line of Maven, as it would be given directly. The option for external configuration files exists since Maven 3.3.1 (ca. 9 years). The maven.config can contain more options for example -T 3.

Conclusion

The usage of the revision property (also the other two), is simplifying the versioning of artifacts. In consequence the release creation is simplified even without changing the pom.xml file(s). The most important part in relationship with Maven 4 is, that this works out-of-the-box. It not necessary to configure or use the flatten-maven-plugin anymore, as it was necessary with CI Friendly in Maven 3. This also means, that the use of e.g. version-maven-plugin to update the version in the pom.xml file, is not needed anymore. One of the things missing here, is the support of the maven-release-plugin creating tags in Git during the release creation, but that can be accomplished by adding some steps in your CI/CD pipeline.

Originally posted https://blog.soebes.io/posts/2024/03/2024-03-31-maven-4-part-i/

JDK22 - Gatherer

Karl Heinz Marbaise — Sun, 07 Jan 2024 00:04:08 +0000

Overview

If you take a look into JDK-22, you will find a very interesting thing, which is called 461: Stream Gatherers (Preview). Those Gatherers are a way to enhance the Stream API, which is existing in the JDK since 2014 (JDK 8). The collectors have been enhanced over the time (You can look up the differences via Javaalmanac check for Collectors). The provided collectors in the JDK already cover a lot of things, but sometimes there are situations, where it's not enough or not flexible enough or would produce code, which is hard to read (more accurate hard to understand). The first thought could be to request an enhancement of the Collectors in the JDK itself, but that would mean to add more methods on the collectors which already has 44 methods (If I have counted correctly). Apart from having a problem, which is so specific so it's worth to add that to the JDK itself. So it might be a better solution to give the users a way to enhance the Stream API based on the their own needs. That is also the summary of the JEP 461:

Enhance the Stream API to support custom intermediate operations.
This will allow stream pipelines to transform data in ways that are not easily achievable
with the existing built-in intermediate operations. This is a preview API.

Let us start with a simple example. We would like to group by a list of strings based on the length of the strings itself.
That can easily being expressed with the existing Stream API and collectors like this:

@Test
void usingGroupingBy() {
  var result = Stream.of(
          "123456", "foo", "bar", "baz", 
          "quux", "anton", "egon", "banton")
      .collect(Collectors.groupingBy(String::length));
  System.out.println("result = " + result);
}

The output of that is:

result = {3=[foo, bar, baz], 4=[quux, egon], 5=[anton], 6=[123456, banton]}

So nothing fancy here. Another example using things like distinct() in a stream. That can look like this:

@Test
void exampleDistinctWithNumbers() {
  var integers = List.of(1, 10, 11, 10, 11);
  var result = integers.stream().distinct().toList();
  System.out.println("result = " + result);
}

And the output of the above:

result = [1, 10, 11]

So let us combine those two ideas and create a distinct based on the length. Unfortunately there is no way to change
distinct because it does not have a parameter at all. Only a single implementation. Ok, there is a way to achieve that, which could look like this (Yes that code is taken from the JEP 461):

record DistinctByLength(String str) {
  @Override public boolean equals(Object obj) {
    return obj instanceof DistinctByLength(String other)
           && str.length() == other.length();
  }

  @Override public int hashCode() {
    return str == null ? 0 : Integer.hashCode(str.length());
  }
}

@Test
void example_one() {
  var result = Stream.of(
          "123456", "foo", "bar", "baz", 
          "quux", "anton", "egon", "banton")
      .map(DistinctByLength::new)
      .distinct()
      .map(DistinctByLength::str)
      .toList();
  System.out.println("result = " + result);
}

and the output:

result = [123456, foo, quux, anton]

So to achieve the goal, we have to do a mapping .map(DistinctByLength::new) first and after that .distinct() and then map back into the string .map(DistinctByLength::str) ok works! Is that maintainable? It might be. Flexibility? The result shows, that the first reached element is emitted in the result. What if I like to have the last one? Or the second one if more than two elements are existing? Wouldn't it be easier if you could write the code like this:

var streamList = List.of(
    "123456", "foo", "bar", "baz",
    "quux", "anton", "egon", "banton");

var result = streamList
    .stream()
    .distinctBy(String::length)
    .toList();

That would be great, but the JEP 461 comes quite near to that. That means, you have to write the code like this:

var streamList = List.of(
    "123456", "foo", "bar", "baz",
    "quux", "anton", "egon", "banton");

var result = streamList
    .stream()
    .gather(distinctBy(String::length))
    .toList();

So finally we have to implement the distinctBy thing here. So how will that work? Starting by taking a look into the .gather(..) method. You can see, that the parameter uses the interface Gatherer<T, A, R>.. which in general comprises of four parts. The initializer, integrator, combiner and the finisher.

Integrator

Let us start with the integrator. This part is called every
time an element from the stream is put through the stream, which means the integrator will see every element of the stream. Based on the implementation it can do anything with the element. For a better understanding we start with an implementation of a mapping, which does nothing. Let us call it mapNoOp (You might already have implied by the name: Map No Operation). That could be used like this:

@Test
void noOperation_withGathererOf() {
  var integerList = List.of(1, 2, 3, 4, 5, 6, 7, 8);

  var resultList = integerList.stream()
      .gather(Gatherer.of(mapNoOp))
      .toList();
  System.out.println("resultList = " + resultList);
}

This could also being achieved based on the existing collectors like this:

@Test
void noOperationMapping() {
  var integerList = List.of(1, 2, 3, 4, 5, 6, 7, 8);

  var resultList = integerList.stream()
      .map(Function.identity())
      .toList();
  System.out.println("resultList = " + resultList);
}

Now let us take a look how the implementation based on the Gatherer looks like:

static final Gatherer.Integrator<Void, Integer, Integer> mapNoOp =
    (state, element, downstream) -> {
      downstream.push(element);
      return true;
    };

So for this, we only have to implement an integrator. The integrator will consume the elements via element and emits the elements via downstream.push(element). The return true here is responsible to tell the stream to consume more elements or don't continue to consume more elements via return false. Currently the state is not used at all here
(I have kept the state for better understanding. It can be replaced with the underscore (_) to make it clear that it is not used see JEP-456). So to use it, as already stated before, you have to use Gatherer.of(mapNoOp). This can be made a bit more convenient:

static <T> Gatherer<T, ?, T> mapNoOp() {
  Gatherer.Integrator<Void, T, T> integrator = (_, element, downstream) -> {
    downstream.push(element);
    return true;
  };
  return Gatherer.ofSequential(integrator);
}

That also makes it easier to generalize it. So now you can use very simple:

@Test
void noOperation_Integration() {
  var integerList = List.of(1, 2, 3, 4, 5, 6, 7, 8);

  var resultList = integerList.stream()
      .gather(mapNoOp())
      .toList();
  System.out.println("resultList = " + resultList);
}

So based on that, it is no problem to use it with other types:

@Test
void noOperation_IntegrationDifferentType() {
  var integerList = List.of("1", "2", "3", "4", "5", "6", "7", "8");

  var resultList = integerList.stream()
      .gather(mapNoOp())
      .toList();
  System.out.println("resultList = " + resultList);
}

That has become very convenient to be used.

Initializer and Finisher

So back to our initial idea. If reconsidering the usage of groupingBy and the distinct examples at the beginning. The groupingBy requires something internally to store intermediate results before emitting the final result. Also the distinct part needs something to remember, which elements already have gone through the stream and already existed. So that's when the initializer comes into the play. Or the state within the integrator becomes important. So this thing to remember is exactly the intention of the initializer, which creates such memory while the state is the access to that memory. The implementation looks like this:

static <T, A> Gatherer<T, ?, T> distinctBy(Function<? super T, ? extends A> classifier) {
  Supplier<HashMap<A, List<T>>> initializer = HashMap::new;
  //
  Gatherer.Integrator<HashMap<A, List<T>>, T, T> integrator = (state, element, downstream) -> {
    A apply = classifier.apply(element);
    state.computeIfAbsent(apply, (_) -> new ArrayList<>()).add(element);
    return true;
  };
  //
  return Gatherer.ofSequential(initializer, integrator);
}

So what do we do here? Based on the initializer we create a memory (HashMap), which stores the elements based on the key classifier. The classifier is something like String::length. So we have a HashMap<Integer,List<String>>. This will be filled during the iteration through the stream in the integrator. First apply the classifier (which results into the length) and adds the element (String) to the list in the HashMap. If you take a deep look into the above implementation you might spotted the issue. It will consume all elements form the stream, but it will never publish any element. So we have to think under which circumstance we want to emit elements? If the HashMap contains an element where one entry is in there or expressed in code:

static <T, A> Gatherer<T, ?, T> distinctBy(Function<? super T, ? extends A> classifier) {
  Supplier<HashMap<A, List<T>>> initializer = HashMap::new;
  //
  Gatherer.Integrator<HashMap<A, List<T>>, T, T> integrator = (state, element, downstream) -> {
    A apply = classifier.apply(element);
    state.computeIfAbsent(apply, (_) -> new ArrayList<>()).add(element);
    if (state.get(apply).size() == 1) {
      downstream.push(element);
    }
    return true;
  };
  //
  return Gatherer.ofSequential(initializer, integrator);
}

The usage looks like this:

@Test
void usingDistinctByExample() {
  var streamList = List.of(
      "123456", "foo", "bar", "baz",
      "quux", "anton", "egon", "banton");

  var result = streamList
      .stream()
      .gather(distinctBy(String::length))
      .toList();

  System.out.println("result = " + result);
}

and the output is this:

result = [123456, foo, quux, anton]

So that means original requirement solved. Go to the next requirement. Emit the last element of appearance. That means, we must be sure the whole stream has been done or in other words all elements haven been seen. That is not solvable only with the integrator for that you need the finisher.

Finisher

The finisher is called after the whole stream is done and before the stream ends so to speak. The finisher is defined as BiConsumer<A, Downstream<? super R>> finisher. That means in other words, the first part is the state and of course downstream which gives us the opportunity to emit elements in it's final state. That means also all elements of the stream have gone through the integrator.

static <T, A> Gatherer<T, ?, T> distinctBy(Function<? super T, ? extends A> classifier) {
  Supplier<HashMap<A, List<T>>> initializer = HashMap::new;
  //
  Gatherer.Integrator<HashMap<A, List<T>>, T, T> integrator = (state, element, downstream) -> {
    A apply = classifier.apply(element);
    state.computeIfAbsent(apply, (_) -> new ArrayList<>()).add(element);
    return true;
  };
  //
  BiConsumer<HashMap<A, List<T>>, Gatherer.Downstream<? super T>> finisher = (state, downstream) -> {
    state.forEach((_, value) -> downstream.push(value.getLast()));
  };
  //
  return Gatherer.ofSequential(initializer, integrator, finisher);
}

So the above definition can now be used as before with the only difference it will emit the last elements and not the first which looks like this:

result = [baz, egon, anton, banton]

Also the other requirement I have expressed (the second element etc.) can be easily solved via the finisher. I leave that as an exercise for the gentle readers.

Combiner

So let us come to the last one. The combiner. This is responsible to combine (You guessed it ;-) different states of the state which can only happen if you run your gatherer in a parallel stream. As you might also already realized I used return Gatherer.ofSequential(initializer, integrator, finisher); which implies sequential processing (wasn't that obvious?). I selected a different requirement to demonstrate the usage of combiner. We should try to select the duplicates in a list or more accurate in a Stream. Ok, let us take a look into a simple solution (I bet there are other or even better solutions):

@Test
void exampleFindDuplicates() {
  var integers = List.of(100, 1, 10, 11, 5, 10, 11, 5, 100, 75, 78, 90);
  var duplicates = findDuplicates(integers);
  System.out.println("duplicates = " + duplicates);
}

List<Integer> findDuplicates(List<Integer> givenList) {
  long count = givenList.stream().distinct().count();
  if (count < givenList.size()) {
    return givenList.stream().filter(i -> Collections.frequency(givenList, i) > 1)
        .distinct().toList();
  } else {
    return List.of();
  }
}

This could be done in more general and more convenient way to be used in streams via Gathers like this:

static <T> Gatherer<? super T, ?, T> duplicatesWithoutCombiner() {
  Supplier<HashMap<T, Integer>> initializer = HashMap::new;
  //
  Gatherer.Integrator<HashMap<T, Integer>, T, T> integrator = (state, element, _) -> {
    var orDefault = state.getOrDefault(element, 0);
    state.put(element, orDefault + 1);
    return true;
  };
  //
  BiConsumer<HashMap<T, Integer>, Gatherer.Downstream<? super T>> finisher = (state, downstream) -> {
    state.forEach((k, v) -> {
      if (v >= 2) {
        downstream.push(k);
      }
    });
  };
  //
  return Gatherer.ofSequential(initializer, integrator, finisher);
}

This solution uses an initializer to create the internal state which is a HashMap<T, Integer>. The type T represents the used type by the stream. The Integer is simply to count the number of occurrence. The integrator does not emit any value, because the integrator can not say for sure if there will be coming more elements. That means the final emitting of the result can only be done in the finisher. That is of course only useful if the number of occurrence is greater or equal than two. If you run that on the defined set of number you will get:

resultList = [100, 5, 10, 11]

So now let us take a look into a solution with a combiner:

static <T> Gatherer<? super T, ?, T> duplicates() {
  Supplier<HashMap<T, Integer>> initializer = HashMap::new;
  //
  Gatherer.Integrator<HashMap<T, Integer>, T, T> integrator = (state, element, _) -> {
    var orDefault = state.getOrDefault(element, 0);
    state.put(element, orDefault + 1);
    return true;
  };
  //
  BiConsumer<HashMap<T, Integer>, Gatherer.Downstream<? super T>> finisher = (state, downstream) -> {
    state.forEach((k, v) -> {
      if (v >= 2) {
        downstream.push(k);
      }
    });
  };
  //
  BinaryOperator<HashMap<T, Integer>> combiner = (s1, s2) -> {
    s1.forEach((k, v) -> {
      var s1def = s2.getOrDefault(k, 0);
      s2.put(k, v + s1def);
    });
    return s2;
  };
  //
  return Gatherer.of(initializer, integrator, combiner, finisher);
}

There at two differences. First the last line uses return Gatherer.of(initializer, integrator, combiner, finisher); instead of the previous mentioned ... ofSequential(..) and of course the implementation of the combiner itself. The combiner is meant literally a combiner and it does not make any difference here if you return s2 or s1. You should of course change the implementation accordingly like this:

BinaryOperator<HashMap<T, Integer>> combiner = (s1, s2) -> {
  s2.forEach((k, v) -> {
    var s1def = s1.getOrDefault(k, 0);
    s1.put(k, v + s1def);
  });
  return s1;
};

You make sure the combining of two of those states results in the right thing. So finally you can now use the gatherer like this:

@Test
void exampleFindDuplicatesWithGathererCombiner() {
  var integers = List.of(100, 1, 10, 11, 5, 10, 11, 5, 100, 75, 78, 90);
  var resultList = integers.parallelStream().gather(duplicates()).toList();

  assertThat(resultList).containsExactlyInAnyOrder(100, 10, 11, 5);
}

If you like to see the combiner working you can simply put a print statement (or better use your IDE's debugger) in there and see that it is call several times.

I think this is a great enhancement for the JDK to make it more convenient customize required functionality based on the Stream API. Unfortunately, I have realized that the IntStream, LongStream and DoubleStream do not contain such an an enhancement. At the moment it looks like there is only a supplemental method missing gather on those three interfaces. Or maybe I misunderstand the complexity for adding such simple additions. The code for the examples here can be found Gatherer.

Happy coding.

Apache Maven JaCoCo Configuration

Karl Heinz Marbaise — Thu, 26 Oct 2023 19:07:54 +0000

Overview

In general having a number of unit- and integration tests in your project (BTW: I hope you have a larger number of tests) is a good thing. Afterwards the question arises: Are those tests enough? Have I covered all possible cases? Now the idea of code coverage comes into my mind. That is a way to check which paths, branches and methods (more accurately: function coverage, statement coverage, branch coverage) etc. have been executed during the tests. That means in other words the higher the rate of coverage is, the more unlikely it is having issues. Or more accurately having issues during the change of code. Yes, some people yell here and say a test does not proof bug free software. Absolutely correct. On the other hand it helps to keep the functionality in your code as it is, even if you change your code (internal implementation, adding new features etc.). So you are not breaking existing functionality (breaking things like that is often called regression).

Based on that, the final question arises: What should be the rate of your coverage? 30%, 50%, 80% or even 100%. Based on experience, I would say 30% is too low and 100% is a bit strange (in such a case you might even test setter's and getter's which does not really makes sense; this means testing the JDK) even if you could reach that level, you seem to be focusing on making the code coverage tool happy. The important part here is to deeply think about your tests and whether what's in your application, is the real critical part. So my practical advice would be ca. 80%. (Yes others might have a different opinion).

The tool to measure code coverage in Java is JaCoCo, but that is not the only one. There are existing things like OpenClover or in Kotlin things like Kover and so on (just to mention some).

I will use here JaCoCo, where also the JaCoCo-Maven-lugin exists for the usage in your Maven builds. This article will show how to configure the code coverage to finally get the results for unit- and integration-tests.

Example Project

Let's start with an example project which has the general structure like this:



src
|-- main
|   {% raw %}`-- java
|       `-- com
|           `-- soebes
|               `-- example
|                   `-- jacoco
|                       `-- First.java
`-- test
    `-- java
        `-- com
            `-- soebes
                `-- example
                    `-- jacoco
                        |-- FirstIT.java
                        `-- FirstTest.java
```
You might already have identified the unit test class based on the naming convention. I strongly recommend to follow that naming convention in any Maven projects. In this example there is a single one `FirstTest.java`. 
Furthermore, we have an integration test `FirstIT.java`. This is a [usual directory structure][usual-directory-structure] of a Maven project. Also, not to forget, we have to have a `pom.xml` file, which contains the configuration for the project.

## Plugin Definition
We should take a look into the configuration for the project and see, which parts are necessary to configure JaCoCo code coverage within the project. In general, you should define all used plugins within your Maven build to keep your build reproducible. The first thing is to define the `jacoco-maven-plugin` including its version and groupId. The ellipses `...`
represent other plugins, which should be defined, but currently are omitted for brevity. Furthermore, the [maven-surefire-plugin][maven-surefire-plugin] and [maven-failsafe-plugin][maven-failsafe-plugin] are required, because the unit tests are executed by the [maven-surefire-plugin][maven-surefire-plugin] while the integration tests are executed by the [maven-failsafe-plugin][maven-failsafe-plugin].

```xml
<project...>
  ...
  <build>
    <pluginManagement>
      <plugins>
        <plugin>
          <groupId>org.jacoco</groupId>
          <artifactId>jacoco-maven-plugin</artifactId>
          <version>0.8.11</version>
        </plugin>
        ...
        <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-surefire-plugin</artifactId>
          <version>3.1.2</version>
          <configuration>
            <skipTests>${skipUTs}</skipTests>
          </configuration>
        </plugin>
        <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-failsafe-plugin</artifactId>
          <version>3.1.2</version>
          <configuration>
            <skipTests>${skipITs}</skipTests>
          </configuration>
        </plugin>
      </plugins>
    </pluginManagement>
    ...
  </build>
  ...
</project...>
```
So basically we define the versions via `pluginManagement` and the global configuration. The next thing you need to configure is the `jacoco-maven-plugin`. In general jacoco works by using agents, which must be executed during the build. That can be accomplished by using the `prepare-agent` goal. This goal handles that the agent is added to the execution of the [maven-surefire-plugin][maven-surefire-plugin].

```xml
<project...>
  ...
  <build>
    ...
    <plugins>
      ..
      <plugin>
        <groupId>org.jacoco</groupId>
        <artifactId>jacoco-maven-plugin</artifactId>
        <executions>
          <execution>
            <goals>
              <goal>prepare-agent</goal>
              ...
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
  ...
</project...>
```

## Examine Execution
If you have applied the previous configuration, the execution can be examined via `mvn -X verify` and you will find something like this in the output. 

```text
'-javaagent:/.../org/jacoco/org.jacoco.agent/0.8.11/org.jacoco.agent-0.8.11-runtime.jar=destfile=/../target/jacoco.exec'
```

Be a little careful, because the output of using the `-X` debugging option is being very long. So best is to redirect the output into a file instead of the console.

The redirection can be done on unix like operating systems (Linux, MacOS) via:

```shell
mvn -X verify | tee mvn.log
```
and on Windows based machines like this:
```shell
mvn -X verify > mvn.log
```
Afterward, you can make a deep dive into the resulting output file `mvn.log`:

This above snippet also shows the resulting `jacoco.exec` file. This file contains the code coverage (binary format) information.

## HTML Report

If you like to produce a human-readable report, you have to add the supplemental `report` goal which looks like this:

```xml
<project...>
  ...
  <build>
    ...
    <plugins>
      ..
      <plugin>
        <groupId>org.jacoco</groupId>
        <artifactId>jacoco-maven-plugin</artifactId>
        <executions>
          <execution>
            <goals>
              <goal>prepare-agent</goal>
              <goal>report</goal>
              ...
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
  ...
</project...>
```
The given `report` goal creates an HTML based report(It includes XML- and CSV-based as well). It is necessary to run `mvn verify` to create that report, because the `report` goal is bound to the `verify` lifecycle-phase by default.
I strongly recommend to leave the bindings to the appropriate lifecycle-phase up to the plugins (or more accurate to the plugin authors). In 99.999% of the cases those configurations are correct. There might be exceptions. In such cases you might need to bind the goals to a different lifecycle-phase on your own.

After running `mvn verify`, the HTML report (including XML and CSV) can be found in the `target/site/jacoco/` directory. The HTML report can be watched by opening the `index.html` file, in a bowser of your choice.

If you don't like to run your whole build up to `verify` lifecycle-phase, but nevertheless you want to create the JaCoCo reports anyway, this can simply be accomplished by using:
```shell
mvn clean test jacoco:report
```
That will run the lifecycle up to `test` phase and separately calling the goal `report` of the `jacoco-maven-plugin` (In most cases you can simply omit the `clean`).

## Test Project
At the beginning I wrote about the test project structure, now we focus on the content of the test project. The production code contains a class `First.java` which contains the following code (JDK17+):
```java
public record First(int sum) {

  public First add(First first) {
    return new First(first.sum + this.sum);
  }

  public First sub(First first) {
    return new First(this.sum - first.sum);
  }
  public First minus(First first) {
    return new First(this.sum - first.sum);
  }
}
```
Do not judge this code, it is only for demonstration purposes 
 of the code coverage. The unit test class `FirstTest.java` contains the following:
```java
class FirstTest {

  @Test
  void first_add() {
    First sum1 = new First(5);
    First sum2 = new First(2);

    assertThat(sum1.add(sum2)).isEqualTo(new First(7));
  }
}
```
and the integration test class `FirstIT.java` is given as this:
```java
class FirstIT {

  @Test
  void first_minus() {
    First sum1 = new First(5);
    First sum2 = new First(2);

    assertThat(sum1.minus(sum2)).isEqualTo(new First(3));
  }
}
```

If we have taken a deeper look into the code, you have realized, that the unit test only covers the `add(..)` method while the integration test covers `minus(..)` method.

## Code Coverage 
So if we run `mvn clean verify`, which means to execute the unit tests. This includes executing of the `report` goal of JaCoCo Maven Plugin. Let us take a look into the code coverage report:

![JaCoCo Unit Test Coverage](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/24q59tbxog72onmr9yio.png)

What you can see is that only the `add(..)` method has been covered by the tests(Of course the constructor as well). Hold on a second?  What about the integration test which should cover the `minus(..)` method? Interestingly it's not shown as eing used which means, there is no test existing. That is a bit scary, because we wrote one `FirstIT.java`, or we did something else wrong. The simple issue here is that we haven't configured Maven, or more accurately, [maven-failsafe-plugin][maven-failsafe-plugin] to be executed during the integration test lifecycle phase. So there couldn't be any coverage based on the integration test at all. That issue can be solved by adding supplemental lifecycle binding for the [maven-failsafe-plugin][maven-failsafe-plugin] like this:

```xml
<project...>
  ...
  <build>
    ...
    <plugins>
      ..
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-failsafe-plugin</artifactId>
        <executions>
          <execution>
            <goals>
              <goal>integration-test</goal>
              <goal>verify</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
  ...
</project...>
```
After adding this, you can execute unit- and integration tests, in one go via `mvn clean verify`. 

![JaCoCo Unit Test Coverage](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yncyiplv634cx0ocpo9t.png)

As a result of the supplemental configuration, you can see that now, both methods `add(...)` as well as `minus(..)` are covered by tests. This is now as expected because `add(...)` is covered by the unit test `FirstTest.java` while `minus(..)` is covered by the integration test `FirstIT.java`. In the end we have accomplished the mission to get code coverage for our unit- and integration tests within a single build. It's important to mentioned that I don't have added supplemental configuration to reach that point. 

## Only Unit- or Integration Tests

Based on the given configuration of the [maven-surefire-plugin][maven-surefire-plugin] and the [maven-failsafe-plugin][maven-failsafe-plugin], you can execute all things separately. So let us execute the unit tests only, but nevertheless creating the reports. To skip the integration tests we can use the defined property `skipITs` of the configuration which looks like this:

```bash
mvn clean verify -DskipITs
```

The two defined properties `skipITs` and `skipUTs` give us the opportunity to control the execution of either unit- 
and or integration tests separately. The `report` goal of `jacoco-maven-plugin` will be executed at the `verify` lifecycle phase. If we want to run only the integration tests and report them separately, this can be achieved like this:

```bash
mvn clean verify -DskipUTs
```
Based on the given setup you can now select what you like to execute unit tests or integration tests or both together. The final result is having also appropriate coverage report.

## Conclusion
Unfortunately I often see configurations, where the agent is being configured separately for the plugins (maven-surefire-plugin, maven-failsafe-plugin) while this is not necessary. This is based on very old version of jacoco-maven-plugin less than 0.5.3 (if I correctly remember seven or eight years ago). That means today you don't need a supplemental configuration to handle that correctly.

There are some exceptions, where you have to add a definition for `argLine` while using [maven-surefire-plugin][maven-surefire-plugin] and/or the [maven-failsafe-plugin][maven-failsafe-plugin]. That's the case while you have to add supplemental configuration to the execution during the tests. That might be while you are using preview language features for JDK21+ or using JDK21 and using things like Mockito (more accurately byte-buddy which used by mockito) where configuration needed to be tweaked like this (Hint: That might change in the future):

```xml
...
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <configuration>
          <argLine>
            -XX:+EnableDynamicAgentLoading
            ...
          </argLine>
        </configuration>
      </plugin>
...
```

The given option `-XX:+EnableDynamicAgentLoading` is required to suppress WARNINGs like this:
```bash
WARNING: A Java agent has been loaded dynamically (/Users/khm/../net/bytebuddy/byte-buddy-agent/1.14.6/byte-buddy-agent-1.14.6.jar)
WARNING: If a serviceability tool is in use, please run with -XX:+EnableDynamicAgentLoading to hide this warning
WARNING: If a serviceability tool is not in use, please run with -Djdk.instrument.traceUsage for more information
WARNING: Dynamic loading of agents will be disallowed by default in a future release
```
So starting with [JDK 21](https://blog.soebes.io/posts/2023/06/2023-06-24-how-to-use-jdk21-preview-features-incubator/) it is required having such configuration while using Mockito. In consequence, you need a kind of tweaked setup to get a code coverage:

For [maven-surefire-plugin][maven-surefire-plugin] and [maven-failsafe-plugin][maven-failsafe-plugin] there is an option for [lazy property evaluation][lazy-property-evaluation], which means, that properties needed adding to the execution of surefire/failsafe plugin. An example of that is the supplemental configuration to prevent the above warnings.  

```xml
...
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <configuration>
          <argLine>
            @{argLine}
            -XX:+EnableDynamicAgentLoading
            ...
          </argLine>
        </configuration>
      </plugin>
...
```
I will emphasize the `@{argLine}` in the above configuration. I would suggest to put that configuration at the beginning of the configuration parts.

So if you like to look into a real project setup I can recommend the example project which I have created on [Github](https://github.com/khmarbaise/example-jacoco).

Analysing Download Statistics for Apache Maven

Karl Heinz Marbaise — Fri, 11 Aug 2023 15:06:34 +0000

Overview

I have access to the download statistics of some projects in Maven Central. That means I can take a look how many times Apache Maven has been downloaded etc. So I wanted to know some more details based on my role as Apache Maven Chairman and PMC member. So the first thing I've done, was to download for each month an csv file back to July 2022 and the latest thing I can access, at the time of writing this, was June 2022. So in the end this means, I have numbers for a full year which means from July 2022 to June 2023.

I have two different sets of files. One set of files, which represents the download numbers for Apache Maven Core (what you usually call Maven; the thing you are calling like this: mvn ...). The other set of files contains the downloads for a larger number of Apache Maven Plugins. Those plugins are under the groupId org.apache.maven.plugins. This represents the plugins, which can be found here: Apache Maven Plugin.

How To Analyse?

So there are several options, which I could have gone to analyse those files. Several people might have chosen things like Excel, Python, Perl (if anyone still knows that at all) or alike. I have to admit, I'm a Java developer, so the inclined reader might already guess what I selected? Yes, of course, I've decided to use Java for that task what else. In the end, there can be only one.

Maven Core Files

The first set of files represent the Apache Maven Core downloads. So let us take a look, how those csv files for Apache Maven Core look like? Here is an excerpt of the file for July 2022 (excerpt):

"2.0.10","32","4.232163064443739E-6"
...
"3.6.1","346818","0.04586844891309738"
"3.6.2","258989","0.03425261750817299"
"3.6.3","1988609","0.2630036771297455"
"3.8.1","580532","0.07677831500768661"
"3.8.2","151217","0.019999219104647636"
"3.8.3","295197","0.03904131054878235"
"3.8.4","666038","0.08808692544698715"
"3.8.5","516146","0.06826294213533401"
"3.8.6","461053","0.06097660958766937"

That means, the first column represents a Maven version, the second column is the number of downloads related to the file, which is monthly based(July 2022). The last column represents the relative number of downloads related to the total number of downloads within this month. So check the first line, which says there had been 32 downloads in July 2022 of Apache Maven 2.0.10 (Yes, that really existed even in that time; In 2009!). The 4.232163064443739E-6 represents the relative number 32 divided by the total number of downloads within that month or in other words 0.0004%. Or let us check the last line, which tells us that Apache Maven version 3.8.6 has been downloaded 461,053 times which means in other words 6.1%.

Maven Plugin Files

The second set of files contains the download number related to Apache Maven Plugin. So let us take a look here as well (excerpt):

"maven-pmd-plugin","448549","0.0024602634366601706"
"maven-project-info-reports-plugin","353876","0.0019409878877922893"
"maven-source-plugin","2585053","0.01417885534465313"
"maven-plugins","34702","1.9033830903936177E-4"
"maven-clean-plugin","16019474","0.08786582201719284"
"maven-resources-plugin","20210250","0.1108519658446312"
"maven-compiler-plugin","21722106","0.11914440244436264"
"maven-surefire-plugin","20446073","0.11214543133974075"
"maven-jar-plugin","16667766","0.09142166376113892"
"maven-site-plugin","10041046","0.05507451295852661"
"maven-install-plugin","13677668","0.0750211551785469"
...
"maven-artifact-plugin","709","3.888821083819494E-6"
"maven-scripting-plugin","113","6.197979587341251E-7"

In the first column is the name of the plugin, for example maven-compiler-plugin and the number of downloads in the second columns and also the relative number of downloads related to the total number of downloads within that month. So this means the maven-compiler-plugin has been downloaded 21,722,106 times (ca. 21 million) in July 2022 and the 0.11914440244436264 means in other words ca. 11.9%.

Starting the task

I start with the set of files, which are related to Maven Core download numbers. So the first task was to read those files or more accurate to identify the files I need to read? That can be done in Java easily:

Predicate<Path> IS_REGULAR_FILE = Files::isRegularFile;
Predicate<Path> IS_READABLE = Files::isReadable;
Predicate<Path> IS_VALID_FILE = IS_REGULAR_FILE.and(IS_READABLE);

static List<Path> allFilesInDirectoryTree(Path start) throws IOException{
    try(var pathStream=Files.walk(start)){
    return pathStream.filter(IS_VALID_FILE).toList();
    }
}

That results in a list with all files, that are existing in the directory start including its subdirectories. Now I need to filter out only those names I'm currently interested in.

var filesInDirectory = allFilesInDirectoryTree(rootDirectory);

var listOfSelectedFiles = filesInDirectory.stream()
    .filter(s -> s.getFileName().toString().startsWith("apache-maven-stats"))
    .toList();

So all file names I'm currently interested in are look like this:

apache-maven-stats-2022-07.cvs
apache-maven-stats-2022-08.cvs
...
apache-maven-stats-2023-01.cvs
apache-maven-stats-2023-02.cvs
..

Now the interesting part. How to read them? The entries in line are separated by "," and the entries itself are quoted with double quotes. The first column contains the version of Maven Core or in other words the Maven version, the second column contains the number of downloads during that month, while the third column contains the number related to the total of downloads during that month. Here an example line from one of the files:

"2.0.10","32","4.232163064443739E-6"

So first things first...read a single file:

static ... convert(Path csvFile) {
  try (var lines = Files.lines(csvFile)) {
    ...
  } catch (IOException e) {
    throw new RuntimeException(e);
  }
}

You might be a bit astonished about those ellipsis (...) within the example, those things are left out on purpose, to show you step-by-step process how to solve such a problem in an easy way and focus on important parts here. So you see a try-catch-with-resources, which is necessary, because the result of Files.lines(csvFile)) is a Stream<String>, whereas the Stream is auto-closable. The translation of the IOException into an RuntimeException might look a bit weird, but it makes it easier to use the convert later in a Stream. So let us fill the ellipsis within the try-catch-with-resources with life, which can be handled like this:

static String unquote(String withQuotes) {
    return withQuotes.substring(1, withQuotes.length() - 1);
}

static ... convert(Path csvFile) {
  try (var lines = Files.lines(csvFile)) {
    return lines.map(s -> s.split(","))
    .map(arr -> Line.of(unquote(arr[0]), unquote(arr[1]), unquote(arr[2])))
    .map(MavenStats::of)
    .toList();
  } catch (IOException e) {
    throw new RuntimeException(e);
  }
}

So lines.map(s -> s.split(",")) is splitting the line at the "," while the line .map(arr -> Line.of(unquote(arr[0]), unquote(arr[1]), unquote(arr[2]))) translates the array into a "Line" record and uses a helper method unquote which removes the quotes.
So let use take a look onto the code of Line type:

record Line(ComparableVersion version, long number, double percentage) {
  static Line of(String version, String number, String percentage) {
    return new Line(new ComparableVersion(version), Long.parseLong(number), Double.parseDouble(percentage));
  }
}

So via the helper method of it will translate the given strings into the appropriate data types. At this point there is a very important part, because using ComparableVersion simplifies a lot of things. This type exists in Maven Core itself, which I can reuse (because I'm doing this in Java) to make my life easier and compare later the version of Maven. So I don't need to write something myself.

This results into a type Line which contains the ComparableVersion, the number(downloads) and the percentage (yeah, I Know that's not accurate).

So now I finally convert it into a domain type MavenStats via MavenStats::of. This looks like this:

record MavenStats(ComparableVersion version, long number, double percentage) {
  static MavenStats of(Line line) {
    return new MavenStats(line.version(), line.number(), line.percentage());
  }
}

You might argue, that I should have used not an intermediate type Line while going to MavenStats, but from my perspective this makes a clear separation between the data coming from the file and the domain specific meaning. So let use take a look to the whole code in one:

record MavenStats(ComparableVersion version, long number, double percentage) {
  static MavenStats of(Line line) {
    return new MavenStats(line.version(), line.number(), line.percentage());
  }
}

record Line(ComparableVersion version, long number, double percentage) {
  static Line of(String version, String number, String percentage) {
    return new Line(new ComparableVersion(version), Long.parseLong(number), Double.parseDouble(percentage));
  }
}

static String unquote(String withQuotes) {
  return withQuotes.substring(1, withQuotes.length() - 1);
}

static List<MavenStats> convert(Path csvFile) {
  try (var lines = Files.lines(csvFile)) {
    return lines.map(s -> s.split(","))
        .map(arr -> Line.of(unquote(arr[0]), unquote(arr[1]), unquote(arr[2])))
        .map(MavenStats::of)
        .toList();
  } catch (IOException e) {
    throw new RuntimeException(e);
  }
}

So in the end the convert method will convert the line oriented content of the file into appropriate type safe records (MavenStats). Now we have to continue to handle all files in a convenient way. The information we are missing about the year and month, which is contained within the file name (Yes I have stored them in that way). Let us take a look onto the given code:

record YearMonth(int year, int month, List<MavenStats> lines) {
}

record YearMonthFile(int year, int month, Path fileName) {
  static YearMonthFile of (Path fileName) {
    String fileNameOnly = fileName.getFileName().toString();
    int month = Integer.parseInt(fileNameOnly.substring(fileNameOnly.length() - 6, fileNameOnly.length() - 4));
    int year = Integer.parseInt(fileNameOnly.substring(fileNameOnly.length() - 11, fileNameOnly.length() - 7));
    return new YearMonthFile(year, month, fileName);
  }
}

static List<YearMonth> readCSVStatistics(Path rootDirectory) throws IOException {
  var filesInDirectory = allFilesInDirectoryTree(rootDirectory);

  var listOfSelectedFiles = filesInDirectory.stream()
      .filter(s -> s.getFileName().toString().startsWith("apache-maven-stats"))
      .toList();

  return listOfSelectedFiles
      .stream()
      .map(YearMonthFile::of)
      .map(ymf -> new YearMonth(ymf.year(), ymf.month(), convert(ymf.fileName())))
      .toList();
}

In the calling of the YearMonth you see convert(ymf.fileName()), which is exactly the method explained previously. So here we go via allFilesInDirectoryTree(rootDirectory) which reads all file and will be filtered via .filter(s -> s.getFileName().toString().startsWith("apache-maven-stats")).
In lines .map(YearMonthFile::of) the month and year will be extracted from the file name converted into the YearMonthFile record type and the line will convert that into YearMonth type, which contains all the information of a single file (year, month and all other information).

So now we are at the point, where we can really use the data we have extracted from the CSV files.

Comparator<YearMonth> YEAR_MONTH_COMPARATOR = Comparator
  .comparingInt(YearMonth::year)
  .thenComparingInt(YearMonth::month);

var mavenVersionStatistics = readCSVStatistics(rootDirectory);

mavenVersionStatistics
    .stream().sorted(YEAR_MONTH_COMPARATOR)
    .forEach(s -> {
      var totalOverAllVersions = s.lines()
          .stream()
          .mapToLong(MavenStats::number).sum();
      out.printf("Year: %04d %02d %,10d %4d %n", s.year(), s.month(), totalOverAllVersions, s.lines().size());
    });

So now it's only a task of combining the stream things correctly to get interesting insights which look like this:

Year: 2022 07  7,561,145   49 
Year: 2022 08  8,115,080   48 
Year: 2022 09  8,186,047   49 
Year: 2022 10  8,228,687   49 
Year: 2022 11  8,666,013   49 
Year: 2022 12  7,958,914   51 
Year: 2023 01  8,799,003   52 
Year: 2023 02  8,914,430   54 
Year: 2023 03 10,526,609   57 
Year: 2023 04  8,981,241   57 
Year: 2023 05  9,606,494   58 
Year: 2023 06  9,621,748   60

So that means, in July 2022 more than 7.5 million downloads of Apache Maven itself have been done. There have existed 49 different versions of Apache Maven (yes so many versions existing in the meantime). In August 2022 there had been 8.1 million downloads of Apache Maven, but a version less, which means a single version has not been downloaded within that month at all and so on. It's even visible, that over the time other versions have been added as you can see because the number of versions is increasing.
So it would be interesting how many downloads in total for that year have been done? That question can easily being answered by using some code:

var totalOfDownloadsOverallMavenVersions = mavenVersionStatistics.stream()
    .map(s -> s.lines().stream().mapToLong(MavenStats::number).sum())
    .mapToLong(__ -> __).sum();

out.printf("totalOfDownloadsOverallMavenVersions: %,12d%n", totalOfDownloadsOverallMavenVersions);

and the result of this is:

totalOfDownloadsOverallMavenVersions:  105,165,411

That means, that during July 2022 and June 2023 more than 105 million downloads have happened from Maven Central for
Apache Maven Core.

So another interesting question could be: How is the distribution over the different Maven versions?

That can be answered easy via the given code:

var groupedByMavenVersion = mavenVersionStatistics.stream()
    .flatMap(s -> s.lines().stream())
    .collect(Collectors.groupingBy(MavenStats::version, Collectors.summingLong(MavenStats::number)));

This will contain the answer, we only need to sort ccordingly (reversed to get the highest number in the first line) and keep the total number in mind and calculating the percentage related to the total:

groupedByMavenVersion
    .entrySet()
    .stream().sorted(Map.Entry.<ComparableVersion, Long>comparingByValue().reversed())
    .forEach(s -> {
      double percentage = s.getValue() / (double)sum * 100.0;
      out.printf("%-15s %,12d %6.2f%n", s.getKey(), s.getValue(), percentage);
    });

with the following result (only excerpts):

3.6.3             25,173,440  23.94
3.8.6             11,324,307  10.77
3.8.4              7,908,716   7.52
3.8.1              7,276,857   6.92
3.5.4              6,998,521   6.65
3.3.9              5,997,064   5.70
3.8.5              5,820,216   5.53
3.6.1              5,202,935   4.95
3.6.0              3,749,450   3.57
3.8.7              3,442,747   3.27
3.8.3              3,310,941   3.15
3.6.2              2,375,723   2.26
3.8.2              2,347,698   2.23
3.5.3              2,305,157   2.19
3.5.2              1,728,117   1.64
3.5.0              1,588,450   1.51
3.9.1              1,489,846   1.42
3.1.1              1,090,059   1.04
3.9.2              1,054,414   1.00
3.9.0              1,014,136   0.96
3.3.3                761,617   0.72
3.2.5                689,786   0.66
3.0.4                640,818   0.61
3.8.8                525,917   0.50
3.3.1                351,825   0.33
3.0                  244,140   0.23
...

That means ca. 24% (23.94% roughly a quarter) of all downloads are using an old version of Maven 3.6.3 (2019!) and ca. 11% are using 3.8.6 (June 2022) etc.

This makes me a bit sad, because 3.6.3 is not only almost four years old, it also contains a lot of bugs etc., which have been fixed in subsequent versions. That is even worse for older versions like 3.5.4(ca. 6.65%) or 3.3.9 (ca. 5.7%) etc. Only a few builds using already more or less recent versions like 3.9.X(for example: 3.9.2 ca. 1%).

Maven Plugins

Based on the given CSV files for plugin downloads I've taken the same approaches to read those files. Using the following code:

var paths = allFilesInDirectoryTree(rootDirectory);
var listOfFiles = paths.stream()
    .filter(s -> s.getFileName().toString().startsWith("org-apache-maven-plugins"))
    .toList();

var mavenPluginStatistics = listOfFiles.stream()
    .map(YearMonthFile::of)
    .map(ymf -> new YearMonth(ymf.year(), ymf.month(), convert(ymf.fileName())))
    .toList();

mavenPluginStatistics.stream()
    .sorted(YEAR_MONTH_COMPARATOR)
    .forEach(s -> {
      var sumOfDownloadsPerMonth = s.lines().stream().mapToLong(MavenPlugin::number).sum();
      out.printf("Year: %04d %02d %3d %,15d %n", s.year(), s.month(), s.lines().size(), sumOfDownloadsPerMonth);

      s.lines().stream()
          .sorted(Comparator.comparing(MavenPlugin::plugin))
          .forEachOrdered(l -> out.printf(" %-36s %,10d%n", l.plugin(), l.number()));

    });

The result only output for a single month looks like this:

Year: 2022 07  66     182,317,478

This means, that only in July 2022, overall more than 182 million downloads over all plugins have been done. Following some excerpts from the data:

 maven-acr-plugin                          2,289
 ...
 maven-clean-plugin                   16,019,474
 ...
 maven-compiler-plugin                21,722,106
 ...
 maven-dependency-plugin               7,349,854
 maven-deploy-plugin                  10,388,130
 ...
 maven-ear-plugin                        199,067
 ...
 maven-ejb-plugin                         92,910
 ...
 maven-enforcer-plugin                 3,469,962
 maven-failsafe-plugin                 3,203,618
 ...
 maven-install-plugin                 13,677,668
 ...
 maven-jar-plugin                     16,667,766
 ...
 maven-resources-plugin               20,210,250
 ...
 maven-surefire-plugin                20,446,073
 ..
 maven-war-plugin                      3,021,061

Ok, diving a bit more into the numbers. The maven-compiler-plugin has been downloaded 21,722,106 only in July 2022. That in consequence means, that in that month at least 21,722,106 builds had been run.

If you summarize all the numbers together, you will get: 2,561,522,497. Yes, you have read correctly, more than 2.5 billion downloads over a year of all plugins. If we select a number of more or less usual plugins (15) from those, we get a list like this:

maven-clean-plugin                       229,660,530
maven-compiler-plugin                    293,531,436
maven-dependency-plugin                  102,662,615
maven-deploy-plugin                      157,013,040
maven-ear-plugin                           2,904,003
maven-ejb-plugin                           1,417,372
maven-enforcer-plugin                     45,466,500
maven-failsafe-plugin                     44,748,498
maven-help-plugin                         34,984,247
maven-install-plugin                     195,701,012
maven-jar-plugin                         233,505,200
maven-javadoc-plugin                      29,235,021
maven-resources-plugin                   276,799,951
maven-surefire-plugin                    283,761,873
maven-war-plugin                          38,618,180

This adds up to ca. 1.97 billion (exactly 1,970,009,478 billion) downloads over a year. Furthermore, that shows some interesting insights. The maven-compiler-plugin is used most, which is not really astonishing, because more or less every build needs to compile code which is done by the maven-compiler-plugin.
On the other hand it's a bit weird, that the maven-clean-plugin is called very often as well. That will destroy the opportunity to reuse already built parts, which in the end requires to build everything from scratch. Reconsider the usage of maven-clean-plugin or in other words using mvn clean...? I recommend to check your own builds, if you really need to use mvn clean ..?
What I don't understand is why people seemed to be using maven-dependency-plugin that often? Why? Analysing dependencies that often? Or copying artifacts? I'm not sure, if that is really necessary or even useful, but finally I don't know.

The number for the maven-deploy-plugin shows, that of those builds ca. 50% (compared to maven-compiler-plugin) are also deployed to a remote repository. A kind of strange is, that the number for the maven-install-plugin is ca. 60,0 million less than the number for maven-deploy-plugin? That means, that many people are using mvn install or mvn clean install or alike. Only use install life cycle, if you really need to (I have my doubts, that you really need it to do install). I bet, that in the majority of cases mvn verify is sufficient. How do I know? Based on the difference between maven-install-plugin and maven-deploy-plugin ca. 38 million times.

It is very good to see, that maven-surefire-plugin is used more or less at the same frequency as maven-compiler-plugin yes, there is a difference of ca. 10 million which I can think of using mvn -DskipTests or alike. On the other hand that means, that a great number of people are running their unit tests. The difference between maven-compiler-plugin and the maven-resources-plugin, which I'm not sure, where it's coming from? An interesting thing is the number for the maven-jar-plugin vs. maven-ejb-plugin, maven-ear-plugin and maven-war-plugin is roughly 43 million while the difference between maven-compiler-plugin and maven-jar-plugin is ca. 60 million. This means there are ca. 17 million builds using something different from jar packaging type. It's even very obvious, that the number of builds using war, ear and ejb is at a very low number (total ca. 43 million) compared to other build types, which are about 14% related to the total number. If you even compare ear and ejb to maven-compiler-plugin builds, the relation is roughly 1.4%.

One more thing, which catches my eyes. The number for maven-failsafe-plugin seemed to be very low (only ca. 15%) in comparison to the number of maven-compiler-plugin? Why? That means, not soo many people are doing integration testing or maybe not using the maven-failsafe-plugin which I have observed very often. There might be reasons not to do integration tests (taking too much time or other reasons) or sometimes people are misusing the maven-surefire-plugin with profiles or alike to do so.

Maybe my conclusions are simply wrong. I know that I know nothing.

All the information I'm using here can be found inclusive the
code in a GitHub repository
https://github.com/khmarbaise/maven-downloads.

Original post: https://blog.soebes.io/posts/2023/08/2023-08-06-download-statistics-for-maven/

Using JDK21 Preview Features and/or incubator classes

Karl Heinz Marbaise — Mon, 26 Jun 2023 19:40:26 +0000

Sometimes you want to play around with those new fancy features of JDK21 (or even newer JDK's) like preview features and maybe some classes from the incubator.

So how can you configure your Maven build to support such a play lesson? It's easier than you think. Let's start the configuration. My assumption is that you would like to play around with a preview features of JDK21 for example String Templates (JEP430). I just selected this JEP for demonstration. You can select whatever JEP in preview.

The first thing is to know that you have to activate the preview features via:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-compiler-plugin</artifactId>
  <configuration>
    <enablePreview>true</enablePreview>
  </configuration>
</plugin>

The configuration enablePreview exists since version 3.10.1 of the Maven Compiler Plugin. So in general I recommend to use the most recent versions of the plugins otherwise it might happen that some configuration options or alike might not exist at all or special things for newer JDK versions.

This should be done in general in the pluginManagement section of your pom.xml file.

So now you are able to use this new feature in your test code (you could use it in your production code as well if you prefer that):

package com.soebes.jdk21;

import org.junit.jupiter.api.Test;

import static java.lang.StringTemplate.STR;
import static org.assertj.core.api.Assertions.assertThat;

class TemplateTest {
  @Test
  void name() {
    String name = "Jon";
    String info = STR."My name is \{name}";
    assertThat(info).isEqualTo("My name is Jon");
  }

}

Ok so far so good, but wait, before you can run this test you have to do a bit more. We need to change the configuration to run the test with the support of preview feature. This is required to give the JVM the option --enable-preview which is used during the execution of the tests. This has to be done like the following:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <argLine>
      --enable-preview
    </argLine>
  </configuration>
</plugin>

As I mentioned before, I strongly recommend to use the most recent versions of the plugins.

One important thing to mention. If you run those tests or the whole build you will get some WARNINGs like the following:

[INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ jdk21 ---
[INFO] Changes detected - recompiling the module! :dependency
[INFO] Compiling 5 source files with javac [release 21] to target/test-classes
[WARNING] ../jdk21/src/test/java/com/soebes/jdk21/TemplateTest.java:[12,23] string templates are a preview feature and may be removed in a future release.
[WARNING] ../jdk21/src/test/java/com/soebes/jdk21/TemplateTest.java:[12,23] string templates are a preview feature and may be removed in a future release.
[WARNING] ../jdk21/src/test/java/com/soebes/jdk21/TemplateTest.java:[5,24] java.lang.StringTemplate is a preview API and may be removed in a future release.
[INFO]

Those WARNINGs would have been emitted during the production code compilation if you used such feature in the production already. This makes it clear that you are using such preview features.

And an extremely important hint:

Never deploy such code which uses preview features anywhere. Not in your company repository manger nor into central repository

The StringTemplate class is marked as a preview feature independent that it is the package java.lang:

@PreviewFeature(feature=PreviewFeature.Feature.STRING_TEMPLATES)
public interface StringTemplate {
  ...

}

So another nice thing would be to use a class from the incubator like the Vector API? Ok. What do we need to do?
The first thing is to enhance our previous configuration with the --add-modules jdk.incubator.vector information
like the following:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-compiler-plugin</artifactId>
  <configuration>
    <enablePreview>true</enablePreview>
    <compilerArgs>
      <arg>--add-modules</arg>
      <arg>jdk.incubator.vector</arg>
    </compilerArgs>
  </configuration>
</plugin>

And of course you have to add the same to add to the maven-surefire-plugin configuration to activate the jdk.incubator.vector module in your tests as well. This means the configuration for maven-surefire-plugin should look like the following:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <argLine>
      --enable-preview
      --add-modules jdk.incubator.vector
    </argLine>
  </configuration>
</plugin>

This results in WARNING as before as the following:

..
[INFO] --- compiler:3.11.0:compile (default-compile) @ jdk21 ---
[INFO] Changes detected - recompiling the module! :source
[INFO] Compiling 2 source files with javac [release 21] to target/classes
[WARNING] using incubating module(s): jdk.incubator.vector
[INFO] 
[INFO] --- resources:3.3.1:testResources (default-testResources) @ jdk21 ---
[INFO] skip non existing resourceDirectory ../jdk21/src/test/resources
[INFO] skip non existing resourceDirectory ../jdk21/src/test/resources-filtered
[INFO] 
[INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ jdk21 ---
[INFO] Changes detected - recompiling the module! :dependency
[INFO] Compiling 5 source files with javac [release 21] to target/test-classes
[WARNING] ../jdk21/src/test/java/com/soebes/jdk21/TemplateTest.java:[12,23] string templates are a preview feature and may be removed in a future release.
[WARNING] ../jdk21/src/test/java/com/soebes/jdk21/TemplateTest.java:[12,23] string templates are a preview feature and may be removed in a future release.
[WARNING] using incubating module(s): jdk.incubator.vector
[WARNING] ../jdk21/src/test/java/com/soebes/jdk21/TemplateTest.java:[5,24] java.lang.StringTemplate is a preview API and may be removed in a future release.
[INFO] 
[INFO] --- surefire:3.1.2:test (default-test) @ jdk21 ---
[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
WARNING: Using incubator modules: jdk.incubator.vector
[INFO] Running com.soebes.jdk21.TemplateTest
..

A small hint based on the usage of JDK21+. Starting with JDK21 the JEP-451 has been integrated into the JDK21 builds which means the usage of dynamic loaded agents for example the famous Mockito library results into WARNINGs as well which looks like this:

..
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.217 s -- in com.soebes.jdk21.SequencedCollectionTest
[INFO] Running com.soebes.jdk21.AFinalClassTest
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
WARNING: A Java agent has been loaded dynamically (/Users/khm/.m2/repository/net/bytebuddy/byte-buddy-agent/1.14.5/byte-buddy-agent-1.14.5.jar)
WARNING: If a serviceability tool is in use, please run with -XX:+EnableDynamicAgentLoading to hide this warning
WARNING: If a serviceability tool is not in use, please run with -Djdk.instrument.traceUsage for more information
WARNING: Dynamic loading of agents will be disallowed by default in a future release
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.472 s -- in com.soebes.jdk21.AFinalClassTest
[INFO] Running com.soebes.jdk21.incubator.VectorTest
..

This makes it necessary to add the supplemental configuration -XX:+EnableDynamicAgentLoading to your maven-surefire-plugin configuration like this:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <argLine>
      --enable-preview
      --add-modules jdk.incubator.vector
      -XX:+EnableDynamicAgentLoading
    </argLine>
  </configuration>
</plugin>

A full working example which obviously requires JDK21 most recent EA build can be found here.

Happy coding.

Maven Plugin Configuration - The (Unknown) Tiny Details

Karl Heinz Marbaise — Mon, 05 Apr 2021 20:39:12 +0000

Overview

If you are using Apache Maven I suppose you have configured several plugins within your pom file already. As an example we use the Maven Enforcer Plugin to prevent starting a given build with a Maven version which is less than 3.5.0 which looks like this:

<project>
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-enforcer-plugin</artifactId>
        <version>3.0.0-M3</version>
        <executions>
          <execution>
            <id>id-enforce</id>
            <goals>
              <goal>enforce</goal>
            </goals>
            <configuration>
              <rules>
                <requireMavenVersion>
                  <version>3.5.0</version>
                </requireMavenVersion>
              </rules>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

Now you are calling Maven like this: mvn enforcer:enforce but now you are astonished about the failure which you will be faced with (for brevity inserted some line breaks into the output):

$ mvn enforcer:enforce 
[INFO] Scanning for projects...
[INFO] 
[INFO] -------------< com.soebes.articles.plugins:configuration >--------------
[INFO] Building configuration 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-enforcer-plugin:3.0.0-M3:enforce (default-cli) @ configuration ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.363 s
[INFO] Finished at: 2021-03-22T20:01:53+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M3:enforce (default-cli)
on project configuration: No rules are configured. Use the skip flag if you want to disable execution. -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

That looks like the configuration we have given in the example is not taken into account. What should
I say. Yes that is exactly the point. Now the crucial question is: Why?

Details

Let's dive more into details why this happens. The reason is that by calling Maven via mvn enforcer:enforce you have not started a life cycle (something like clean, package or alike). You have requested to execute the goal enforce of the Maven Enforcer Plugin.

This means in the end there is a difference for configuring plugins between calling a life cycle and a goal.

How can we check that what I have claimed is correct?
That's easy to accomplish by just calling a life cycle phase like initialize (you can also use verify or package instead of initialize only takes longer to run.):

$ mvn initialize 
[INFO] Scanning for projects...
[INFO] 
[INFO] -------------< com.soebes.articles.plugins:configuration >--------------
[INFO] Building configuration 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-enforcer-plugin:3.0.0-M3:enforce (id-enforce) @ configuration ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.355 s
[INFO] Finished at: 2021-03-22T20:02:40+01:00
[INFO] ------------------------------------------------------------------------

So first mission accomplished and proved that the previously given configuration is now taken into account.

So based on those insights can we somehow create a configuration which is taken into account while calling a goal enforcer:enforce ? Yes there is a way to do so via the following configuration:

<project>
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-enforcer-plugin</artifactId>
        <version>3.0.0-M3</version>
        <executions>
          <execution>
            <id>default-cli</id>
            <goals>
              <goal>enforce</goal>
            </goals>
            <configuration>
              <rules>
                <requireMavenVersion>
                  <version>3.5.0</version>
                </requireMavenVersion>
              </rules>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

Now we are calling the goal:

$ mvn enforcer:enforce 
[INFO] Scanning for projects...
[INFO] 
[INFO] -------------< com.soebes.articles.plugins:configuration >--------------
[INFO] Building configuration 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-enforcer-plugin:3.0.0-M3:enforce (default-cli) @ configuration ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.516 s
[INFO] Finished at: 2021-03-22T20:29:34+01:00
[INFO] ------------------------------------------------------------------------

You can also call the life cycle like this:

$ mvn initialize 
[INFO] Scanning for projects...
[INFO] 
[INFO] -------------< com.soebes.articles.plugins:configuration >--------------
[INFO] Building configuration 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-enforcer-plugin:3.0.0-M3:enforce (default-cli) @ configuration ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.498 s
[INFO] Finished at: 2021-03-27T16:18:03+01:00
[INFO] ------------------------------------------------------------------------

Have you spotted the difference? The only difference was between <id>id-enforce</id> (of the first configuration example) and <id>default-cli</id> in the second one. The name default-cli is, as the name implies, the default used for command line execution. This means you can define configuration/execution which are only used for calling goals from command line and not part of the life cycle.

In the next step we would like to make configurations where one is applied for command line execution <id>default-cli</id> and another for life cycle execution <id>another-id</id>.

<project>
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-enforcer-plugin</artifactId>
        <version>3.0.0-M3</version>
        <executions>
          <execution>
            <id>default-cli</id>
            <goals>
              <goal>enforce</goal>
            </goals>
            <configuration>
              <rules>
                <requireJavaVersion>
                  <version>25</version>
                </requireJavaVersion>
              </rules>
            </configuration>
          </execution>
          <execution>
            <id>another-id</id>
            <goals>
              <goal>enforce</goal>
            </goals>
            <configuration>
              <rules>
                <requireMavenVersion>
                  <version>3.9.0</version>
                </requireMavenVersion>
              </rules>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

The above configuration requires a JDK version 25 being used for building instead of the one I'm currently using (Just for demonstration purposes; Ok maybe a bit in the future). The second configuration checks for the Maven version which is expected to be used for that build. So let use call the goal first:

$ mvn enforcer:enforce 
[INFO] Scanning for projects...
[INFO] 
[INFO] -------------< com.soebes.articles.plugins:configuration >--------------
[INFO] Building configuration 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-enforcer-plugin:3.0.0-M3:enforce (default-cli) @ configuration ---
[WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireJavaVersion failed with message:
Detected JDK Version: 11.0.9 is not in the allowed range 25.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.549 s
[INFO] Finished at: 2021-03-22T20:45:27+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M3:enforce (default-cli)
 on project configuration: Some Enforcer rules have failed. Look above for specific messages explaining 
 why the rule failed. -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

You can see the command line only configuration is taken into account which can be identified by the message Detected JDK Version: 11.0.9 is not in the allowed range 25.. So worked as expected. Now we try to execute within life cycle:

$ mvn initialize 
[INFO] Scanning for projects...
[INFO] 
[INFO] -------------< com.soebes.articles.plugins:configuration >--------------
[INFO] Building configuration 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-enforcer-plugin:3.0.0-M3:enforce (default-cli) @ configuration ---
[WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireJavaVersion failed with message:
Detected JDK Version: 11.0.9 is not in the allowed range 25.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.450 s
[INFO] Finished at: 2021-03-27T16:40:31+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M3:enforce (default-cli) 
on project configuration: Some Enforcer rules have failed. Look above for specific messages explaining 
why the rule failed. -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

This hasn't been expected? What went wrong? The problem is simply related to the plugin or more accurate to the enforce goal which has a default binding to the life cycle which can be identified by reading the doc of the plugin (excerpt of it):

enforcer:enforce
Full name:
  org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M3:enforce

Description:
  This goal executes the defined enforcer-rules once per module.
  Attributes:

* Requires a Maven project to be executed.
* Binds by default to the lifecycle phase: validate.

The important part is the last line: Binds by default to the lifecycle phase: validate.. This describes the behaviour we can observe here. We are using the goal enforce simply by defining the goal. If you bind a plugin like the following:

<project>
.....
        <executions>
          <execution>
            <id>default-cli</id>
            <goals>
              <goal>enforce</goal>
            </goals>
            <configuration>
              <rules>
                <requireJavaVersion>
                  <version>25</version>
                </requireJavaVersion>
              </rules>
            </configuration>
          </execution>
          <execution>
            <id>another-id</id>
            <goals>
              <goal>enforce</goal>
            </goals>
            <configuration>
              <rules>
                <requireMavenVersion>
                  <version>3.9.0</version>
                </requireMavenVersion>
              </rules>
            </configuration>
          </execution>
        </executions>
  ...
</project>

This implies that the life cycle phase to which the plugin is bound is done by the plugin itself. That means in the end the final configuration it looks like this:

<project>
.....
        <executions>
          <execution>
            <id>default-cli</id>
            <goals>
              <goal>enforce</goal>
            </goals>
            <phase>initialize</phase>
            <configuration>
              <rules>
                <requireJavaVersion>
                  <version>25</version>
                </requireJavaVersion>
              </rules>
            </configuration>
          </execution>
          <execution>
            <id>another-id</id>
            <goals>
              <goal>enforce</goal>
            </goals>
            <phase>initialize</phase>
            <configuration>
              <rules>
                <requireMavenVersion>
                  <version>3.9.0</version>
                </requireMavenVersion>
              </rules>
            </configuration>
          </execution>
        </executions>
  ...
</project>

This means by calling the life cycle mvn initialize both executions will be executed and in the end also the first one (for java version) is executed even by giving a <id>default-cli</id>.

Life Cycle Binding

In cases where you want to make clear in which life cycle phase a plugin is bound just write it into the configuration. That makes it possible to override the defaults which a plugin (better the plugin author(s)) has defined. We can use this information to solve our problem by changing the configuration via the following approach: We simply decouple the enforce goal for one execution from the life cycle by defining a life cycle phase (UNKNOWN) which simply does not exist.

<project>
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-enforcer-plugin</artifactId>
        <version>3.0.0-M3</version>
        <executions>
          <execution>
            <id>default-cli</id>
            <goals>
              <goal>enforce</goal>
            </goals>
            <phase>UNKNOWN</phase>
            <configuration>
              <rules>
                <requireJavaVersion>
                  <version>25</version>
                </requireJavaVersion>
              </rules>
            </configuration>
          </execution>
          <execution>
            <id>another-id</id>
            <goals>
              <goal>enforce</goal>
            </goals>
            <configuration>
              <rules>
                <requireMavenVersion>
                  <version>3.9.0</version>
                </requireMavenVersion>
              </rules>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

The other configuration <id>another-id</id> can be keep unchanged. This (intended) coupling could have been identified in earlier outputs where a line was printed out:

$ mvn initialize
...
[INFO] --- maven-enforcer-plugin:3.0.0-M3:enforce (default-cli) @ configuration ---
..

This indicates that the goal enforce of the plugin is bound to a default life cycle initialize by the plugin itself in contradiction to the output (default-cli) (which might brought us to the wrong path).

This is a strong indicator that the given goal (enforce) is not intended to be called from command line. I generally recommend not to try to configure/execute goals on command line while the docs shows: Binds by default to the lifecycle phase: ... There are most likely exceptions but in general do not try it otherwise you would enter the dark side of the force ;-)

There are also examples like Maven Assembly Plugin which has exactly one goal single which is not bound to a life cycle phase at all. This does not mean to use the single goal via command line (technically you can) but usually it does not make sense.
Generally it makes sense to bind the single goal to a life cycle phase via the appropriate configuration. Based on different project setups it could make sense to bind it to prepare-package or to the package phase instead. This is exactly the reason that the plugin is not bound to any phase by default otherwise it would be necessary to decouple it from one of the phases if you are using the other one.

Goals For Command Line

Let us take a look to a plugin which is used from the command more or less exclusively. The Versions Maven Plugin has a lot of goals which are all intended for calling via command line and not within the life cycle.

Take a look at a typical example how the Versions Maven Plugin is used. We have a project where we would like to change the version number. That can easily being done by using the following call:

$ mvn versions:set -DnewVersion=1.2.0 -DgenerateBackupPoms=false
...

So the plugin will change the versions in the pom.xml file. You have to remember the option -DgenerateBackupPoms=false everytime you are calling it otherwise there will backups being created of the original pom.xml files (pom.xml.versionsBackup). Afterwards you have either delete those files manually or with the help of your version control but this situation can be improved by using the configuration for the command line only. The configuration looks like the following:

<project>
  <build>
    <pluginManagement>
      <plugins>
        <plugin>
          <groupId>org.codehaus.mojo</groupId>
          <artifactId>versions-maven-plugin</artifactId>
          <version>2.8.1</version>
          <executions>
            <execution>
              <id>default-cli</id>
              <configuration>
                <generateBackupPoms>false</generateBackupPoms>
              </configuration>
            </execution>
          </executions>
        </plugin>
        ...
      </plugins>
    </pluginManagement>
  </build>
</project>

Now you can make a simpler call like this:

$ mvn versions:set -DnewVersion=2.0.0-SNAPSHOT
[INFO] Scanning for projects...
[INFO] 
[INFO] -------------< com.soebes.articles.plugins:configuration >--------------
[INFO] Building configuration 1.0.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- versions-maven-plugin:2.8.1:set (default-cli) @ configuration ---
[INFO] Local aggregation root: /.../configuration
[INFO] Processing change of com.soebes.articles.plugins:configuration:1.0.0-SNAPSHOT -> 2.0.0-SNAPSHOT
[INFO] Processing com.soebes.articles.plugins:configuration
[INFO]     Updating project com.soebes.articles.plugins:configuration
[INFO]         from version 1.0.0-SNAPSHOT to 2.0.0-SNAPSHOT
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.853 s
[INFO] Finished at: 2021-03-28T20:32:58+02:00
[INFO] ------------------------------------------------------------------------

Configuration for different Command Line Invocation

We have reached an interesting level of configuration options for Maven plugins, unfortunately there is a limitation. You can configure exactly one command line call (<id>default-cli</id>). If you want to make different configuration for different CLI calls it is simply not possible that way.

Starting with Maven 3.3.1+ (in the meantime 5 years ago) there had been an interesting enhancement for goal invocation.

Let us begin with the default-cli based on previous xamples (using exec-maven-plugin here):

<project...>

  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>exec-maven-plugin</artifactId>
        <version>3.0.0</version>
        <executions>
          <execution>
            <id>default-cli</id>
            <configuration>
              <mainClass>com.soebes.test.First</mainClass>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins> 
  </build>
</project>

As described in detail in previous parts you can now call Maven like this:

$ mvn exec:java
...

That will use the above configuration. So now we enhance the example like this:

<project...>

  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>exec-maven-plugin</artifactId>
        <version>3.0.0</version>
        <executions>
          <execution>
            <id>default-cli</id>
            <configuration>
              <mainClass>com.soebes.test.First</mainClass>
            </configuration>
          </execution>
          <execution>
            <id>second-cli</id>
            <configuration>
              <mainClass>com.soebes.test.Second</mainClass>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins> 
  </build>
</project>

Now you can call Maven like this:

$ mvn exec:java@second-cli

I would like to emphasize the @second-cli after the goal java which lets you select which of the given entries (<id>..</id>) will be used while invoking the goal. So now you can make a default configuration via <id>default-cli</id> and several other configuration which can be selected simply by giving the id during the command line invocation.

This opens a new level of command invocation configuration.

You might know the build-helper-maven-plugin which can be used in combination with the versions-maven-plugin to increment your versions without the maven-release-plugin via command line like this:

 mvn build-helper:parse-version versions:set \
     -DnewVersion=\${parsedVersion.nextMajorVersion}.0.0 \
     versions:commit

In this invocation we are calling two different plugins with three goals. The first one is the build-helper-maven-plugin with the goal parse-version and the second versions-maven-plugin using the goal set and finally the versions-maven-plugin with the goal commit.

The parse-version goal reads the information of the pom.xml related to the version tag into plugin properties and produces some supplemental helpful new properties which can be used later for usage on the command line part: -DnewVersion=\${parsedVersion.nextMajorVersion}.0.0.
This will use the property parsedVersion.nextMajorVersion and use it as input into the set goal to define a new major version. This means in the end it will increment the major version by one. The final commit goal will make the change of the pom.xml permanent. This could also being achieved by using the previously mentioned -DgenerateBackupPoms=false property.

As you already seen we have to use an escape before the $ to prevent any kind of shell to expand the information before even calling the plugin goals. Now a more important point is:
Would you remember all the information needed to call that? I don't. There are further things possible like this (increment minor version by one):

mvn build-helper:parse-version versions:set \
     -DnewVersion=\${parsedVersion.majorVersion}.\${parsedVersion.nextMinorVersion}.0 \
     versions:commit

That becomes more and more tedious and error-prone. We can improve that by using a pom.xml
configuration (excerpt).

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>versions-maven-plugin</artifactId>
  <executions>
    <execution>
      <id>major</id>
      <goals>
        <goal>set</goal>
      </goals>
      <configuration>
        <generateBackupPoms>false</generateBackupPoms>
        <newVersion>${parsedVersion.nextMajorVersion}.0.0-SNAPSHOT</newVersion>
      </configuration>
    </execution>
  </executions>
</plugin>

So now you can use the following call to increment the major version of your project:

$ mvn build-helper:parse-version versions:set@major

Much simpler isn't it? Apart from that it's much easier to remember and even simpler to configure
in any kind of CI/CD solution and you don't have to fight with escaping in different environments.
We can enhance further like this:

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>versions-maven-plugin</artifactId>
  <executions>
    <execution>
      <id>major</id>
      <goals>
        <goal>set</goal>
      </goals>
      <configuration>
        <generateBackupPoms>false</generateBackupPoms>
        <newVersion>${parsedVersion.nextMajorVersion}.0.0-SNAPSHOT</newVersion>
      </configuration>
    </execution>
    <execution>
      <id>minor</id>
      <goals>
        <goal>set</goal>
      </goals>
      <configuration>
        <generateBackupPoms>false</generateBackupPoms>
        <newVersion>${parsedVersion.majorVersion}.${parsedVersion.nextMinorVersion}.0-SNAPSHOT</newVersion>
      </configuration>
    </execution>
    <execution>
      <id>patch</id>
      <goals>
        <goal>set</goal>
      </goals>
      <configuration>
        <generateBackupPoms>false</generateBackupPoms>
        <newVersion>${parsedVersion.majorVersion}.${parsedVersion.minorVersion}.${parsedVersion.nextIncrementalVersion}-SNAPSHOT</newVersion>
      </configuration>
    </execution>
  </executions>
</plugin>

So we can increment the minor version of our project via:

$ mvn build-helper:parse-version versions:set@minor

The patch number like this:

$ mvn build-helper:parse-version versions:set@patch

You might have spotted a small issue in the configuration because in each execution the
<generateBackupPoms>false</generateBackupPoms> is duplicated?
That can be solved very easy by refactoring that out like this:

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>versions-maven-plugin</artifactId>
  <configuration>
    <generateBackupPoms>false</generateBackupPoms>
  </configuration>
  <executions>
    <execution>
      <id>major</id>
      <goals>
        <goal>set</goal>
      </goals>
      <configuration>
        <newVersion>${parsedVersion.nextMajorVersion}.0.0-SNAPSHOT</newVersion>
      </configuration>
    </execution>
    <execution>
      <id>minor</id>
      <goals>
        <goal>set</goal>
      </goals>
      <configuration>
        <newVersion>${parsedVersion.majorVersion}.${parsedVersion.nextMinorVersion}.0-SNAPSHOT</newVersion>
      </configuration>
    </execution>
    <execution>
      <id>patch</id>
      <goals>
        <goal>set</goal>
      </goals>
      <configuration>
        <newVersion>${parsedVersion.majorVersion}.${parsedVersion.minorVersion}.${parsedVersion.nextIncrementalVersion}-SNAPSHOT</newVersion>
      </configuration>
    </execution>
  </executions>
</plugin>

These are only some examples how the command line invocation can be tweaked by using the configuration
options in Maven. I hope that helped a bit.

Maven Plugin Testing - In a Modern way - Part VI

Karl Heinz Marbaise — Mon, 28 Sep 2020 18:37:58 +0000

In the previous part of the series - Maven Plugin Testing - In a Modern way - Part V we have seen how to define system properties to run Maven. In this part we will take a deeper look how we can define profile(s) for a Maven call to be used.

Let us take a look at a simple example taken from the previous part.

@MavenJupiterExtension
class BaseIT {

  @MavenTest
  @MavenGoal("verify")
  void the_first_test_case(MavenExecutionResult result) {
     ...
  }
}

This will run Maven with the goal verify. In real life it makes sometimes sense to run integration tests only by activating a profile like this:

mvn verify -Prun-its

So the question is now: How can we do that in the Integration Testing Framework]itf? This can be achieved by using the @MavenProfile annotation like this:

@MavenJupiterExtension
class BaseIT {

  @MavenTest
  @MavenGoal("verify")
  @MavenProfile("run-its")
  void the_first_test_case(MavenExecutionResult result) {
     ...
  }
}

This would execute our integration with the following command line (except the things we mentioned in previous parts of the series; Just omitted them for brevity.):

mvn verify -Prun-its

In the following example you can see that the @MavenProfile annotation used only on the method first:

@MavenJupiterExtension
class BaseIT {

  @MavenTest
  @MavenGoal("verify")
  @MavenProfile("run-its")
  void first(MavenExecutionResult result) {
     ...
  }

  @MavenTest
  void second(MavenExecutionResult result) {
     ...
  }

  @MavenTest
  void third(MavenExecutionResult result) {
     ...
  }
}

This will execute all above integration test cases except the first without a profile.

The @MavenProfile annotation can be put on different methods for running a different profile within different test cases like this:

@MavenJupiterExtension
class BaseIT {

  @MavenTest
  @MavenGoal("verify")
  @MavenProfile("run-its")
  void first(MavenExecutionResult result) {
     ...
  }

  @MavenTest
  void second(MavenExecutionResult result) {
     ...
  }

  @MavenTest
  @MavenProfile("run-e2e")
  void third(MavenExecutionResult result) {
     ...
  }
}

The above example would execute the test case third with the given profile run-e2e activated on command line and of course the first test case with the given profile run-its.

There are cases where it is needed to run a bunch of integration tests with the same profile which can be achieved by defining the @MavenProfile annotation on the class level like the following:

@MavenJupiterExtension
@MavenProfile("run-its")
class BaseIT {

  @MavenTest
  @MavenGoal("verify")
  void first(MavenExecutionResult result) {
     ...
  }

  @MavenTest
  void second(MavenExecutionResult result) {
     ...
  }

  @MavenTest
  void third(MavenExecutionResult result) {
     ...
  }
}

This will result in executing all three test cases using the same profile. Now let us think about another scenario. You have defined a set of integration tests (let us assume 25)
and create the appropriate test class based on the pattern of the above. Now you have to add a single test case which should not use the profile which is defined on the class level. It should use a different one.

There are two solutions for this. The first one would be to create a separate integration test class which defines the appropriate profile on the class- or on the method level (it is a matter of taste). The other solution would be to define another test case method with another @MavenProfile annotation on it as the following example shows:

@MavenJupiterExtension
@MavenProfile("run-its")
class BaseIT {

  @MavenTest
  @MavenGoal("verify")
  void first(MavenExecutionResult result) {
     ...
  }

  @MavenTest
  @MavenProfile("run-somethingdifferent")
  void second(MavenExecutionResult result) {
     ...
  }

  @MavenTest
  void third(MavenExecutionResult result) {
     ...
  }
}

The result will be to execute the test cases first and third with the -Prun-its profile and the second test case will be executed with the run-somethingdifferent profile. In other words it means the defined profiles
are not additive. Let me show an example of a limitation of this concept. If you would like to have the second test case executed without a profile at all this is not possible.
This would lead into the solution having to define a separate class instead.

Apart from that there are cases where it is useful to define several profiles on a Maven build like this:

mvn -Prun-its,run-e2e

This can also done via using several @MavenProfile annotations on the appropriate test case (or on the class level) like this:

@MavenJupiterExtension
class BaseIT {

  @MavenTest
  @MavenGoal("verify")
  @MavenProfile("run-its")
  @MavenProfile("run-e2e")
  void the_first_test_case(MavenExecutionResult result) {
     ...
  }
}

So based on the previous parts of the series you could conclude that it is possible to define your own meta annotation to make combinations easier.

@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RUNTIME)
@Inherited
@MavenProfile("run-its")
@MavenProfile("run-e2e")
public @interface ExecuteIntegrationAndEndToEnd {
}

This is not limited to the @MavenProfile annotation. You can combine that with the annotation like @MavenGoal etc. as mentioned in the previous articles of the series.

So this it is for Part VI. If you like to learn more about the Integration Testing Framework you can consult the users guide. If you like to know the state of the release you can take a look into the release notes.

If you have ideas, suggestions or found bugs please file in an issue on github.

An example project which shows the previous example can be found on GitHub.

Spring Boot: Strategy Design Pattern - Convenience and Limitation

Karl Heinz Marbaise — Sun, 20 Sep 2020 13:56:10 +0000

You might have already used the strategy pattern in relationship with Spring Boot where it is very convenient to use.

You simply define an interface for example (I use the prefixing I only in these examples for clarity):

public interface IOneStrategy {
  void executeTheThing();
}

Define some implementations like this:

@Service("FIRST")
public class OneStrategyFirst implements IOneStrategy {

  @Override
  public void executeTheThing() {
    System.out.println("OneStrategyFirst.executeTheThing");
  }
}

Now you can simply implement a service which will execute the appropriate strategy based on the given name which looks similar like this:

@Service
public class ExecuteStrategyOne {
  private Map<String, IOneStrategy> strategies;

  public ExecuteStrategyOne(Map<String, IOneStrategy> strategies) {
    this.strategies = strategies;
  }

  public void executeStrategyOne(String name) {
    if (!strategies.containsKey(name)) {
      throw new IllegalArgumentException("The strategy " + name + " does not exist.");
    }
    strategies.get(name).executeTheThing();
  }

}

In real world you make several implementations of the strategy interface like OneStrategyFirst, OneStrategySecond and OneStrategyThird. Sometimes the usage is to use the parameter of executeStrategyOne which is provided by a REST API or some other domain specific code which needs different implementations.

The convenience here is that Spring Boot (Spring Framework to be more accurate) handles the injection of the different implementation into the strategies Map within ExecuteStrategyOne via the constructor. This results in a Map where the key is the value which is given by @Service("FIRST") and the value of the map contains an instantiates class of every implementation of the interface IOneStrategy which can be found.

Really convenient.

In real life it happens that you need to have a different strategy which use the same keys as FIRST, SECOND and THIRD in the examples? Let us define the following:

@Service("FIRST")
public class TwoStrategyFirst implements ITwoStrategy {

  @Override
  public void executeTheThing() {
    System.out.println("TwoStrategyFirst.executeTheThing");
  }
}

If you try to start that Spring Boot application you will see an exception like this:

Caused by: org.springframework.context.annotation.ConflictingBeanDefinitionException: 
Annotation-specified bean name 'FIRST' for bean class [com.soebes.examples.strategies.functions.two.TwoStrategyFirst] 
conflicts with existing, non-compatible bean definition of same name and class 
[com.soebes.examples.strategies.functions.one.OneStrategyFirst]
    at org.springframework.context.annotation.ClassPathBeanDefinitionScanner.checkCandidate(ClassPathBeanDefinitionScanner.java:349) ~[spring-context-5.2.9.RELEASE.jar:5.2.9.RELEASE]
    at org.springframework.context.annotation.ClassPathBeanDefinitionScanner.doScan(ClassPathBeanDefinitionScanner.java:287) ~[spring-context-5.2.9.RELEASE.jar:5.2.9.RELEASE]

So what can we do to solve the problem without losing much of the convenience which Spring Boot provides us here?

First we have to define in each of the strategy implementation class the annotations like this:

@Service
@Qualifier("FIRST")
public class TwoStrategyFirst implements ITwoStrategy {

  @Override
  public void executeTheThing() {
    System.out.println("TwoStrategyFirst.executeTheThing");
  }
}

By using the key in a different annotation we prevent the duplication of the bean names in contradiction to use @Service("FIRST") instead. The usage of @Qualifier("FIRST") gives us a criteria to handle that different.

Now we have to change the ExecuteStrategyOne class like the following:

@Service
public class ExecuteStrategyOne {

  private Map<String, IOneStrategy> strategies;

  public ExecuteStrategyOne(List<IOneStrategy> strategies) {
    this.strategies = strategies.stream()
        .collect(
            toMap(k -> k.getClass().getDeclaredAnnotation(Qualifier.class).value(), 
                  Function.identity()));
  }
  ...
}

I would like to emphasis the usage of the constructor parameter List<IOneStrategy> strategies instead of the previously used Map<String, IOneStrategy> strategies which is a convenience to get a list of all implementations off the given interface into that list by Spring Boot. Now we need to translate that into a map with the key we have defined by using @Qualifier annotation. The whole thing can be solved by a stream like this:

this.strategies = strategies
  .stream()
  .collect(
    Collectors.toMap(k -> k.getClass().getDeclaredAnnotation(Qualifier.class).value(), 
                     Function.identity()));

We go through the implementations and extract the annotation @Qualifier and read out the value() which is the key we want to have. We collect the result by using the Collectors.toMap into a Map and assign the result to the instance variable private Map<String, IOneStrategy> strategies;.
Depending on your need it is of course possible to define the instance variable as final and you can create an unmodifiable map by using the appropriate Collectors.toUnmodifiableMap instead of the toMap(..) if needed.

So with a few changes in the code we can easily solve the problem of having different strategies which using the same keys in our code.

The given code is available as a full working example on GitHub.

Maven Plugin Testing - In a Modern way - Part V

Karl Heinz Marbaise — Thu, 10 Sep 2020 17:40:51 +0000

In the previous part of the series - Maven Plugin Testing - In a Modern way - Part IV we have seen how to define goals to run Maven. In this part we will take a deeper look how we can define system properties for a Maven call to be used.

Let us take a look at a simple example taken from the previous part.

@MavenJupiterExtension
class BaseIT {

  @MavenTest
  @MavenGoal("verify")
  void the_first_test_case(MavenExecutionResult result) {
     ...
  }
}

This will run Maven with the goal verify. So what could you do to prevent running the unit tests or even any tests at all within that integration test case? What would you usually do on plain command line? You would have added -DskipTests to your Maven command line. How could you do the above integration test example? Very easy like this:

@MavenJupiterExtension
class PropertyIT {

  @MavenTest
  @MavenGoal("verify")
  @SystemProperty("skipTests")
  void goal_clean_skiptests (MavenExecutionResult result) {
     ...
  }
}

That means in the end that Maven will be called like the following:

mvn -DskipTests verify

Let us go a step further and see how to write a test case for a plugin which should be called like this:

mvn ...:failure -DexecutionException=true -Dexception="This is the value of exception."

The above could be expressed like this:

@MavenJupiterExtension
class PropertyIT {

  @MavenTest
  @MavenGoal("clean")
  @MavenGoal("${project.groupId}:${project.artifactId}:${project.version}:failure")
  @SystemProperty(value = "executionException", content = "true")
  @SystemProperty(value = "exception", content = "This is the value of exception.")
  void goal_failure_execution(MavenExecutionResult result) {
     ...
  }
}

You can see two things here. First you can repeat the @SystemProperty to define multiple system properties and second you can make a clear separation between the name and property value.

As you might already expected you can create a meta annotation where you can combine the properties used in the previous example into this single meta annotation:

@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RUNTIME)
@Inherited
@SystemProperty(value = "executionException", content = "true")
@SystemProperty(value = "exception", content = "This is the value of exception.")
public @interface ExecutionException {
}

If reconsider this it's thought a bit too short, because that can be made better like this:

@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RUNTIME)
@Inherited
@MavenTest
@MavenGoal("clean")
@MavenGoal("${project.groupId}:${project.artifactId}:${project.version}:failure")
@SystemProperty(value = "executionException", content = "true")
@SystemProperty(value = "exception", content = "This is the value of exception.")
public @interface FailureGoalWithExecutionException {
}

So the defined meta annotation can now being used to make the test looking like this:

@MavenJupiterExtension
class PropertyIT {

  @FailureGoalWithExecutionException
  void goal_failure_execution(MavenExecutionResult result) {
     ...
  }
}

You can define the previous described meta annotation in the example project and see how it feels to use it.

Based on convenience you can of course create a meta annotation like this:

@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RUNTIME)
@Inherited
@SystemProperty("skipTests")
public @interface MavenSkipTests {
}

or other things. If you have good ideas do not hesitate to file in a feature request for the project so it can be made permanent part of it.

So this it is for Part V. If you like to learn more about the Integration Testing Framework
you can consult the users guide. If you like to know the state of the release you can take a look into the release notes.

If you have ideas, suggestions or found bugs please file in an issue on github.

An example project which shows the previous example can be found on GitHub.

Maven Plugin Testing - In a Modern way - Part IV

Karl Heinz Marbaise — Sun, 30 Aug 2020 17:53:47 +0000

In the previous part of the series - Maven Plugin Testing - In a Modern way - Part III we have seen how to define command line options. In this part we will take a deeper look which goals will run for each test case and how we can change that.

Let us start with simple example test case like the following:

@MavenJupiterExtension
class BaseIT {

  @MavenTest
  void the_first_test_case(MavenExecutionResult result) {
     ...
  }
}

If we ran that integration test Maven will be called like the following:

mvn -Dmaven.repo.local=<Directory> --batch-mode --show-version --errors package

We will concentrate on the part package in the above example. This is a life cycle phase of Maven. So what can we do if we like to call something like mvn .. verify instead? This can simply being achieved by using the @MavenGoal annotation like this:

@MavenJupiterExtension
class BaseIT {

  @MavenTest
  @MavenGoal("verify")
  void the_first_test_case(MavenExecutionResult result) {
     ...
  }
}

So let us take a deeper look onto the following example:

@MavenJupiterExtension
class BaseIT {

  @MavenTest
  @MavenGoal("verify")
  void first(MavenExecutionResult result) {
     ...
  }

  @MavenTest
  void second(MavenExecutionResult result) {
     ...
  }

  @MavenTest
  void third(MavenExecutionResult result) {
     ...
  }
}

The test case first will be called with the phase verify whereas second and third will be called with the package. So this means you can overwrite the default behaviour for each test case separately.

Sometimes you want to execute Maven during an integration test like this: mvn clean verify or in general with multiple life cycle phase. This can be achieved by using multiple MavenGoal annotations as in the following example:

@MavenJupiterExtension
class BaseIT {

  @MavenTest
  @MavenGoal("clean")
  @MavenGoal("verify")
  void first(MavenExecutionResult result) {
     ...
  }
  ...
}

Of course there are situations where you have a bunch of integration tests which needed to be executing the previously defined goals. This can handled by defining the @MavenGoal annotation on a class level instead like this:

@MavenJupiterExtension
@MavenGoal("clean")
@MavenGoal("verify")
class BaseIT {

  @MavenTest
  void first(MavenExecutionResult result) {
     ...
  }

  @MavenTest
  void second(MavenExecutionResult result) {
     ...
  }

  @MavenTest
  void third(MavenExecutionResult result) {
     ...
  }
}

This also gives the opportunity to let run a single test (or more than one) case within a test class with different goals depending on what you like to achieve.

Another example on how to define the @MavenGoal annotation on a class level which looks like this:

@MavenJupiterExtension
@MavenGoal("clean")
class GoalsOnClassIT {

  @MavenTest
  @DisplayName("This will check the goal which is defined on the class.")
  void goal_clean(MavenExecutionResult result) {
    assertThat(result)
        .isSuccessful()
        .out()
        .info()
        .containsSubsequence(
            "Scanning for projects...",
            "-------------------< com.soebes.katas:kata-fraction >-------------------",
            "Building kata-fraction 1.0-SNAPSHOT",
            "--------------------------------[ jar ]---------------------------------",
            "--- maven-clean-plugin:3.1.0:clean (default-clean) @ kata-fraction ---"
        );
    assertThat(result)
        .isSuccessful()
        .out()
        .warn().isEmpty();
  }
}

The next logical step is to create a meta annotation to make life easier. We would like to combine clean and verify within a single annotation @GoalsCleanVerify which can be done like this:

@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RUNTIME)
@Inherited
@MavenGoal({"clean", "verify"})
public @interface GoalsCleanVerify {
}

Such kind of meta annotation can be used on class level (defined by the annotation itself) as well as on method level like this:

@MavenJupiterExtension
@GoalsCleanVerify
class MetaAnnotationGoalIT {
 ...
}

So now lets think about Part III where we defined this meta annotation:

@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RUNTIME)
@Inherited
@MavenOption(MavenCLIOptions.FAIL_AT_END)
@MavenOption(MavenCLIOptions.NON_RECURSIVE)
@MavenOption(MavenCLIOptions.ERRORS)
@MavenOption(MavenCLIOptions.DEBUG)
public @interface MavenTestOptions {
}

This meta annotation can now being enhanced by the needed @MavenGoal annotations.

@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RUNTIME)
@Inherited
@MavenOption(MavenCLIOptions.FAIL_AT_END)
@MavenOption(MavenCLIOptions.NON_RECURSIVE)
@MavenOption(MavenCLIOptions.ERRORS)
@MavenOption(MavenCLIOptions.DEBUG)
@MavenGoal("clean")
@MavenGoal("verify")
public @interface MavenTestOptions {
}

This means you can define easily your set of annotation or combination of command line options and goals or more sophisticated combine it with @MavenJupiterExtension
like this:

@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RUNTIME)
@Inherited
@MavenJupiterExtension
@MavenOption(MavenCLIOptions.FAIL_AT_END)
@MavenOption(MavenCLIOptions.NON_RECURSIVE)
@MavenOption(MavenCLIOptions.ERRORS)
@MavenOption(MavenCLIOptions.DEBUG)
@MavenGoal("clean")
@MavenGoal("verify")
public @interface MavenTestOptions {
}

This will give us the option to use it like this:

@MavenTestOptions
class FailureIT {

  @MavenTest
  void case_one(MavenExecutionResult project) {
    ..
  }

  @MavenTest
  void case_two(MavenExecutionResult result) {
    ..
  }

  @MavenTest
  void case_three(MavenExecutionResult result) {
    ..
  }

  @MavenTest
  @MavenOption(MavenCLIOptions.DEBUG)
  void case_four(MavenExecutionResult result) {
    ..
  }

}

This combines the given options with the defined goals in one single annotation. If you need to change something you have to fix only a single point.

So did we miss something? Yes we did. Sometimes you want to call your plugin with a separate goal like this:

mvn org.test.maven.plugin:maven-x-plugin:goal

This can be achieved by using the @MavenGoal annotation like this:

@MavenJupiterExtension
class BaseIT {

  @MavenTest
  @MavenGoal("org.test.maven.plugin:maven-x-plugin:goal")
  void first(MavenExecutionResult result) {
     ...
  }
  ...
}

Now let us assume the given plugin is the one which should be tested via the given integration test. Then you need to define the correct groupId, artifactId, version and the
correct goal. Unfortunately with each release of your plugin the version changes etc.

@MavenJupiterExtension
class BaseIT {

  @MavenTest
  @MavenGoal("${project.groupId}:${project.artifactId}:${project.version}:goal-to-test")
  void first(MavenExecutionResult result) {
     ...
  }
  ...
}

The placeholders ${project.groupId}, ${project.artifactId} and ${project.version} are
exactly the information from your project pom in which you define the coordinates of the plugin your are developing. This makes sure only this plugin version is being used
and not any other version is being tried to download from central or other repositories during your integration tests.

Good examples can be found in
maven-invoker-plugin based integration tests.

So this it is for Part IV. If you like to learn more about the Integration Testing Framework you can consult the users guide. If you like to know the state of the release you can take a look into the release notes.

If you have ideas, suggestions or found bugs please file in an issue on github.

An example project which shows the previous example can be found on GitHub.

Maven Plugin Testing - In a Modern way - Part III

Karl Heinz Marbaise — Tue, 25 Aug 2020 20:51:54 +0000

In the second part of the series - Maven Plugin Testing - In a Modern way - Part II we have seen how to make the basic integration test while checking the log output of Maven builds.

In this third part we will dive into how Maven will be called by default during the integration tests and how we can influence that behaviour.

Let us begin with the following basic integration test (we ignore the project which is used for testing at the moment.)

@MavenJupiterExtension
class BaseIT {

  @MavenTest
  void the_first_test_case(MavenExecutionResult result) {
     ...
  }
}

The above test will execute Apache Maven by using the following options by default:

--batch-mode
--show-version
--errors

That means that each execution within the Integration Testing Framework will be done like this:

mvn --batch-mode --show-version --errors

To get that correctly working and in particular having a local cache for each integration test case the integration testing framework will add: -Dmaven.repo.local=... to each call as well. This is necessary to get the following result:

.
└──target/
   └── maven-it/
       └── org/
           └── it/
               └── FirstMavenIT/
                   └── the_first_test_case/
                       ├── .m2/
                       ├── project/
                       │   ├── src/
                       │   ├── target/
                       │   └── pom.xml
                       ├── mvn-stdout.log
                       ├── mvn-stderr.log
                       └── mvn-arguments.log

The option -Dmaven.repo.local=... can't be changed at the moment. The used command line arguments are wrote into the mvn-arguments.log file which can be consulted for later analysis. So in the end a command line for an integration test looks like this:

mvn -Dmaven.repo.local=<Directory> --batch-mode --show-version --errors package

The goal which is used (package) and how it can be changed will be handled in Part IV of the series.

So far so good, but sometimes or maybe more often it is needed to add other command line options to a Maven call in particular in relationship with integration tests.

Sometimes your own plugin/extension will printout some useful information on debug level but how to test that? We have mentioned before that the options which are used for an integration test does not contain the debug option.

We can now express the need via this(basic_configuration_with_debug):

@MavenJupiterExtension
class FailureIT {
  ...
  @MavenTest
  @MavenOption(MavenCLIOptions.DEBUG)
  void basic_configuration_with_debug(MavenExecutionResult result) {
    assertThat(result)
        .isSuccessful()
        .out()
        .info()
        .containsSubsequence(
            "--- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven) @ basic_configuration_with_debug ---",
            "--- jacoco-maven-plugin:0.8.5:prepare-agent (default) @ basic_configuration_with_debug ---",
            "--- maven-resources-plugin:3.1.0:resources (default-resources) @ basic_configuration_with_debug ---",
            "--- maven-compiler-plugin:3.8.1:compile (default-compile) @ basic_configuration_with_debug ---",
            "--- maven-resources-plugin:3.1.0:testResources (default-testResources) @ basic_configuration_with_debug ---",
            "--- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ basic_configuration_with_debug ---",
            "--- maven-surefire-plugin:3.0.0-M4:test (default-test) @ basic_configuration_with_debug ---",
            "--- maven-jar-plugin:3.2.0:jar (default-jar) @ basic_configuration_with_debug ---",
            "--- maven-site-plugin:3.9.1:attach-descriptor (attach-descriptor) @ basic_configuration_with_debug ---"
        );
    assertThat(result)
        .isSuccessful()
        .out()
        .warn()
        .containsSubsequence(
            "Neither executionException nor failureException has been set.",
            "JAR will be empty - no content was marked for inclusion!");

    assertThat(result)
        .isSuccessful()
        .out()
        .debug()
        .containsSubsequence(
            "Created new class realm maven.api",
            "Project: com.soebes.itf.maven.plugin.its:basic_configuration_with_debug:jar:1.0",
            "Goal:          org.apache.maven.plugins:maven-resources-plugin:3.1.0:resources (default-resources)"
        );

  }
}

It's important to say that by using the @MavenOption(..) automatically all other previously mentioned command line options will not being used anymore. In this example the final command line looks like this for the test case
basic_configuration_with_debug:

mvn -Dmaven.repo.local=<path> --debug package

So based on turning on debugging output it means that you can check debugging output like this:

assertThat(result)
    .isSuccessful()
    .out()
    .debug()
    .containsSubsequence(
        "Created new class realm maven.api",
        "Project: com.soebes.itf.maven.plugin.its:basic_configuration_with_debug:jar:1.0",
        "Goal:          org.apache.maven.plugins:maven-resources-plugin:3.1.0:resources (default-resources)"
    );

If you like to have the --batch-mode option, --show-version as well as the --error option back in your test case have to add them like this:

  @MavenTest
  @MavenOption(MavenCLIOptions.BATCH_MDOE)
  @MavenOption(MavenCLIOptions.SHOW_VERSION)
  @MavenOption(MavenCLIOptions.ERRORS)
  @MavenOption(MavenCLIOptions.DEBUG)
  void basic_configuration_with_debug(MavenExecutionResult result) {
  ...
  }

The result will be that your Maven command line now looks like before including the supplemental --debug option:

mvn -Dmaven.repo.local=<Directory> --batch-mode --show-version --errors --debug package

That shows that you can easily combine several command line options for a test case via the @MavenOption annotation.

Some command line options in Maven need supplemental information like --log-file <arg> which requires the name of the log file to redirect all the output into. How can we express this with the @MavenOption annotation? This can simply being achieved like the following:

  @MavenTest
  @MavenOption(QUIET)
  @MavenOption(SHOW_VERSION)
  @MavenOption(value = LOG_FILE, parameter = "test.log")
  void basic_configuration_with_debug(MavenExecutionResult result) {
  ...
  }

As you can see in the above example you have to give the option via the value of the annotation. The parameter of the option has to be given via parameter. In this case we have used static imports to make it more readable. You can of
course work without static imports like this (Just a matter of taste):

  @MavenTest
  @MavenOption(MavenCLIOptions.QUIET)
  @MavenOption(MavenCLIOptions.SHOW_VERSION)
  @MavenOption(value = MavenCLIOptions.LOG_FILE, parameter = "test.log")
  void basic_configuration_with_debug(MavenExecutionResult result) {
  ...
  }

So what about using the same command line options for several test cases? You can simply add the command line options onto the test class level which looks like this:

@MavenJupiterExtension
@MavenOption(MavenCLIOptions.FAIL_AT_END)
@MavenOption(MavenCLIOptions.NON_RECURSIVE)
@MavenOption(MavenCLIOptions.ERRORS)
@MavenOption(MavenCLIOptions.DEBUG)
class FailureIT {

  @MavenTest
  void case_one(MavenExecutionResult project) {
    ..
  }

  @MavenTest
  void case_two(MavenExecutionResult result) {
    ..
  }

  @MavenTest
  void case_three(MavenExecutionResult result) {
    ..
  }

  @MavenTest
  void case_four(MavenExecutionResult result) {
    ..
  }

}

This means that for each given test case ( case_one, case_two, case_three and case_four) the same set of command line options will be used. Apart from that it is much more convenient to define the command line options only once
and not for every single test case.

Wait a second. I want to execute case_four with different command line options? Ok no problem just define the set command line options onto that particular test case like in the following example:

@MavenJupiterExtension
@MavenOption(MavenCLIOptions.FAIL_AT_END)
@MavenOption(MavenCLIOptions.NON_RECURSIVE)
@MavenOption(MavenCLIOptions.ERRORS)
@MavenOption(MavenCLIOptions.DEBUG)
class FailureIT {

  @MavenTest
  void case_one(MavenExecutionResult project) {
    ..
  }

  @MavenTest
  void case_two(MavenExecutionResult result) {
    ..
  }

  @MavenTest
  void case_three(MavenExecutionResult result) {
    ..
  }

  @MavenTest
  @MavenOption(MavenCLIOptions.DEBUG)
  void case_four(MavenExecutionResult result) {
    ..
  }

}

The test case case_four will not inherit the command line options defined on the class level. It will use only those defined on the test case itself.

Now a usual developer question: I'm really lazy I don't want to write all those four MavenOption annotation for each test class. In particular if I need to change those command line options I have to go through each test class one by one?

There is of course a more convenient solution for that problem. This is called a meta annotation. We can simply create a meta annotation which we call MavenTestOptions. This meta annotation looks like this:

@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RUNTIME)
@Inherited
@MavenOption(MavenCLIOptions.FAIL_AT_END)
@MavenOption(MavenCLIOptions.NON_RECURSIVE)
@MavenOption(MavenCLIOptions.ERRORS)
@MavenOption(MavenCLIOptions.DEBUG)
public @interface MavenTestOptions {
}

This meta annotation can now being used to change the test class of the previous example like this:

@MavenJupiterExtension
@MavenTestOptions
class FailureIT {

  @MavenTest
  void case_one(MavenExecutionResult project) {
    ..
  }

  @MavenTest
  void case_two(MavenExecutionResult result) {
    ..
  }

  @MavenTest
  void case_three(MavenExecutionResult result) {
    ..
  }

  @MavenTest
  @MavenOption(MavenCLIOptions.DEBUG)
  void case_four(MavenExecutionResult result) {
    ..
  }

}

The above can be improved a bit more. We can integrate the annotation @MavenJupiterExtension into our self defined meta annotation like this (In the example project I have named the meta annotation @MavenITExecution
to have different examples in one project.):

@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RUNTIME)
@Inherited
@MavenJupiterExtension
@MavenOption(MavenCLIOptions.FAIL_AT_END)
@MavenOption(MavenCLIOptions.NON_RECURSIVE)
@MavenOption(MavenCLIOptions.ERRORS)
@MavenOption(MavenCLIOptions.DEBUG)
public @interface MavenTestOptions {
}

Based on that we can change the test case to look like this:

@MavenTestOptions
class FailureIT {

  @MavenTest
  void case_one(MavenExecutionResult project) {
    ..
  }

  @MavenTest
  void case_two(MavenExecutionResult result) {
    ..
  }

  @MavenTest
  void case_three(MavenExecutionResult result) {
    ..
  }

  @MavenTest
  @MavenOption(MavenCLIOptions.DEBUG)
  void case_four(MavenExecutionResult result) {
    ..
  }

}

So this it is for Part III. If you like to learn more about the Integration Testing Framework you can consult the users guide. If you like to know the state of the release you can take a look into the release notes.

If you have ideas, suggestions or found bugs please file in an issue on github.

An example of the shown uses cases can be found on GitHub.

Maven Plugin Testing - In a Modern way - Part II

Karl Heinz Marbaise — Fri, 21 Aug 2020 11:06:35 +0000

In the first part of the series - Maven Plugin Testing - In a Modern way - Part I
we have seen how to make the basic setup with The Integration Testing Framework and run very basic integration test.

In this second part we will take a deeper look into other aspects of testing Maven plugins in particular how we check the logging output of a Maven build process.

Let us begin with writing more than a single integration test case. You can of course write multiple test cases within a single test class like the following:

@MavenJupiterExtension
class SeveralMavenIT {

  @MavenTest
  void the_first_test_case(MavenExecutionResult result) {
     ...
  }
  @MavenTest
  void the_second_test_case(MavenExecutionResult result) {
     ...
  }
  @MavenTest
  void the_third_test_case(MavenExecutionResult result) {
     ...
  }
}

Apart from the test cases them self we need the according projects which are used as test projects which looks like this:

.
└── src/
    └── test/
        └── resources-its/
            └── org/
                └── it/
                    └── SeveralMavenIT/
                        ├── the_first_test_case/
                        │   ├── src/
                        │   └── pom.xml
                        ├── the_second_test_case/
                        │   ├── src/
                        │   └── pom.xml
                        └── the_this_test_case/
                            ├── src/
                            └── pom.xml

So after we have executed the integration tests (mvn verify) the resulting directory structure will look like this:

.
└──target/
   └── maven-it/
       └── org/
           └── it/
               └── SeveralMavenIT/
                   ├── the_first_test_case/
                   │   ├── .m2/
                   │   ├── project/
                   │   │   ├── src/
                   │   │   ├── target/
                   │   │   └── pom.xml
                   │   ├── mvn-stdout.log
                   │   ├── mvn-stderr.log
                   │   └── other logs
                   ├── the_second_test_case/
                   │   ├── .m2/
                   │   ├── project/
                   │   │   ├── src/
                   │   │   ├── target/
                   │   │   └── pom.xml
                   │   ├── mvn-stdout.log
                   │   ├── mvn-stderr.log
                   │   └── mvn-arguments.log
                   └── the_third_test_case/
                       ├── .m2/
                       ├── project/
                       │   ├── src/
                       │   ├── target/
                       │   └── pom.xml
                       ├── mvn-stdout.log
                       ├── mvn-stderr.log
                       └── mvn-arguments.log

Based on the resulting directory structure you can see each test case completely separated from each other. This means also that each test case contains it's own maven cache (.m2/repository). You can find also the separated log file outputs and the separate project directory which contains the test project after the test run. That is very helpful for later issue analysis.

So now let us take a deeper look into the test cases:

  @MavenTest
  void the_first_test_case(MavenExecutionResult result) {

  }

In each test you have seen a parameter to the test method MavenExecutionResult result. The injected parameter gives you access to the result of the test build of project.

This class contains appropriate methods to access the result of the build process, the project cache, the project itself (in other words to the directory) and of course to the different logging output files which have been created during the run of the test.

So the first thing you usually check would be if the built has been successful or not. This depends on the type of integration test you are writing. This can be achieved by using the following:

assertThat(result).isSuccessful();

This expects the built being successful as you already suspected. You can of course write a test which assumes that your built must fail which can be expressed like this:

assertThat(result).isFailure();

So the isSuccessful() means the return code 0 whereas isFailure() represents a return code which is not 0.

You can combine the check for a successful build and no warning output like this:

assertThat(result)
    .isSuccessful()
    .out()
    .warn().isEmpty();

So .out() will access the created output file of the build mvn-stdout.log. The .warn() will filter out all lines which start with [WARNING]. The .isEmpty() is part of AssertJ framework for assertion against lists which implies that the result is empty.

So now let us check some output which is produced by a usual build. The output emits [INFO] so the test can use .out().info(). instead which looks like this:

@MavenJupiterExtension
class FirstIT {
  void base_test (MavenExecutionResult result) {
    assertThat(result)
        .isSuccessful()
        .out()
        .info()
        .containsSubsequence(
            "--- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven) @ kata-fraction ---",
            "--- jacoco-maven-plugin:0.8.5:prepare-agent (default) @ kata-fraction ---",
            "--- maven-resources-plugin:3.1.0:resources (default-resources) @ kata-fraction ---",
            "--- maven-compiler-plugin:3.8.1:compile (default-compile) @ kata-fraction ---",
            "--- maven-resources-plugin:3.1.0:testResources (default-testResources) @ kata-fraction ---",
            "--- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ kata-fraction ---",
            "--- maven-surefire-plugin:3.0.0-M4:test (default-test) @ kata-fraction ---",
            "--- maven-jar-plugin:3.2.0:jar (default-jar) @ kata-fraction ---",
            "--- maven-site-plugin:3.9.1:attach-descriptor (attach-descriptor) @ kata-fraction ---"
        );
  }
}

The .containsSubsequence(..) checks that the sequence is in the correct order with optional supplemental parts in between.

While writing plugins/extensions it sometimes happens that you emit an information on WARN Level to give some hints what needs to be mentioned but will not fail a build.

The maven-jar-plugin for example will emit a warning if no content will be added into the jar. This could be checked like the following:

assertThat(result)
    .isSuccessful()
    .out()
    .warn()
    .contains("JAR will be empty - no content was marked for inclusion!");

So this can be combined to create a more comprehensive test case like this:

@MavenJupiterExtension
class FailureIT {

  @MavenTest
  void basic_configuration_checking_logout(MavenExecutionResult result) {
    assertThat(result)
        .isSuccessful()
        .out()
        .info()
        .containsSubsequence(
            "--- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven) @ basic_configuration_checking_logout ---",
            "--- jacoco-maven-plugin:0.8.5:prepare-agent (default) @ basic_configuration_checking_logout ---",
            "--- maven-resources-plugin:3.1.0:resources (default-resources) @ basic_configuration_checking_logout ---",
            "--- maven-compiler-plugin:3.8.1:compile (default-compile) @ basic_configuration_checking_logout ---",
            "--- maven-resources-plugin:3.1.0:testResources (default-testResources) @ basic_configuration_checking_logout ---",
            "--- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ basic_configuration_checking_logout ---",
            "--- maven-surefire-plugin:3.0.0-M4:test (default-test) @ basic_configuration_checking_logout ---",
            "--- maven-jar-plugin:3.2.0:jar (default-jar) @ basic_configuration_checking_logout ---",
            "--- maven-site-plugin:3.9.1:attach-descriptor (attach-descriptor) @ basic_configuration_checking_logout ---"
        );
    assertThat(result)
        .isSuccessful()
        .out()
        .warn()
        .contains("JAR will be empty - no content was marked for inclusion!");
  }
}

If you write a plugin which contains a parameter for encoding an output like this (might look familiar to you) should be produced:

Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!

This can be checked within a test case like this:

assertThat(result)
  .out()
  .warn()
  .containsExactly("Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!");

So this it is for Part II. If you like to learn more about the Integration Testing Framework you can consult the users guide. If you like to know the state of the release you can take a look into the release notes.

If you have ideas, suggestions or found bugs please file in an issue on github.

An example can be found on GitHub.