Property-Based Matrix Testing in Java

I've used a common approach for property-based matrix testing in Java. Using JUnit's ParameterizedTest and a MethodSource annotations to point to a method as an argument supplier that spits out a stream representing a matrix.

It's pretty straightforward, but the more parameter types we have in our matrix, the harder it is to read or write the method supplying them. 😵

Let's take a look.
First, these are the two types for our matrix, these should spit out 6 test cases:

  enum Direction {
    INCOMING,
    OUTGOING
  }

  enum Status {
    SUCCESS,
    FAILURE,
    WAITING
  }

Implementing a matrix from these Enums with JUnit's ParameterizedTest and a MethodSource:

  @ParameterizedTest
  @MethodSource("getArguments")
  void using_junit_parameterized_test_with_method_source(
      final Direction direction, final Status status) {
    assertTrue(true);
  }

  static Stream<Arguments> getArguments() {
    return Stream.of(Direction.values())
        .flatMap(d -> Stream.of(Status.values()).map(s -> arguments(d, s)));
  }

Adding members to the existing Enums will dynamically increase the matrix and, therefore, the number of tests performed; there's no need to modify the test code.

But, adding a third type to the matrix and the getArguments method will start losing its readability.

Lately, I discovered JUnit Pioneer, which is a JUnit 5 Extension Pack. Using its CartesianProductTest and CartesianEnumSource annotations, we can implement the same matrix simply and elegantly: 😃

  @CartesianProductTest
  @CartesianEnumSource(Direction.class)
  @CartesianEnumSource(Status.class)
  void using_junit_pioneer_cartesian_product_test_with_enum_source(
      final Direction direction, final Status status) {
    assertTrue(true);
  }

This will spit out the same matrix, only now, adding a third element is quite simple: add another CartesianEnumSource annotation.

You can find other types of sources beside Enums, in JUnit Pioneer's Documentation.

As demonstrated in this repository, executing both matrix tests will print out:

[INFO] '-- JUnit Jupiter [OK]
[INFO]   '-- Property Based Matrix Test [OK]
[INFO]     +-- using junit pioneer cartesian product test with enum source (Direction, Status) [OK]
[INFO]     | +-- [1] INCOMING, SUCCESS [OK]
[INFO]     | +-- [2] INCOMING, FAILURE [OK]
[INFO]     | +-- [3] INCOMING, WAITING [OK]
[INFO]     | +-- [4] OUTGOING, SUCCESS [OK]
[INFO]     | +-- [5] OUTGOING, FAILURE [OK]
[INFO]     | '-- [6] OUTGOING, WAITING [OK]
[INFO]     '-- using junit parameterized test with method source (Direction, Status) [OK]
[INFO]       +-- [1] INCOMING, SUCCESS [OK]
[INFO]       +-- [2] INCOMING, FAILURE [OK]
[INFO]       +-- [3] INCOMING, WAITING [OK]
[INFO]       +-- [4] OUTGOING, SUCCESS [OK]
[INFO]       +-- [5] OUTGOING, FAILURE [OK]
[INFO]       '-- [6] OUTGOING, WAITING [OK]

Check out the code for this tutorial in Github.