DEV Community: Geoff Bourne

Spring Boot DataJdbcTest with PostgreSQL and Liquibase

Geoff Bourne — Wed, 28 Feb 2024 04:29:13 +0000

In this article I wanted to share a little journey I took creating a Spring Boot unit test that exercises a Spring Data JDBC repository running against a Postgresql database initialized with Liquibase. I made a couple of mistakes along the way that I wanted to document here and provide the solutions for those.

In the Spring Initializr I chose the dependencies:

Spring Data JDBC
Liquibase Migration
PostgreSQL Driver

To initialize the schema, I created the Liquibase changelog file at src/main/resources/db/changelog/db.changelog-master.yaml where I create a user table:

databaseChangeLog:
  - changeSet:
      id: 1
      author: gdb
      changes:
        - createTable:
            tableName: user
            columns:
              - column:
                  name: id
                  autoIncrement: true
                  type: BIGINT
                  constraints:
                    primaryKey: true
                    nullable: false
              - column:
                  name: identifier
                  type: VARCHAR(50)
                  constraints:
                    nullable: false
              - column:
                  name: issuer
                  type: VARCHAR(100)
                  constraints:
                    nullable: true

The User entity record/class is declared as

import org.springframework.data.annotation.Id;

public record User(
    @Id
    Long id,
    String identifier,
    String issuer
) {
}

And finally, the CRUD repository interface:

import org.springframework.data.repository.CrudRepository;

public interface UserRepository extends CrudRepository<User, Long> {
}

At first I started with the test class written as

import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.data.jdbc.DataJdbcTest;
import org.springframework.boot.test.autoconfigure.jdbc.AutoConfigureTestDatabase;
import org.springframework.boot.testcontainers.service.connection.ServiceConnection;
import org.testcontainers.containers.PostgreSQLContainer;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.junit.jupiter.Testcontainers;

import static org.assertj.core.api.Assertions.assertThat;

@Testcontainers
@DataJdbcTest
class UserRepositoryTest {
    @Container
    static PostgreSQLContainer<?> pg = 
        new PostgreSQLContainer<>("postgres:16");

    @Autowired
    UserRepository userRepository;

    @Test
    void saveUser() {
        final User result = userRepository.save(
            new User(null, "name1", "issuer1")
        );

        assertThat(result.id()).isNotNull();
    }
}

but that resulted in the error

Error creating bean with name 'liquibase' defined in class path resource
[org/springframework/boot/autoconfigure/liquibase/LiquibaseAutoConfiguration$LiquibaseConfiguration.class]: 
Failed to instantiate [liquibase.integration.spring.SpringLiquibase]: 
Factory method 'liquibase' threw exception with message: 
Error creating bean with name 'dataSource': 
Failed to replace DataSource with an embedded database for tests. 
If you want an embedded database please put a supported one on the classpath 
or tune the replace attribute of @AutoConfigureTestDatabase.

It was one of those errors that actually tells you the solution. Nice! So, I added:

@Testcontainers
@AutoConfigureTestDatabase(
    replace = AutoConfigureTestDatabase.Replace.NONE
)
@DataJdbcTest
class UserRepositoryTest {

The next error was

Error creating bean with name 'dataSource' defined in class path resource 
[org/springframework/boot/autoconfigure/jdbc/DataSourceConfiguration$Hikari.class]: 
Failed to instantiate [com.zaxxer.hikari.HikariDataSource]: 
Factory method 'dataSource' threw exception with message: 
Failed to determine a suitable driver class

I remembered that Spring Boot added the service connections feature, so added that to the container field's declaration:

    @Container
    @ServiceConnection
    static PostgreSQLContainer<?> pg = 
        new PostgreSQLContainer<>("postgres:16");

Now the test passes!

Trim trailing zeroes when formatting float in Java

Geoff Bourne — Sun, 11 Dec 2022 17:03:56 +0000

Simple problem, but surprisingly non-obvious answer:

The output of

System.out.printf("%f%n", 1.0);

1.000000

however, I really don't want those extra, trailing zeroes and I don't want to lose precision in other cases.

Looking again through the Formatter docs, I was surprised to find that only a fixed number of decimal places could be specified, such as %.1f. That won't work for this problem since 1.0005 gets formatted as 1.0.

Since this is a "decimal point" problem, it turns out to not be a surprise that BigDecimal is the solution.

From this

System.out.println(BigDecimal.valueOf(1.0));
System.out.println(BigDecimal.valueOf(1.00000000));
System.out.println(BigDecimal.valueOf(1.000000001));

I get what I wanted:

1.0
1.0
1.000000001

Note that the toString is actually implemented like toEngineeringString, but the formatting can be constrained to plain decimal formatting with toPlainString.

For example, the following

System.out.println(BigDecimal.valueOf(0.000000005300));
System.out.println(BigDecimal.valueOf(0.000000005300).toEngineeringString());
System.out.println(BigDecimal.valueOf(0.000000005300).toPlainString());

outputs

5.3E-9
5.3E-9
0.0000000053

Finally, to make sure the desire to remove trailing zeroes didn't introduce a huge performance problem, I benchmarked 10 million iterations of each formatting 5,000 random values. BigDecimal formatting seems to be only very slightly slower:

formatted took PT5.3975531S
BigDecimal took PT5.6573979S

Jackson JSON parsing top-level map into records

Geoff Bourne — Tue, 07 Jun 2022 16:25:47 +0000

Java 14 added the record type and Jackson JSON works great with them.

I hit an edge case where I needed to parse a JSON structure where the top level was an object with arbitrary keys.

I also didn't want to read into a Map but rather a record that mapped each keyed entry into a record each.

Here's a contrived example JSON I wanted to parse:

{
  "Some Child's Name": {
    "age": 5
  },
  "Some Adult's Name": {
    "age": 21
  }
}

The @JsonAnySetter annotation (1) on a Map field got me most of the way, but the final piece of the solution was to pre-instantiate the map at (2).

Here are the final record declarations that can accommodate the JSON above:

record People(
    @JsonAnySetter // (1)
    Map<String, Person> people
) {
    public People {
        people = new HashMap<>(); // (2)
    }
}

record Person(
    int age
) { }

For example, following use of an ObjectMapper ...

final People people = objectMapper.readValue("""
    {
      "Some Child's Name": {
        "age": 5
      },
      "Some Adult's Name": {
        "age": 21
      }
    }""", People.class);
System.out.println(people);

outputs

People[people={Some Adult's Name=Person[age=21], Some Child's Name=Person[age=5]}]

Customize POJO generation from IntelliJ Database tool window

Geoff Bourne — Tue, 09 Feb 2021 19:23:19 +0000

I was already a huge fan of IntelliJ Ultimate's Database Tool Window; however, today I found a gem hiding further down in the context menu: Scripted Extensions.

What brought me there in the first place was that I wanted to generate some bare-bones POJOs that were Lombok @Data annotated and had simple JPA @Column fields for use with JOOQ. IntelliJ Ultimate comes with a Scripted Extension, Generate POJOs.groovy, that got pretty close. So, I started with that and created a copy of that file...

...with this content:

Now I can select tables, right-click, and run the extension:

Re-ordering git commits using IntelliJ

Geoff Bourne — Sun, 31 Jan 2021 03:42:56 +0000

In the "Log" tab of the "Git" tool window, select and right click the revision prior to commits to re-order. Choose "Copy revision number":

Use the "Git -> Rebase" menu.

Paste the revision into the text field.

Remove any other options enabled and add --interactive option:

Use up/down button to re-order the commits where the commits are ordered top-to-bottom, oldest-to-newest:

Click "Start rebasing". You will be prompted to resolve any conflicts that might occur due to the re-ordering.

Check out the IntelliJ documentation for more information

Git merge without commit in IntelliJ

Geoff Bourne — Sat, 30 Jan 2021 22:52:21 +0000

By default, using the git merge (not rebase) operation in IntelliJ will immediately create a merge-commit. Sometimes you may want to selectively revert changes or alter content before concluding the merge.

Using either the "Git" menu or version control status area in the bottom right of the project window, checkout the destination branch of the merge:

Use the "Git -> Merge" menu to bring up the following window. There's many options in this window, which simplify the command-line equivalents. In any scenario, first choose the source branch from the dropdown:

Remove any options you already have:

and add the --no-commit option:

Click the "Merge" button to proceed.

The "Commit" tool window will show the merge changes, but not yet selected/staged for commit:

At this point you can edit files as normal, selectively revert lines from the diffs...

...or rollback/revert entire files:

Select/stage the changes from the "Commit" tool window and proceed with the merge's commit as you normally would.

Looking at the "Git" tool window's "Log" tab, the merge-commit appears as it should:

MacOS Automator action to move screenshots

Geoff Bourne — Thu, 28 Jan 2021 19:36:17 +0000

By default, MacOS' shortcut to save a screenshot to a file, such as Shift-Command-4 will save the screenshot image to the Desktop. If you're like me and prefer little to nothing on your desktop (it's hidden by various applications most of the time anyway), then this post describes an Automator folder action to automatically move screenshots into your Pictures folder or any folder of your choosing.

When you open Automator or create a new file, choose the Folder Action type, shown here:

Choose the Desktop folder in the action's configuration at the top and add the steps shown here. You can change the destination folder in the last step to any location you prefer.

Now, when a screenshot image file is saved to the Desktop, a few moments later the folder action will automatically move it.

Running Kafka with Docker Compose

Geoff Bourne — Wed, 27 Jan 2021 17:01:56 +0000

Confluent is the primary sponsor of Apache Kafka and not surprisingly they provide great Docker images for Kafka (and its dependency, ZooKeeper) via their Confluent Platform.

The following docker-compose.yml file spins up a single-node Kafka "cluster", but the INTERNAL/EXTERNAL bits enable multi-node clusters to be declared with a small amount of changes.

version: "3"

services:
  zk:
    image: confluentinc/cp-zookeeper:${CP_VERSION:-5.5.3}
    environment:
      JAVA_TOOL_OPTIONS: -Xmx256m
      ZOOKEEPER_CLIENT_PORT: 2181
    volumes:
      - zk:/var/lib/zookeeper/data
  kafka:
    image: confluentinc/cp-kafka:${CP_VERSION:-5.5.3}
    environment:
      JAVA_TOOL_OPTIONS: -Xmx512m
      KAFKA_ZOOKEEPER_CONNECT: zk:2181
      KAFKA_ADVERTISED_LISTENERS: EXTERNAL://localhost:9092,INTERNAL://kafka:9192
      KAFKA_LISTENERS: EXTERNAL://0.0.0.0:9092,INTERNAL://0.0.0.0:9192
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_LOG_RETENTION_HOURS: 1
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: EXTERNAL:PLAINTEXT,INTERNAL:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
      # some loggers are set to TRACE by default...and they're noisy
      KAFKA_LOG4J_LOGGERS: kafka.controller=INFO,state.change.logger=INFO
    volumes:
      - kafka:/var/lib/kafka/data
    ports:
      - 9092:9092
    depends_on:
      - zk

volumes:
  zk:
  kafka:

Docker Compose also makes it easy to exec the usual helper scripts provided by the image, such as listing topics:

docker-compose exec kafka \
  kafka-topics --bootstrap-server localhost:9092 \
  --list

and consuming from a topic:

docker-compose exec kafka \
  kafka-console-consumer --bootstrap-server localhost:9092 \
  --topic your-topic

Confluent has extensive documentation on the images used here.

Also, make sure to update or pass in the CP_VERSION appropriate from the newest tags published.