DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’»

DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’» is a community of 968,547 amazing developers

We're a place where coders share, stay up-to-date and grow their careers.

Create account Log in
Cover image for How to Use Liquibase to Update the Schema of Your JHipster Application
Anthony Viard πŸ₯‘ for Entando

Posted on • Originally published at entando.com

How to Use Liquibase to Update the Schema of Your JHipster Application

Hey my fellow developers,

Liquibase is an open-source solution that helps you track, version, and deploy database changes. It adds version control for changes to your database schema, makes database upgrades repeatable across environments, and supports rollbacks so you can undo changes when it’s needed.

It’s the solution chosen by JHipster as well as Entando to manage database updates.

In this blog post, we’ll explore the Liquibase architecture, and how we can incrementally upgrade the database schema for our own JHipster and Entando applications.

Liquibase Architecture

Liquibase defines a master file (aka master.xml in JHipster) as well as changelog files that represent the incremental updates to your database schema. A changelog file contains a changeSet (e.g add, edit or delete a table) while the master file defines the order in which the database updates are to be run.

Here are some examples of what a Liquibase architecture looks like from the official documentation.

The directory structure

 com
      example
        db
          changelog
            db.changelog-master.xml
            db.changelog-1.0.xml
            db.changelog-1.1.xml
            db.changelog-2.0.xml
          DatabasePool.java
          AbstractDAO.java
Enter fullscreen mode Exit fullscreen mode

The master file

<?xml version="1.0" encoding="UTF-8"?>  
<databaseChangeLog 
 xmlns="http://www.liquibase.org/xml/ns/dbchangelog" 
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
 xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
                     http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.8.xsd"> 
 <include  file="com/example/db/changelog/db.changelog-1.0.xml"/>  
 <include  file="com/example/db/changelog/db.changelog-1.1.xml"/>  
 <include  file="com/example/db/changelog/db.changelog-2.0.xml"/>  
</databaseChangeLog>
Enter fullscreen mode Exit fullscreen mode

A sample changelog

<?xml version="1.0" encoding="UTF-8"?>  
<databaseChangeLog  
 xmlns="http://www.liquibase.org/xml/ns/dbchangelog"  
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
 xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
                     http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.8.xsd">  
 <changeSet  author="authorName"  id="changelog-1.0"> 
   <createTable  tableName="TablesAndTables"> 
     <column  name="COLUMN1"  type="TEXT"> 
       <constraints  nullable="true"  primaryKey="false"  unique="false"/> 
     </column> 
   </createTable> 
 </changeSet> 
</databaseChangeLog>
Enter fullscreen mode Exit fullscreen mode

Your Liquibase survival kit

Every time you start your application, Liquibase checks if it is synchronized with the latest configuration in your master file, and deploys the new changesets if it’s not up to date. Liquibase uses the checksums of the previous changelogs to ensure their integrity and fires an alert if they have changed. Here are some tips to follow to ensure your database upgrades go smoothly.

#1 Avoid modifying a changelog

You should not change the content of a changelog once it has been executed on a database. Instead, perform a rollback or add a new changelog.

If the validation fails, your database will not be able to start correctly.

Image description

#2 Keep database changes in order

The master file executes the changelogs in the order they are defined. So, if you have a changelog that modifies a table created in a previous changelog, be sure to respect the order to have the proper plan executed. It’s recommended to keep a clear and maintainable file.

#3 Keep changesets small

The changeset is contained in a changelog and defines actions to perform in the database (e.g. create, drop, alter tables, etc.). By default, JHipster creates one changeset per entity. This is a good pattern to follow unless there’s a reason to make updates to multiple tables in a single changelog. Avoid creating BIG one-shot changesets.

How to Update Your Schema in JHipster 6

JHipster makes it simple and intuitive to generate entities using JDL Studio or via the command line. Every time a new entity is created, a new changeset is generated and the master file is updated. However, modifying an existing entity will simply update the original changelog. If this changeset has already been run, we break rule #1.

Consolidate schema changes in a single changeset

When adding a new database entity, it’s quite common during local development to make several changes to the data model before arriving at the final schema. In this case, the easiest approach is to make updates to the entity as needed and only commit the final changelog that’s generated once you’ve finished development.

However, a changelog that has already been applied to a given environment cannot be modified without risking potential data loss or other breaking changes.

The solution is to generate incremental changes using Maven and Gradle plugins.

Generate incremental changesets with plugins

Maven and Gradle plugins, combined with the Liquibase Hibernate plugin, can be used to generate incremental changesets without breaking rule #1.

JHipster provides the configuration to make it work out of the box with most databases but you may need to modify it in some circumstances. Check the official guide for more information.

Below, we’ll cover a sample app from Entando that’s using an H2 database.

Maven configuration

<plugin>
   <groupId>org.liquibase</groupId>
   <artifactId>liquibase-maven-plugin</artifactId>
   <version>${liquibase.version}</version>
   <configuration>
       <changeLogFile>${project.basedir}/src/main/resources/config/liquibase/master.xml</changeLogFile>
       <diffChangeLogFile>${project.basedir}/src/main/resources/config/liquibase/changelog/${maven.build.timestamp}_changelog.xml</diffChangeLogFile>
       <driver>org.h2.Driver</driver>
       <url>jdbc:h2:file:${project.build.directory}/h2db/db/sample</url>
       <defaultSchemaName></defaultSchemaName>
       <username>sample</username>
       <password></password>
    <referenceUrl>hibernate:spring:com.entando.sample.domain?dialect=org.hibernate.dialect.H2Dialect&amp;hibernate.physical_naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy&amp;hibernate.implicit_naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy</referenceUrl>
       <verbose>true</verbose>
       <logging>debug</logging>
       <contexts>!test</contexts>
       <diffExcludeObjects>oauth_access_token, oauth_approvals, oauth_client_details, oauth_client_token, oauth_code, oauth_refresh_token</diffExcludeObjects>
   </configuration>
Enter fullscreen mode Exit fullscreen mode

Workflow

  1. Update your entity with JHipster using the command jhipster entity yourEntityName
  2. Discard the changes made by JHipster in the Entity changelog git checkout -- [ENTITY_CHANGELOG].xml
  3. Run ./mvnw liquibase:diff for Maven or ./gradlew liquibaseDiffChangelog -PrunList=diffLog for Gradle
  4. A new changelog will be generated that contains the diff between your updated Entity and the original database table.

In my application, I have added a customer entity with a field β€œname”. Later on, I want to add an β€œage” field so I update it with JHipster: jhipster entity customer

Image description

Image description

The changelog is now updated with the β€œage” field, but because my first version has already been run against my database I will break rule #1. I’ll need to revert this file and replace it with the original version.

Image description

Next, I can run ./mvnw liquibase:diff. I now have an incremental changelog that contains just the updates that I made. Now, I can add this new changelog to the master.xml file.

Image description

Note: This could not work if you’re using an H2 database. However, you can easily create incremental changelogs manually by extracting the generated XML code from the changelog generated by JHipster.

Incremental changelogs in JHipster 7

With the release of JHipster 7, the --incremental-changelog option allows us to generate a separate changelog when modifying an Entity.

This means we don’t need to rollback the original changelog anymore, and we can generate incremental changelogs for our entities without breaking rule #1.

You can run JHipster with the incremental option: jhipster --incremental-changelog. It also works when you import a JDL, e.g. jhipster jdl app.jdl --incremental-changelog --with-entities.

You can check the β€œincrementalChangelog” entry is set to β€œtrue” in .yo-rc.json.

Image description

Then, create the entity as usual with jhipster entity customer.

Image description

Finally, update the entity by running the same command to add a new field.

Two new changelogs are generated, one for updating the entity and the second for injecting sample data for development.

Image description

Image description

Image description

Incremental changelogs work well when you don’t need to execute a lot of changes for a given entity, but multiplying the number of changelogs can lead to complex change management. If there’s no risk of breaking your existing database, consider merging changelogs to simplify your project structure.

Conclusion

Upgrading your database schema can be tricky, especially when working with multiple environments. In today’s blog, we learned how Liquibase helps you version control updates to your schema, and how to generate incremental changesets for entities generated by JHipster.

Entando is an application composition platform for Kubernetes that adds support for generating micro frontends along with JHipster microservices. Entando 6.3.2 currently supports JHipster 6 and will be updated to JHipster 7 in the next Entando release.

Top comments (3)

Collapse
 
codenameone profile image
Shai Almog

How do you handle PR collisions in these cases?

One of the big difficulties we had with this is that multiple team members would change the DB. Because our PR review cycles can stretch for days you would start out against one schema and another person would get his PR merged in the meanwhile. So you had to go back and redo the PR. Liquibase became pretty painful in these situations because it's impossible to read/understand what went on.

Eventually we decided we need something "simpler" and downgraded to Flyway which is more readable in the PRs so easier to fix during the process.

Collapse
 
avdev4j profile image
Anthony Viard πŸ₯‘ Author

Hi @codenameone ,
I can understand reviewing process can make merging painful when you deal with DB schema updating.

I used Flyway in the past and faced the same issue. If somebody had to change the schema before me, this meant updating my SQL scripts to match with the new requirements.

Liquibase uses Changelogs to isolate changes (and even changesets) so I guess only the "master file" has to be updated, isn't it?
However, if two scripts change the same table, it could be a mess and force the developer to update his own changelog.

This is why I would more rethink the way we are developing the features. As I said, in the past I faced that (with Flyway) and we decided to evaluate, before each sprint, which story could be an issue about schema updating.
Finally, it was successful for us because we avoided starting 2 stories that could end to conflicts, or we defined priorities between both. But it's not the only source of truth and it really depends on teams (teams should choose the best solution they are comfortable with).

Two questions for you:

  • Why did you find Liquibase painful to read/understand?
  • How Flyway helps you to improve that?
Collapse
 
codenameone profile image
Shai Almog

Flyway is just SQL commands. It's really simple to read in code reviews and if there's a conflict it's a pain but it can all be fixed.

As you said, Liquibase conflicts are really hard to read and follow. Then redoing the process is painful.

I agree that we should segment DB features better. Unfortunately that wasn't my call on the team. The features are sometimes concurrent and backported so you end up with different schemas all over the place and people going back/forth between schemas.
All the teams are vertical/feature oriented so everyone touches the database all the time.

I can't say we love Flyway. It just fails in a way that's easier to understand.

🌚 Friends don't let friends browse without dark mode.

Sorry, it's true.