As developers, we often rely on tools like Flyway to streamline the management of database migrations, ensuring that our applications evolve smoothly over time. However, there are subtleties that might be unexpected beneath the surface of this seemingly straightforward process. We came across one and would like to share it with you.
Let's examine Flyway's response to newer database versions and its implications for migration management!
Understanding Flyway's Startup Checks and the Usual Use Cases
Before diving into the peculiar behaviour that we've uncovered, let's quickly grasp Flyway's startup checks. Upon initialization, Flyway compares the checksums of migration files with those applied to the database, ensuring consistency and integrity. Any discrepancies, be they altered files or missing (executed) scripts, prompt Flyway to halt, warning us of potential issues before they escalate.
Try this at home, NOT on prod!
In some scenarios, developers may resort to manual interventions for various reasons. For instance, if there's a need to repeat the execution of the last script, they might opt to delete the corresponding migration entry in the database. This action essentially prompts Flyway to rerun the script during the next migration cycle. Similarly, if a script needs to be amended or fixed directly in the database, developers might execute the necessary changes manually, adjust the script accordingly, and adjust the stored checksum. However, such manual adjustments should be approached with caution to avoid inadvertently introducing inconsistencies or breaking the migration process.
When you start doing these things you might expect the following errors in the logs:
Checksum mismatch - when the script is altered and the checksum is not updated:
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource
[org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Validate failed: Migrations have failed validation
Migration checksum mismatch for migration version 1.00
-> Applied to database : 1993450877
-> Resolved locally : 1305205389
Either revert the changes to the migration, or run repair to update the schema history.
Missing migration - when script is deleted from the database, or if you're using timestamps as migration versions and ignore scripts merged into the master/main in the meantime after you've created the migration file:
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource
[org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Validate failed: Migrations have failed validation
Detected resolved migration not applied to database: 1.50.
To ignore this migration, set -ignoreMigrationPatterns='*:ignored'. To allow executing this migration, set -outOfOrder=true.
An Unexpected Twist: Detecting Newer Database Versions
Here's where things get interesting. While exploring Flyway's inner workings, I stumbled upon a behaviour that defied my expectations. Imagine this scenario: our database is humming along with the latest migrations applied, proudly sporting version 2.00. However, upon inspecting our migration files, we find ourselves stuck in the past, lingering at version 1.00. One might expect Flyway to raise a red flag at this apparent mismatch, halting operations until the files catch up to the database. Instead, Flyway surprises us with a mere warning – a gentle (WARN) reminder in the log that a migration file is missing.
We can see what is happening in the logs (below).
When starting the application multiple times with different versions, we observe the following sequence of events:
(Instance 1, Version 1): The first instance of the application is started with version 1. Flyway validates and applies the migration, initializing the database schema to version 1.00.
INFO 1 --- [flyway-testing] [main] o.f.c.i.c.DbValidate: Successfully validated 1 migration (execution time 00:00.194s)
INFO 1 --- [flyway-testing] [main] o.f.c.i.c.DbMigrate : Current version of schema "public": << Empty Schema >>
INFO 1 --- [flyway-testing] [main] o.f.c.i.c.DbMigrate : Migrating schema "public" to version "1.00 - init car"
INFO 1 --- [flyway-testing] [main] o.f.c.i.c.DbMigrate : Successfully applied 1 migration to schema "public", now at version v1.00 (execution time 00:00.036s)
(Instance 2, Version 1): A second instance of the application, meant for high availability, is launched also with version 1. Flyway validates the schema, finding it already at version 1.00, thus no migration is necessary.
INFO 1 --- [flyway-testing] [main] o.f.c.i.c.DbValidate: Successfully validated 1 migration (execution time 00:00.296s)
INFO 1 --- [flyway-testing] [main] o.f.c.i.c.DbMigrate : Current version of schema "public": 1.00
INFO 1 --- [flyway-testing] [main] o.f.c.i.c.DbMigrate : Schema "public" is up to date. No migration necessary.
(Instance 2, Version 2): A new version 2 is deployed to the second instance. Flyway validates the schema, still finding it at version 1.00. However, it proceeds to migrate the schema to version 2.00, adding the necessary changes.
INFO 1 --- [flyway-testing] [main] o.f.c.i.c.DbValidate: Successfully validated 2 migrations (execution time 00:00.243s)
INFO 1 --- [flyway-testing] [main] o.f.c.i.c.DbMigrate : Current version of schema "public": 1.00
INFO 1 --- [flyway-testing] [main] o.f.c.i.c.DbMigrate : Migrating schema "public" to version "2.00 - add column"
INFO 1 --- [flyway-testing] [main] o.f.c.i.c.DbMigrate : Successfully applied 1 migration to schema "public", now at version v2.00 (execution time 00:00.034s)
(Instance 1, Version 1): The first instance is then redeployed with version 1, effectively restarting it. Flyway validates the schema and detects that it's already at version 2.00, even though the migration files only go up to version 1.00. It issues a warning about the discrepancy but proceeds without applying any migrations or startup failure.
INFO 1 --- [flyway-testing] [main] o.f.c.i.c.DbValidate: Successfully validated 2 migrations (execution time 00:00.184s)
INFO 1 --- [flyway-testing] [main] o.f.c.i.c.DbMigrate : Current version of schema "public": 2.00
WARN 1 --- [flyway-testing] [main] o.f.c.i.c.DbMigrate : Schema "public" has a version (2.00) that is newer than the latest available migration (1.00) !
INFO 1 --- [flyway-testing] [main] o.f.c.i.c.DbMigrate : Schema "public" is up to date. No migration necessary.
Implications of Flyway's Decision
This seemingly lenient approach raises questions about the trade-offs involved. While it doesn't explicitly grant us the freedom to apply migrations out of sequence, it does allow the application to start even if later migrations have been applied to the database. This can serve as an indicator of potential skipped migrations and it also enables a rolling migration approach where older instances of the application can be restarted even after the newer instance version has been started.
Conclusion
One might say that the flyway is "forward compatible" in this case. It is engineered to manage future database migrations and schema changes seamlessly, allowing ongoing functionality without needing complete future code.
Understanding Flyway's behaviour is key to successful database management. By following best practices and embracing their quirks, such as keeping migration files current and avoiding manual schema alterations, developers can navigate migrations with confidence. Hope this interesting quirk that we've found helps you!
Happy migrating!
Top comments (0)