DEV Community

Discussion on: What's Your Approach to Data Migration & Database Versioning?

Collapse
 
theaccordance profile image
Joe Mainwaring • Edited

I've had a decent amount of pain over the years regarding this topic as it's not particularly easy when you choose tech stack layers that don't account for this as a first-class citizen in your application.

Database Migration - You have to be careful with how you architect your database as you can impede future efforts to migrate data between instances of your application. I'm stuck with a legacy app that is a significant revenue generator but the data structure is so complex we're unable to justify the resources required (time/labor) to build a migration tool.

Data Versioning - I once dealt with a database versioning situation where there was no data point on which version the database was actually at, and at that time we had nearly a hundred data versioning scripts for the project. If you dealing with versioning on your database, make sure it also provides you context on which version you're currently at.

Scalability - Full Stack Developers should learn database normalization, even if they're working with NoSQL datasources. I've honestly been shocked at the lack of knowledge by developers in this area over the last decade, it's something I took for granted as part of my college courses in the 2000s. These fundamentals are important as they will impede scalability - likely when the engineers who wrote the non-optimized code are gone.

Data Integrity - Privacy regulations like GDPR have a direct impact on data integrity as you may not be able to simply purge data depending on the context of your application. This will likely be more prevalent in B2B or team-based software as a data point may be primarily owned by one users but other users also have a right to it. Thankfully, anonymizing PII is a legal approach to addressing regulatory requirements so you don't have to destroy relational integrity in your system.

Minimizing disruptions - Test, Test, and Test some more, and do your best to keep your prod instance as similar to your non-prod instances as possible, they can drift if your team isn't on top of it which will create periodic disruptions.