Recently, I found myself in a position where an application was heavily reliant on a state object. This is fairly typical for single page applications (SPAs) and can pose a challenge when your state object's schema changes significantly and you have users that have data saved under an old schema.
In this post, I'll explore a proof-of-concept solution I put together to explore the topic. I figured this would be an interesting and educational exploration of the topic!
Do you know of any packages that do this already? Please let me know in the comments!
An Example Problem
Let's say I've created an app in which there's a user and that user can enter their pet type and breed. Upon launching the MVP, my state object looks something like this:
const state = {
person: {
name: 'Edgar',
pets: {
type: 'dog',
name: 'Daffodil',
},
},
};
This works great for the MVP, but soon I realize I don't want the pets property to live under the person property, but rather I want it to be it's own propert under state. In other words, my ideal state might look like this:
const state = {
person: {
name: 'Edgar',
},
pets: {
type: 'dog',
name: 'Daffodil',
},
};
While I'd like to simply be able to make this change in my SPA, I'm concerned that existing app users have my original schema saved somewhere (e.g., local storage, nosql, a JSON string, etc.). If I load that old data but my app expects the new schema, I may try to access properties in the wrong place (e.g., state.pets.type versus state.person.pets.type), causing issues.
Schema Migration to the Rescue!
Schema migration isn't a new concept; it's been used for quite some time to migrate database tables between different versions of applications. In this post, I'm going to use the same basic concepts behind schema migrations to migrate JavaScript objects.
Defining our Migration Array
Let's define an array of migrations to run. Each migration will have a from, to, up, and down property. The from and to props will represent the lower and higher version respectively, and the up and down props will be functions that move a schema from the from version to the to version and vice-versa. That may sound a bit confusing, but I think it'll make a bit more sense in the context of our person/pets example.
Let's write the first migration.
const migrations = [
{
from: '1.0',
to: '1.1',
up: schema => {
const newSchema = {
version: '1.1',
person: {
name: schema.person.name,
},
pets: {
...schema.person.pets,
},
};
return newSchema;
},
down: schema => {
const newSchema = {
version: '1.0',
person: {
...schema.person,
pets: { ...schema.pets },
},
};
return newSchema;
},
},
];
If we have a version "1.0" schema, the up method of this object will convert that schema to "1.1". Conversely, if we have a version "1.1" schema, the down method will convert that schema to "1.0".
Making the Migration Happen
This is cool in concept, but we need to create a function that actually executes the migration. To do so, we'll create a migrate function that takes as arguments a schema and the version number to which that schema should be migrated.
const migrate = (schema, toVersion) => {
const fromVersion = schema.version;
const direction = upOrDown(fromVersion, toVersion);
if (direction === 'same') {
return schema;
}
const currentMigration = migrations.find(
migration => migration[direction === 'up' ? 'from' : 'to'] === fromVersion
);
const newSchema = currentMigration[direction](schema);
return migrate(newSchema, toVersion);
};
You may notice a couple things about this function: it's recursive (it won't stop until we've migrated to our target version), and it references a helper function, upOrDown, which I have defined below. This function just helps determine the direction of the migration (1.0 to 1.1 is up, 1.1 to 1.0 is down).
const upOrDown = (fromVersion, toVersion) => {
const fromNumbers = fromVersion.split('.').map(el => Number(el));
const toNumbers = toVersion.split('.').map(el => Number(el));
for (let i = 0; i < fromNumbers.length; i++) {
if (fromNumbers[i] < toNumbers[i]) {
return 'up';
}
if (fromNumbers[i] > toNumbers[i]) {
return 'down';
}
}
return 'same';
};
Taking it for a Test Run
Let's create two objects, one is a version "1.0" schema and the other a version "1.1" schema. The goal will be to migrate the "1.0" schema to "1.1" and the "1.1" schema to "1.0".
const schemaA = {
version: '1.0',
person: {
name: 'Edgar',
pets: {
type: 'dog',
name: 'Daffodil',
},
},
};
const schemaB = {
version: '1.1',
person: {
name: 'Edgar',
},
pets: {
type: 'dog',
name: 'Daffodil',
},
};
Now, let's run our migrations.
// From 1.0 to 1.1
console.log(migrate(schemaA, '1.1'));
/*
{ version: '1.1',
person: { name: 'Edgar' },
pets: { type: 'dog', name: 'Daffodil' } }
*/
// From 1.1 to 1.0
console.log(migrate(schemaB, '1.0'));
/*
{ version: '1.0',
person: { name: 'Edgar', pets: { type: 'dog', name: 'Daffodil' } } }
*/
Perfect! We can now migrate "up" from one schema version to the next or migrate back "down".
Another Schema Change!
I'm now realizing that a person can have multiple pets–why not? So, our pets key should actually be an array, not an object. Furthermore, I'm realizing our person key could probably just be the person's name rather than having a name key (I've decided we won't have any more props associated with the person). That means a new schema, version 1.2, which will look something like this:
const state = {
person: 'Edgar',
pets: [
{
type: 'dog',
name: 'Daffodil',
},
],
};
So, let's write a migration from version 1.1 to 1.2.
const migrations = [
{
from: '1.0',
to: '1.1',
up: schema => {
const newSchema = {
version: '1.1',
person: {
name: schema.person.name,
},
pets: {
...schema.person.pets,
},
};
return newSchema;
},
down: schema => {
const newSchema = {
version: '1.0',
person: {
...schema.person,
pets: { ...schema.pets },
},
};
return newSchema;
},
},
{
from: '1.1',
to: '1.2',
up: schema => {
const newSchema = {
version: '1.2',
person: schema.person.name,
pets: [schema.pets],
};
return newSchema;
},
down: schema => {
const newSchema = {
version: '1.1',
person: {
name: schema.person,
},
pets: schema.pets[0],
};
return newSchema;
},
},
];
Multi-Version Migrations
Remember how our migrate function is recursive? That becomes pretty helpful when we need to migrate multiple versions. Let's say we want to migrate from a 1.0 schema to a 1.2 schema and vice versa. We can do that!
// 1.0 to 1.2
console.log(migrate(schemaA, '1.2'));
/*
{ version: '1.2',
person: 'Edgar',
pets: [ { type: 'dog', name: 'Daffodil' } ] }
*/
const schemaC = {
version: '1.2',
person: 'Edgar',
pets: [
{
type: 'dog',
name: 'Daffodil',
},
],
};
// 1.2 to 1.0
console.log(migrate(schemaC, '1.1'));
/*
{ version: '1.0',
person: { name: 'Edgar', pets: { type: 'dog', name: 'Daffodil' } } }
*/
Hey, it works!
Conclusion
This has been a fun dive into the world of schema migration! Having hacked together some schema migration functionality, I'm now fairly confident in being able to implement this using either a "roll-your-own" method or an existing package.
Top comments (3)
not directly, but we can use ...
AST transformers
github.com/unifiedjs/unified
lihautan.com/manipulating-ast-with...
XSLT
database schema migration tools
github.com/schemahero/schemahero
schemahero.io/learn/declarative-sc...
github.com/waptik/mongoose-cli (Mongoose is a MongoDB object modeling tool)
stackoverflow.com/questions/324725...
stackoverflow.com/questions/278358...
github.com/flexxnn/sequelize-auto-...
github.com/topics/migrations-gener...
github.com/topics/database-migrations
github.com/flyway/flyway
github.com/liquibase/liquibase
parser generators
tree-sitter
lezer-parser
lenses = bidirectional transformers
github.com/vmx/jslens
github.com/calmm-js/partial.lenses
keywords:
object-schema-migrations
graph-schema-migrations
Beautiful.👏
Data versioning it's one of those hard problems I've faced in my life as developer. Once, I spent months working in a SQL version of this.
Thanks for sharing. I will try it soon.
Do you think is hard to apply this in some NoSQL database?