Schemaless to the rescue

Schemaless is the opposite of defining a schema for your data.

If you are used to SQL, you define a table/collection for your data, and the columns/fields of the table. This is the schema of your data. It tells you what data you can store in the database.

This is very useful to make sure you are storing the correct data, with consistency. It is also important to define indexes to make sure the patterns of access are fast enough.

However, this is not useful if you don't know what data will be stored in the future. Or if the data is more heterogeneous than homogeneous.

When to use Schemaless?

Imagine you want to let your users define some custom properties for your data. You want this because each user is unique, and they have different needs. As you can't predict what data they will store, you can't define a schema for it.

Another use case for schemaless is when the data is too heterogeneous. We are building a mini zapier, where each integration will have different configurations. As each integration will have different configurations, we can't define a schema for it.

Why Woovi uses MongoDB for schemaless data?

MongoDB is a document database that store your data using BSON, a binary representation of JSON, you can read more here MongoDB document.
To define a field as schemaless, you use the type Mixed. So you can store any data type there. We usually store a JSON object there. It let the backend and frontend code handle all variations of the data for us.

Below is an example of a schemaless collection field that we use in our Webhook data modeling

actionPayload: {
   type: mongoose.SchemaTypes.Mixed,
   description:
        "The schemaless payload of each webhook, it's based on action type (CALL_API, SEND_EMAIL, etc).",
}

We can also use Wildcard indexes to define index against unknown or arbitrary fields.
Therefore, schemaless queries are fast as known schema.

db.userData.createIndex( { "userMetadata.$**" : 1 } )

How to manage heterogeneous schemas?

Imagine you want to store these two payload types in the same field

{
   url: 'https://woovi.com/',
   headers: {
     Authorization: 'mysecret',
   },
}

{
   to: '[e-mail]',
   subject: 'New transaction',
}

They are very different, so we can't define a data schema upfront in the database. Hence, the backend/frontend code will read the schemaless data and decide what schema type it has.
For webhook action we can call an API or send an email.
You can design this using a tagged union in your code

type ActionType = 'CALL_API' | 'SEND_EMAIL'

type CallRestApiActionPayload = {
  url: string;
  headers: { [key: string]: string };
};

type SendEmailActionPayload = {
  to: string;
  subject: string;
};

Conclusion

Schemaless gives a lot of flexibility for your application.
You can let your users define custom properties in any collection of your database. In addition, it handles heterogeneous and unknown data very well.
However, you need to be careful when using it, to avoid increasing the complexity of your database and your code, and also avoiding inconsistency of data.

---

Woovi is a Startup that enables shoppers to pay as they like. To make this possible, Woovi provides instant payment solutions for merchants to accept orders.

If you want to work with us, we are hiring!

---

Photo by Usman Yousaf on Unsplash