Handle data heavy application development on frontend

Are you frustrated by changing data structure? Or upset about deeply nested data that increases the complexity of frontend development? Don't. Don't be emotional over code.

Jokes aside, modern day applications usually have to deal with data that is nested or relational in nature. For instance, when we use Redux, we could use normalizr to store a normalized state structure.

The increase complexity of data structure makes it tedious to develop. It will also cause bloat to the client devices. Frontend is meant to be light-weight and fast. Especially since we have to cater to mobile devices, we want to be careful in handling our data. The expensive rendering and computation should be delegated to the backend servers, which is a more controlled and scalable environment.

Read: Backend-in-the-frontend: a pattern for cleaner code -- Hackernoon

Even so, due to business requirements, we as frontend developers might have no choice but to parse complex data structure or data structure that is not optimised for the UI.

We'll be using contrived example to get the point across. Let's take a look at the following JSON object.

The requirements

Imagine if you have to create a data table for a list of dogs showing their favorite food and the supplier. Why that will be required, I don't know. Maybe, the software users are pet shop owners and they want to have an overview of the suppliers whom they could order the most items from and possibly negotiate a better deal.

So back to the data. Let's say you are using a datatable plugin. The API of the datatable requires you to specify the dataKey for each column.

Then comes the product owner, he has requested an additional requirement to show all the suppliers in one column.

So right now, you have a few things to solve:

1. You need to be able to access a nested dataKey, favourites.food.
2. You will need to access all the supplier.name in the array and concatenate them into a string for the data column.

If your table is just simple presentational logic, and if the datatable allows dataKey definition with dot notation, i.e. favourites.food, it will be fine to simply use the dataKey as it is. I believe in keeping things simple as it is, until proven otherwise.

What if, throughout the app, there are multiple logic that needs to access this data? Are you going to get the dot notation in each cases, even if they might be deeply nested?

Or because of fast changing business requirements, the data structure from the API changes frequently? What will you do to minimise the disruption of the data object?

For me, I will transform the data object for use throughout my frontend code, so that the data integrity of the object in the frontend could remain even when the data structure from the API changes.

Learning from the giants

I have been using react-jsonschema-form by Mozilla and react BaseTable by Autodesk. I love how easy it is to define the data fields and data columns respectively. Using JSON object simplifies the parsing of the data and make it obvious to change the dataKey if the data structure changes. Both of the plugins also keep the data parsing logic loosely coupled to the UI rendering logic.

If you have not used the plugins before, here is a sample code of how it is being used.

React-jsonschema-form

React BaseTable

My take is, you can roughly guess what each the plugin API do.

The key formData and data is used to feed the data into the plugin. The schema and columns instruct the plugin on the dataKey to look for the data. The uiSchema and cellRenderer is where you could define any customisation to the UI rendering portion.

Back to the drawing block

With the above inspriations, I decide to contain all my data transformation through a json object. For lack of a better term, we will refer to this data object as schema.

Using the requirements stated above, we determine that we need to build the following columns in the data table:

1. Id of the dog
2. Name of dog
3. Type of dog
4. Cost of dog
5. Supplier

For us to be able to work with datatable easily, we want to transform our data to a single layer data object that we could pass into our datatable easily.

This is my desired input for the datatable.

This is what I want to define for my schema in order to achieve the input above. Note, in simple cases, it is easier to parse the data directly. However, if we want to conditionally render the schema and combine different schema, I find that having a data schema like below make it easy to read and extend my data structure.

./schema.js

All I have to do to transform the backend data to my desired input is this.

./feData.js

More interested in the code? Find it here

Breakdown

There are a few premise that this data adapter is designed on.

1. We can extend the schema easily.
2. We can define a default return value.
3. If the schema cannot find a dataKey, it will return the full data.
4. If the schema did not find a key, it will assume the usage of dataKey as key.

The concept of the dataKey and key is similar to the one used in base table. A renderer will allow you to compute any relevant data. If you need to build a data that are in different levels, you just have to get to the highest level of the data structure and parse it accordingly.

What makes react json-schema-form a powerful plugin is that the templates, widgets and plugins of the form are stored in an object and parse accordingly. With that principle in mind, I created a buildSchema helper function that will return the whole schema with the data and user-defined value.

The schema output will map the schema with the existing data.

While it might look trivial, by defining a defaultSchema parameter, you can add any uiRenderer or extra key, value pair to be added to the schema. Returning a schema object will also help if you have custom UI rendering logic for different key. In your application, you will be able to define the UI rendering logic in the json schema. To use it in your UI component, you just have to map through the schema like this

Conclusion

That's it. This is how I handle nested data and introduce a data adapter to a data intensive application. Having a schema thaat I could quickly refer to contributes to making my data easy to reason about. This reduces the cognitive load for me when I'm developing. The added benefit of easily adding a uiRenderer allows me to build flexible custom UI. Want to render it as row? Want to give it a different font size? With this structure, it is easy to pinpoint the exact dataKey to update.

What are some techniques that you use to reduce the complexity of your project? Share with me below.

Comments

[nat_souljah]

Thats a great article, i currently have a normalised state in my redux, but now I cannot iterate over the object since its not an array, any help with that?