“Nobody goes offline” on Web Apps — or do we?

“Nobody goes offline” on Web Apps — or do we?

tl;dr

This article is about ServiceWorkers and architecture in modern web applications including consideration of the following statement.

“Nobody goes offline […] there is just intermittency”

[Lee Byron, Facebook, #FullStackFest 2016]

At the Full Stack Fest 2016 in Barcelona, Lee Byron talked about Immutable User Interfaces and the problems of established architectures like MVC. This was a presentation about how we can build better architectures in the Frontend.

Trying to summarize with very few words:

Use advanced query languages in the Frontend to retrieve data (REST really does not handle relations and sub-queries well) and build state-based Frontend applications that accumulate states instead of vanishing the old state and overwriting with new data (which would be MVC btw). Why? Because this allows you to have a so-called optimistic state which you can apply before getting the server response and rollback when you have the server response which failed.

Okay, Lee is Facebook Engineer so translating this in Facebook Terms would be: Use GraphQL and React , Redux and avoid frustrating the user with loaders. So far, so good.

“Nobody goes offline”

What he meant is: Nobody goes willingly offline saying “i would really like to cut-off my internet connection right now” — which is true.

Now take a look at famous platforms like Twitter, Facebook, eBay, Pinterest, etc. At all of these platforms we can assume that the user at maximum has “long intermittency” but does not go willingly offline. So from this perspective the claim that “Nobody goes offline” can be confirmed.

I built a PWA web application that goes offline — for hours

In a project I am doing right now I work together with the people of nodus medical GmbH. These are people from the airforce as well as surgeons who requested me to build an MVP to improve efficiency in the medical field.

Now, let’s not talk about what this application does but let’s talk about the requirements/circumstances and how to solve the problems. It is used in fields where internet connection is partially not available for several hours or not allowed because of security reasons. So you can say that the user at least goes knowingly offline.

What is the problem with knowingly going offline?

1. If you have missing assets like uncached templates, that part of the application will not be useable
2. Having temporary (non-persistent) optimistic states is an easy way of loosing data in the meantime you were offline (unintentional reload, tablet os crashed, …)
3. Trying to synchronize with the server every time is senseless (even though from performance perspective it is cheap but still: senseless as the app is aware of the fact that this is not only a matter of seconds or minutes)
4. Dealing with 2 types of data: synced and unsynced — including establishing relations between unsynced data and synced data. This may sound easy at first but let’s dig deeper into that in the corresponding section.

1. Cache all the assets

Some say offline first means providing a way of using the website/app when internet connection is broken. I’d call that offline-featured. Offline First on the other hand is making the app work offline and then providing a way of connecting to the web.

ServiceWorkers it is. I use the Cache and tell the SW which assets my application will need in order to run. In my case I want the whole (Angular) app to work offline so I needed to cache the lazy-loaded templates (partials) too. Cool, that’s easy.

2. Persistent and safe data: Use IndexedDB

As mentioned above I do not think using temporary states is a good idea when you know that user will at least be offline for an hour. So if you’re like me and like building your own abstraction layer, you could use Dexie.js (SQL-like querying in the frontend). If you’re using Redux you can search the web for an extension that allows persistent storage for Redux.

For my project the whole database had to be available in the Frontend so it made also sense to go “local database first” in the Frontend with Dexie. That said: I am not even trying to fetch data from the server when the user requests data. Everything done locally.

BTW, If you think of WebSQL right now: It’s dead.

3. Trying to sync with the server. BackgroundSync?

So we cached all the assets and have all the data locally. But we still need to synchronize with the server to ensure that this data can be also accessed from other devices and to make redundant backups.

ServiceWorkers implement an API which is broadely known as BackgroundSync but lives under .sync . There is also an analogous APIcalled .periodicSync which implements similiar behaviour but periodically.

Sync allows you to send data to the ServiceWorker which then is processed as soon as your internet connection allows it. Assuming you created a chat application and you want all messages to be on the server ASAP this comes in very handy. Assuming you have implemented a ToDo-List you’d probably even want to periodically check and compare your data.

But you are leaving decision-making up to the browser. E.g. it could be that your sync is delayed because of battery capacity or because other processes have higher priority. Also it could happen that the browser decides to not further try syncing at all.

BackgroundSync is extremely helpful if you consider failure and delay happening. Being aware of what it does in the background is mandatory. More complex when you want to implement active control of the syncing in the FE as a communication channel between SW and FE must be established then.

In my ServiceWorker application the Sync APIs could not provide additional business value so I discarded the use of them. This was because of UX requirements that the syncing must be done consciously. The only thing that could’ve been done would be to allow the user to click the Sync button, leave the application and let the user know via Notification when syncing is done. But then again you would not even need the Sync API for this but could just fetch in the ServiceWorker.

4. Welcome to the fight club: Synced vs. Unsynced data

Building a truely offline-first application you’ll easily run into trouble if you’re not having second thoughts on how data is synchronized.

A lot of iOS/Android/Web applications nowadays just show you a dialogue locking your application like:

Ouch, you’re offline. Please go online again.

That is a simple but annoying trick to not create conflicts. However, if you want to allow to manipulate data offline from multiple different users, you can easily destroy your data if you did not take care of this in your architecture.

If you know Git Merge Conflicts you already know the solution. Do not overwrite any data on the server but rather give them a full-fledged history (user, time, accepted, …) to then allow the person that is syncing to resolve conflicts in the UI by selecting the wanted item.

Wrap-Up

With web apps partially replacing native apps there are true offline-first apps. We can make use of the Sync Interfaces for data synchronization and IndexedDB for complex storage. By having data offline for management but on the server for backup and syncing reasons you have decentralized data which leads to Multiple Points of Truth. This must be solved via versioning.

Comments

[Jordan Soo Yen Yih]

Just a sharing that to have a better user experience, you may take a look on PouchDB (client side), and CouchDB (server side). You can do a master master data replication to your client side which gives the advantage of reading what you wrote during offline, for cache offline approach, u still able to write your data but you cannot read what you wrote. Of course there are also trade off for the pouchdb approach, which you cannot have running number in your data, extra effort to solve some distributed problems like Idempotent and etc...