How hidden files can screw you over when deploying progressive web apps

#hiddenfiles #firebase #webpack #swprecache

Hidden files such as the DS_Store on macOS can really make your life miserable and produce strange abnormalities during deployment if you are not paying attention. Read on to find out how I wasted 1 hour trying to figure out why the service worker on my progressive web app was not working correctly.

Background

I have been working quite frequently on developing progressive web apps recently and I usually fire up a quick PWA with generators such as create-react-app or in this case using preact-cli. These generators have an advantage that all the hard service worker configurations are already done for me and I just need to focus on building the app logic while the starting point is already highly optimized for browser performance. We will see why this is not always a positive if we are not careful.

What are hidden files

These are files such as .DS_Store on macOS or Thumbs.db if you are on Windows. .DS_Store stores the directory properties such as the position of icons or background image of the folder, which is a generally a pretty useless file for most developers to even pay attention to and that makes it even more dangerous when it causes problems for us during a deployment.

Service worker caching

In a preact-cli inited PWA, the service worker will aggressively cache all assets and resources using the browser Cache API for faster loading. This happens as part of the automated build process. Preact is using the sw-precache library here to do most of the heavy lifting for us. The auto-generated files array looks like this:

var precacheConfig = [
        ["/app/index.html", "fa243faa5a7ba7dd91c2241a62010a67"],
        ["/app/not-exist/index.html", "14e3d8a57e9e195437405c1631413240"],
        ["/app/pnr/index.html", "741939de6641dfeb520d8f3bf8288192"],
        ["/app/schedule/index.html", "330df6ceb7147e93a2e3e6daec6acb72"],
        ["/app/static/about/index.html", "4accf5ab4d635725099660c1ec9b09fd"],
        ["/app/static/contact/index.html", "a4a1b570fdb015996fa4d9d52bf38733"],
        ["/app/status/index.html", "9a8ed5b95b2d6dbf4a637e084dedc99c"],
        // more files...

The .html are pre-rendered files generated by preact-cli which will be cached automatically for us to allow first load to be as fast as possible.

When caching goes wrong

So in this particular progressive web application, we have been deploying and registering the service workers just fine until I received a few emails from my client at 4am saying certain features like "Add to Home Screen" is broken and all the pages are being slow.

The first thing I do is to open up the web app and use the Chrome Lighthouse auditing tool to quickly find out potential problems with performance and service workers. This is what I see.

Ouch, a PWA score of 64, that is not good because I remember promising the client a score of at least 85. So something is clearly wrong with the last deployment. Let's scroll down a bit more to see:

Here is the issue. The service worker has not been installed correctly so none of its benefits are going to work in the application. The service worker must have encountered some kind of error during the installation process. The next step is to take a look at the Application tab to see what's going on with the service worker. Looks like we have an error log here as well.

Looks like there is an error that the DS_Store hidden file is not founded. Let's see the full sw.js file and prettify the javascript to see what's going on.

It seems that the build process wrongly assumed that this file should be cached and is therefore included in the list of files to be cached by the browser. So what is the problem? Well, during the deployment we rightfully ignore all hidden files such as this firebase.json indicates here:

  "hosting": {
    "public": "build",
    "ignore": [
      "firebase.json",
      "**/.*",
      "**/node_modules/**"
    ],

So the .DS_Store files will not be found and therefore will cause the service worker to not install correctly and rendering all our performance optimizations useless.

The solution to this problem was to remove the hidden files, rebuild, and redeploy the app. Now the service worker installs just fine. I might make a shell script that checks the existence of nasty hidden files before running the build as a further step to be further sure if I really wanted to make sure this does not happen again.

Conclusion

This caught me off guard as I have deployed this application several times already without any problems. One solution is to disable the automatic creation of DS_Store files on macOS.

I hope reading this will mitigate some potential frustrations for some of you or at least save some time finding the problem. Let me know if you came up with any good solutions or have come across similar issues in your work.