DEV Community: Ruben Costa Martinez

Performance | Vodafone x Google Mobile Web Hackathon 🏎️

Ruben Costa Martinez — Tue, 13 Apr 2021 16:15:15 +0000

3 eager Vodafone developers and I joined Google at their Kings Cross Campus to take part in a Mobile Web Hackathon 🛠.
The day consisted of how to improve the performance of your mobile web applications to present as quickly as possible content to your users.

Why does it matter? 🤷

Today consumers expect to have everything accessible to them and that access available at relative speed. At Vodafone, we aim to create web applications that run fast enough so that our customers without a top tier device or solid connection can still enjoy our applications without seeing a dreaded loading spinner.

Time to get organised 🗄️

The team we brought along to hack consisted of developers who all worked in different areas of Vodafone’s web portfolio, so we had to get organised:

To start, we split our developers into two groups to be more agile and focused on common solutions to improve the performance of our project, we looked at:

Preparing the development environment so we could test each solution faster and see if it had a performance impact.
We used our pay-as-you-go repository because it has a clean structure and was easier to approach for all our developers.
We tried not to get too distracted by the amazingly good looking food they brought. (Probably the hardest thing we had to do!) 🍕

Once we got this in place we started working on the solutions.

What are thooooose!? 👟

These are the initial and final metrics after applying all the performance solutions. Among them are:

Preload fonts + convert WOFF to WOFF2 🐶 (woof?)

We preloaded the fonts and changed the format of our fonts to WOFF2 which offers a 30% average compression gain over the original WOFF, but is not supported on IE 11. That being said, for that particular case we need to fallback to WOFF.

Image optimisation + next-gen formats 🖼️

We changed all the PNG images to the WEBP format which offers 26% smaller file sizes. Also, we applied some lazy load instead of load all of them at once. We only needed to load the first 6 images that appear on screen in order to achieve faster times. The rest could be lazy loaded.

Brotli compression 🥦

We used Brotli to compress the files:

Javascript files compressed with Brotli are 14% smaller than gzip.
HTML files are 21% smaller than gzip.
CSS files are 17% smaller than gzip

For the browsers that don’t support Brotli compression we fallback to gzip which is still better than no compression at all.

Bundle splitting 🖖

Before we were loading all the files at once in one big fat bundle so we split the bundle by route level. This gave us quite a big score bump in the lighthouse metrics.

In the future, it would be great to go deeper into the component level and load them with dynamic imports.

Async loading analytics 👨‍🏫

Still with all those changes above we were not getting the results we wanted, and you could tell, after profiling the page, that analytics were the bottleneck.

Our application is waiting until the analytics were loaded giving us a very low TTI (time to interact).

We changed that by loading the analytics asynchronously. This might be controversial as there’s no way to guarantee 100% that the first events will be fired but when we tried, they were indeed being fired as it they were being loaded in async mode which is faster than defer, so we decided to go with it.

After doing that, we observed another big bump in the lighthouse metrics.

Next Steps

And just like that, we went from 15% to 80% lighthouse score but… there is still room for improvement!

1️⃣ We would like to set a performance budget and compare current and previous score on Pull Request.

2️⃣ Techniques like tree-shaking could be nice to apply to attempt to not load all of the libraries in one go.

3️⃣ We want to try to inline the critical CSS and JS, improve the cache control as well as compress the SVGs.

4️⃣ Last one would be to take a look at the unused code. We could observe in the google dev tools that more that 90% of our JS and CSS code was unused. That’s a lot! But might be related to number 2.

We are also going to look at our other journeys to see if we can leverage what we learned to improve our customers experience online with faster load times!

Progressive Web Apps

Ruben Costa Martinez — Mon, 12 Apr 2021 14:11:01 +0000

With home broadband and smart phones, we are increasingly relient on the internet, but sometimes, the internet isn’t there. That feeling of being abandoned by connectivity is just terrible. You could be watching your 13 video of funny cats in a row but all you get is the loading icon. Total failure of connectivity is a worst case scenario, that’s why we are going to cover how to create apps that work great what ever connection the user has ussing the Service Worker to intercept network traffic.

We should approach the application from an offline first point of view, this means showing many things on the screen as possible using stuff already on the user’s device in caches and such.

What is the Service Worker?

So, this Service Worker thing is really good, and I mean REALLY good, it’s just a simple JavaScript file that sits between you and network requests, it can’t access the DOM and the user doesn’t see it. What it does is it intercepts requests as the browser makes them. From there, you can send the request off the network, as per usual or you can skip the network, run some cache stuff or code a custom response, whatever you wish.

You register for a Service Worker giving in the location of your service work a script and it returns a promise.

if (!navigator.serviceWorker) return; // For old browsers
navigator.serviceWorker.register('/sw.js').then((reg) => {
 console.log('Registered!');
}).catch((err) => {
 console.log('Whoopss :(');
});

You can give it a scope too and it will control any page at that URL and also anything with a deeper URL but not anything with a shallower URL.

navigator.serviceWorker.register('/sw.js', {
 scope:'/app/'
});

And what does the Service Worker do? Well, it listens to events like:

self.addEventListener('install', (event) => {
 // ...
});
self.addEventListener('activate', (event) => {
 // ...
});
self.addEventListener('fetch', (event) => {
  // ...
});

The event response contains information about all the requests made by controlled pages as CSS, images, fonts and you can mess around with requests, changing headers or responding with something entirely different.

That’s why Service Workers are limited to HTTPS because when you’re serving across plain unencrypted HTTP through the network, any one in the middle can remove, modify or add content. Malicious scripts could also capture the data you input, read cookies, modify databases…

Service Worker Lifecycle

In a refresh of a page, there is an overlap between the old page and the new page. With the new request the Service Worker is registered. The Service Worker only takes control of pages when they’re loaded that means any additional requests by this page will bypass the Service Worker. Refreshing the page makes the Service Worker that was up and running take control of the window client, now the request did pass through it. That explain why does it take two refreshes to see logs request.

What happens if we change the thing we are logging? We still see our old request being logged(our old code) that is because if a page loads by its Service Worker, it will check for an update to the Service Worker in the background. In the case of finding it has changed it becomes the next version, but it doesn’t take control until all pages using the current version are gone, to make sure, there’s only one version of our site running at the same time like native apps. Thats why a refresh doesn’t let the new version take over. Because the window clients are overlapping when refreshing, the Service Worker doesn’t stop controlling it. In order to let the new waiting Service Worker take place we need to close the actual window client. When we do that, the new Service Worker takes over and future page loads will go through the new one.

This update process is the same as browsers such as Chrome use. Chrome downloads the update in the background and it doesn’t take over until the browser closes and opens again. It notifies you with an icon changing the color. When the browser refetches a service worker looking for updates it will go through the browser cache, that’s why it is important to keep cache time on the Service Workers short as zero. Also in case of setting the Service Worker script to cache for more than a day, the browser will just set the cache to 24 hours, that means the update checks will bypass the browser cache if the Service Worker is over a day old.

Hijacking Requests

How to catch the request so nothing goes to the network and respond ourselves is really easy, we just need to call event.respondWith(). It takes a response object or a promise that resolves with a response. The first parameter is the body of the response, which can be a buffer, blob and other things like, in this case, a simple string. We can add headers too to create some HTML response!

self.addEventListener('fetch', function(event) {
 event.respondWith(
  new Response('Hi! my name is <b>Ruben</b>', {
   headers: {'Content-Type': 'text/html'}
  })
 );
});

Now that we know how to respond to a hijack request let’s go to the network for the response and with the API Fetch we will make a network request and respond to requests if their URL ends with “.png” with our own image.

self.addEventListener('fetch', function(event) {
 if (event.request.url.endsWith('.png')) {
  event.respondWith(
   fetch('/imgs/sample-image.jpg')
  );
 }
});

Right know we only showed something based on the request URL but in the real world we need to be more dynamic like for example, the page can send a request which we intercept and send it to the network but instead of sending the response back we are going to look at it and do another stuff.

The fetch method can take an URL or all the request and returns a promise, so we can use .then and .catch, and if the responses are 404 Not Found, we could respond with our own message.

self.addEventListener('fetch', function(event) {
  event.respondWith(
   fetch(event.request).then(function(response) {
    if(response.status == 404) {
     return new Response("404 Not Found!");
    }
    return response;
   }).catch(function(){
    return new Response("Something went reeeaally wrong!");
   })
  );
});

Now if we go to a page that doesn’t exists, we get our 404 message, and if we turn our page offline we get our custom message too! We could use this technique to get a response from multiple sources and act based on them, you can even go to the network and if that fails… get another thing from the network!

Caching Assets

If we want to load our app without using the network we need somewhere to store the HTML, CSS, JavaScript, images, fonts… and we can do that thanks to the cache API.

The cache API give us this caches object on the global so if I want to create or open a cache we can use caches.open() passing the name of the cache. That returns a promise for a cache of that name and if is not yet created, it will create one and return it.

caches.open('my-cache').then(function(cache) {
 // ...
});

The cache box contains request and response pairs. We can use to store anything from both our own origin or from elsewhere on the web.

We can add cache items using cache.put() and pass a request or a URL and the response or we can use cache.addAll() this takes an array of requests or URLs, then fetches them and puts the request and its response pair into the cache.

cache.put(request, response);
cache.addAll([
 '/thing1',
 '/thing2'
]);

The operation addAll is atomic, that means if any of these fail to cache, non of them will be added to the cache, addAll uses fetch under the hood so understand that requests will be going via browser cache.

If we want to get something out of the cache we can use cache.match() passing in a request or a URL. This will return a promise when there is a match or null otherwise. Caches.match() is the same but it tries to find a match in any cache starting with the oldest.

cache.match(request);
caches.match(request);

If we now want to start storing things we can use another Service Worker event. The install event is fired after a Service Worker is runned by a browser for the first time.

self.addEventListener('install', function(event) {
 event.waitUntil(
  //...
 );
});

Event.waitUntil() lets us control the progress of the install. We pass in a promise and if the promise resolves then the browser knows the install is complete, and if the promise rejects then it will know it failed and this Service Worker should be discarded.

Let’s use cache to cache our stuff then. Event.waitUntil() takes a promise and cache.open returns one, so we can create the cache inside de waitUntil() event and use addAll() to cache all the URLs of our site.

self.addEventListener('install', function(event) {
 event.waitUntil(
  // We open a cache named example-static-v1
  // then we add cache the urls we want

  caches.open('example-static-v1').then(function(cache) {
   return cache.addAll([
    '/',
    'js/main.js',
    'css/main.css',
    'imgs/example.png',
    'https://fonts.gstatic.com/stats/Roboto/normal/400'
   ]);
  })
 );
});

Now if we run our site, we can go to the Application panel and we can see the resources we stored inside the Cache Storage.

It’s time to put this in practise and know that we have resources in the cache we can delete the previous code handling the 404 response in the fetch event and use it to respond with an entry for the cache if there is one and if there isn’t, fetch it from the network.

self.addEventListener('fetch', function(event) {
  event.respondWith(
   caches.match(event.request).then(function(response) {
    if (response) return response;
    return fetch(event.request);
   })
  );
});

Now if we refresh the page seems like we didn’t do any change but if we go offline we get our content still thanks to the cache and if we get a low connection we will get out content a lot faster now.

To have this cycle working better now we need to update our content in the cache. This is because we cached the HTML once at install time so we are stuck with the same content in the cache.

So to have the perfect offline experience we need to do:

Unobtrusive app updates.
Give the latest version to the user.
Keep updated the cache of the content.
Cache photos, avatars, etc.

Updating The Static Cache

Say we wanted to change the CSS of the app, if we refresh our app we won’t see any changes because our cache still contains the old CSS. Our cached CSS is updated in the install event but we are changing the CSS not making any changes in the Service Worker so the install event is not being called. We need to work with the Service Worker to get it to pick up changes.

To make the CSS to update, we need to make a change to the Service Worker so the browser knows there is a new version of it, and we’ll have an installing Service Worker because is new, so it’ll fetch the JavaScript, HTML, and the updated CSS and store them in the new cache. This doesn’t happens automatically, we must change the name of our cache to make this happen. Then after the new Service Worker takes control we should delete the old cache so when we refresh the page we can get the latest CSS. To make that change in the Service Worker we could just rename the name of the cache from v1 to v2.

To get rid of the old cache we are going to use another Service Worker event: activate. The activate event will be fired when the Service Worker becomes active, when it’s ready to control pages and the previous one is gone.

self.addEventListener('activate', function(event) {
 //...
});

This is the perfect time to delete old caches too. In this event we can also use event.waitUntil() to control the process. While you are activating, the browser will queue other Service Worker events such as fetch so we can delete the old cache because the new one are coming. To delete the old cache we can use caches.delete()

caches.delete(cacheName);

and you can get the name of all of your caches using caches.keys()

caches.keys();

both methods return promises.

The easy way to do that would be to change the version name and call caches.delete() passing in the old name in the activate event but this is not scalable. To make it scalable we should store the name of the actual cache in a variable.

let staticCacheName = 'example-static-v2';

and refactor the code so we’ll place the variable instead of the actual name. Then, in the activate event we can get all the names of the caches that exist and filter them because I’m only interested in the caches that start with ‘example’ but that they are not staticCacheName. So in that way we will have a list of caches that we don’t need anymore, that we can delete. We can put this logic inside a Promise that looks like this.

self.addEventListener('activate', function(event) {
 event.waitUntil(
  caches.keys().then(function(cacheNames) {
   return Promise.all(
    cacheNames.filter(function(cacheName) {
     return cacheName.startsWith('example-') &&
            cacheName != staticCacheName;
    }).map(function(cacheName){
     return cache.delete(cacheName);
    })
   );
  })
 );
});

We have now completed the ‘Unobtrusive app updates’ of our to-do list.

Give the latest version to the user

When a new Service Worker is discovered it waits until all pages using the current version go away, before it can take over and that might take a long time, instead we can warn the user there is a new version of the app and give him a button to dismiss the update or another one to take the new version. To achieve this we need the update notification. Thankfully we have APIs that give us insight of the Service Worker lifecycle.

When we register the Service Worker it returns a promise. That promise will give us a Service Worker registration object which has properties and methods related to the Service Worker registration. We get methods to unregister or trigger an update on the Service Worker and free properties: installing, waiting, active that will point to a Service Worker object or be null. Also the registration object will emit an event when a new update is found called updatefound of course, classic.

navigator.serviceWorker.register('/sw.js').then(function(reg) {
 reg.unregister();
 reg.update();
 reg.installing;
 reg.waiting;
 reg.active;
 reg.addEventListener('updatefound',function() {
  // reg.installing has changed
 });
});

On the Service Worker objects themselves, we can look at their state

let sw = reg.installing;
console.log(sw.state); // Logs the state

and it could be:

installing: the install event has fired but hasn’t yet completed.

installed: installation completed successfully but hasn’t yet activated.

activating: the activate event has fired but not yet completed.

activated: the Service Worker is ready and it can receive fetch events.

redundant: the Service Worker has been thrown away, being replaced by a new one or by a fail of an install.

The Service Worker fires an event when the value of the state property changes.

sw.addEventListener('statechange', function() {
 //sw.state has changed
});

Also, navigator.serviceWorker.controller refers to the Service Worker that controls this page, so, if you want to tell the user there’s an update ready we need to look at the state of things when the page loads and listen for future changes. For instance, if there is no controller, it means the page didn’t load using a Service Worker so the content we see is from the network.

if (!navigator.serviceWorker.controller){
 // page didn't load using a service worker
}

Otherwise we need to look at the registration:

if (reg.waiting) // there's an update ready, we tell the user about it.

otherwise if there is a installing worker there’s an update in progress

if (reg.installing){ // there's an update in progress
// this update might fail so we listen to the state changes to track it
 reg.installing.addEventListener('statechange', function() {
  if(this.state == 'installed') {
   // there's an update ready, we tell the user about it
  }
 });
}

otherwise we listen for the update found event, when that fires we track the state of the installing worker and if it reaches the installed state, we tell the user!

reg.addEventListener('updatefound', function () {
 reg.installing.addEventListener('statechange', function() {
  if (this.state == 'installed') {
   // there's an update ready, we tell the user about it
  }
 });
});

In conclusion, when there’s an update ready we could call a method that prompts an alert to the users telling them there’s a new version. Something like a toast for example.

MainController.prototype._updateReady = function () {
 let toast = this._toastView.show("New version of the app available!" , {
  buttons: ['exampleButton']
 });
};

Now if we make any changes to the Service Worker, for example, bumping the version up, the user will get a notification. The notification doesn’t do anything yet, it’s useless right now, we have to make it work so it actually updates the app version for the user.

Triggering an update

We have a notification with an useless button that we need to make it work. Clicking this button needs to tell the waiting Service Worker that it should take over straight away bypassing the usual life cycle. Then we want to refresh the page so it reloads with the latest assets from the newest cache.

There are three components that let us accomplish this. A Service Worker can call skipWaiting() while it’s waiting or installing.

self.skipWaiting();

This tells it shouldn’t queue behind another Service Worker, it should take over straight away. We want to call this when the user hits the ‘Update’ button in our update notification.

But, how do we send the action from the page to the waiting Service Worker? We can send messages to any Service Worker using postMessage,

reg.installing.postMessage({foo: 'bar'}); // from a page

and we can listen to messages in the Service Worker using the message event,

self.addEventListener('message', function(event) { // in the SW
 event.data; //{foo: 'bar'
});

so we will send a message to our Service Worker telling it to call skip waiting.

Finally we’ve already seen navigator.serviceWorker.controller but the page gets an event when its value changes, meaning the new Service Worker has taken over.

We’ll use this as an action to refresh the page.

navigator.serviceWorker.addEventListener('controllerchange', function() {
 // navigator.serviceWorker.controller has changed
});

We’ll apply this to our previous mentioned method _updateReady.

MainController.prototype._updateReady = function () {
 let toast = this._toastView.show("New version of the app available!" , {
  buttons: ['update', 'dismiss']
 });

 toast.answer.then(function(answer) {
  if (answer != 'update') return; // If he clicks on dismiss return
  // Otherwise tell the Service Worker to skipWaiting
  worker.postMessage({action: 'skipWaiting'});
 });
};

And in the Service Worker file we are going to listen for the message event, and call skipWaiting if we get the appropriate message to make this Service Worker take over pages.

self.addEventListener('message', function(event) {
 if (event.data.action == 'skipWaiting') {
  self.skipWaiting();
 }
});

Now we need to react to skipWaiting being called so back in the page controller (MainController) im going to listen for the controlling Service Worker changing and reload the page.

navigator.serviceWorker.addEventListener('controllerchange', function() {
 window.location.reload();
});

You can use this in any way you wish. If the update has minor things, maybe don’t bother the user at all, let them get the update naturally, or for example if the update contains an urgent security fix maybe refresh the page directly, no matter what the user is doing. That wouldn’t be a nice experience, but it’s better that the user using an app with a major security flaw.

We have now completed the ‘Give the latest version to the user ’ of our to-do list.

Now we need to ‘keep updated the cache of the content’.

Caching The Page Content

Finally instead of caching the root page we will cache the /skeleton and bump the version of the cache.

self.addEventListener('install', function(event) {
 event.waitUntil(
  // We open a cache named example-static-v1
  // then we add cache the urls we want

  caches.open('example-static-v1').then(function(cache) {
   return cache.addAll([
    // '/',   <--- We change the root
    '/skeleton',
    'js/main.js',
    'css/main.css',
    'imgs/example.png',
    'https://fonts.gstatic.com/stats/Roboto/normal/400'
   ]);
  })
 );
});

Now in the fetch event we are going to respond to requests for the root page with the page skeleton from the cache.

self.addEventListener('fetch', function(event) {
  let requestUrl = new URL(event.request.url);

  if (requestUrl.origin === location.origin) {
   if (requestUrl.pathname === '/') {
    event.respondWith(caches.match('/skeleton'));
    return;
   }
  }
  event.respondWith(
   caches.match(event.request).then(function(response) {
    if (response) return response;
    return fetch(event.request);
   })
  );
});

In this case we only want to intercept route requests from the same origin that’s why if it’s the ‘root ’we respond with the ‘skeleton ’ from the cache, we don’t need to go to the network as a fallback because whe have skeleton cached as part of the install step.

At this point we have a pretty good product, something that we could launch. It doesn’t change a lot if we have perfect connection but perfect doesn’t exist.

On a slow connection we get to render the top bar and content much quicker because our JavaScript is arriving sooner and with offline we are showing something at least better than the no connection message of the browser.

Even at this point, we can really improve the user experience by caching the posts of the content and to do that we can use one of the most hated APIs. Tense music IndexedDB.

But that will be in another post, we’ll try to use the good things of it, maybe it won’t be that bad right?

Maybe.

Test post with long title for testing purposes only to manage this as a cms

Ruben Costa Martinez — Mon, 05 Apr 2021 19:39:34 +0000

This is a test post to try to use dev.to as a cms. This is a test post to try to use dev.to as a cms. This is a test post to try to use dev.to as a cms. This is a test post to try to use dev.to as a cms. This is a test post to try to use dev.to as a cms. This is a test post to try to use dev.to as a cms. This is a test post to try to use dev.to as a cms. This is a test post to try to use dev.to as a cms.