What you want to know about Web Push

The web-push API in the browsers today. And has its first tests mastered, but there is still room for clarification.

First we need to light the shade on what the web push actually is. Then How it looks and feels. Why we want web push. How to use it on our pages and apps and and finally what we need to be aware of when using push notifications.

With web push the user can get informed about updates on a website or web application. As the user, you don't need have that page open. The notification can contain any kind of information that is interesting to the app. In a social media app a private message can be send to the users device via web push. An online shop might want to inform about the availability of a new product. A blog or news page can let you know about new posts and articles.

Why Web Push?

The idea of notifications is not very new. From native mobile apps we know them for long. And they even got integrated to our desktop operating systems.

They became popular with the wide availability of mobile internet. I remember, when back in the days, my galaxy s1 had an empty battery after only a few hours. And I had to walk home from a party, because my mobile had no more power, but I was using facebook. That is why operating systems android and iOS have integrated a notification service. With it, many apps can receive updates through a single network connection. That can save lots of energy spend on Wifi or mobile signals.

If there is no general notifications channel on the OS or browser, the devices have to check one by one, each service that the user uses. Also, through a single high quality connection to a push notification service, messages can also be received very reliable.

Alternatives to web push notifications

RSS feeds are simple xml files, containing the complete or partial content of an article. These files are quite general and useful for blogs and news pages. There are other similar file formats like atom or json-feed. These work similar.

Social media apps often use webSockets or http polling, to bring messages in realtime to the user. Of cause that only works when the page is open and can cause extra bandwidth used.

Email or sms. yes, email or sms are viable alternatives. But they open in separate apps. There is a disconnect, compares with a notification that brings the user direct into the own app ready for engagement.

How does web push look.

Today, we sometimes get asked by a website if we want to receive push notifications. When using browser or a progressive web app (PWA), on the technical side web push notifications will be used. Often the website first ask in a custom html view/modal view, if we want to receive notifications. Sometimes even providing a choice of what kind of notifications are interesting to you.

After agreeing this dialog a second dialog will show up. This time native from the browser. When we agree to this dialog, the browser will create a subscription. The app return some credentials needed for sending notifications back to the app.

Now, as a user we can close the browser tab or window and can still be sure to sure not to miss any updates.

How does web push work

There are some very distinct unavoidable steps, that are happening in a very specific order.

First, there is a device, that is opening your webapp or website in a browser.
Embedded into the HTML or in a separate request, the page will load a cryptographic public key.

Using Javascript in the page, a new ServiceWorker get registered and create a web push subscription. When the browser has a subscription on the push notification service(PNS), the credentials get returned to the application. At the same time the browser will start listen with a single tcp connection to the PNS for new notification. The notifications on that one tcp connection can also belong to other apps and sites. But this communication is secure due to encryption and cryptographic signatures. The browser will make sure that the right app receive the notifications that belong to it.

The device listen continuously, but can also go offline and return back online. Notifications are still received.

As long as the subscription is valid, the app can send notifications to the users device. To push a message, the app developer needs the subscription credentials. These credentials consist of a cryptographic public key of the user and an authentication key. The server can package a notification. To identify the user device, the subscription credentials get used. To identify itself the app or site uses its own privateKey, that corresponds to the before mentioned public key.

The flow whole process get illustrated in the following animation:

1. Open the app or page.
2. Acknowledge wep push notifications with the user.
3. Create a subscription on the PushNotificationService (PNS).
4. Listen for notifications with a single connection to the PNS.
5. Send the subscription credentials back to the app.
6. Wait for an event to notify the user about.
7. App send notification to PNS.
8. Notification gets delivered to the device.
9. The app can show a browser notification.

How to implement web push notifications for your website and service

To work out how to use web-push on your site, you have three options. First read and follow the w3c rfc standard document.
Second: read the docs at MDN to figure out how service worker and web push works together. Or Third, follow some example, like in this article.

The code that I present in this article is mostly a derivations of the serviceWorker cookbook. But rewritten using async/await. Because browsers that are new enough to support web push are likely also support async functions. And async functions are better for understanding the examples. For more browser compatibility, I am sure you know how to compile the examples using bable, typescript or webpack.

For the browser we just need two files, a script that runs in the scope of our page and a js file that contains the logic for our service worker.

We start with the registration code:

<script>
// start by running an async function
registerWorker();
async function registerWorker(){
  // test if service workers are supported
  if ('serviceWorker' in navigator) {
    // create/register a serviceWorker.
    // the scope is the entire page
    // the code for js/serviceWorker.js is down below.
    //
    // Also, note that the scope is /, the root of your website.
    // The serviceWorker script is not allowed to be in a subdirectory such as /js.
    // if you need it, you can give that script a longer name.
    const serviceWorkerRegistration = await navigator.serviceWorker.register('/serviceWorker.js', {
      scope: '/'
    });

    // not sure why we are not directly await
    // the register method,
    // but this .ready property that is a promise.
    // it is just how the ServiceWorker API works.
    const registration = await navigator.serviceWorker.ready;

    // actually, the next line are for the
    // case the user refresh the page.
    // We test if there is already
    // a subscription and if so just stop here.
    const existingSubscription = await registration.pushManager.getSubscription();
    if (existingSubscription) {
      return;
    }

    // We want the subscription to be secure,
    // so take the public key from our apps
    // server.
    // The key is very short, so you can also
    // inline the key here to avoid the extra
    // request.
    const vapidPublicKey = await (await fetch('./vapidPublicKey')).text();

    // It is said, this is needed for chrome
    // browsers
    const applicationServerKey = urlBase64ToUint8Array(vapidPublicKey);

    // At this point the user is not
    // subscribed, so we do.
    const subscription = await registration.pushManager.subscribe({
      userVisibleOnly: true,
      applicationServerKey,
    });

    // After subscribing we need to store the
    // subscription information on our apps
    // server.
    // Later when we take a look at sending
    // notifications, we imagine there is a
    // json-server, that stores the
    // subscriptions into a db.json file.
    // but of cause you can store the
    // subscriptions how ever you want
    // depending on your needs and existing
    // architecture. In mongodb, sql, firebase,
    // cloud bucket,... you name it.
    await fetch('/yourAPI/webPushSubscriptions', {
      method: 'post',
      headers: { 'Content-type': 'application/json' },
      body: JSON.stringify(subscription),
    });
  }
}

// This function is needed because Chrome doesn't
// accept a base64 encoded string as value for
// applicationServerKey in
// pushManager.subscribe yet
// https://bugs.chromium.org/p/chromium/issues/detail?id=802280
function urlBase64ToUint8Array(base64String) {
  var padding = '='.repeat((4 - base64String.length % 4) % 4);
  var base64 = (base64String + padding)
    .replace(/\-/g, '+')
    .replace(/_/g, '/');
  var rawData = window.atob(base64);
  var outputArray = new Uint8Array(rawData.length);
   for (var i = 0; i < rawData.length; ++i) {
    outputArray[i] = rawData.charCodeAt(i);
  }
  return outputArray;
}

</script>

So this is the registration of a serviceWorker and creating a web push subscription. You see, it can be done almost without a server, that is why web push notifications are very well suited for a modern JAMstack.

Just so we don't miss it, here is the link to the json-server. In fact my personal blog is served using json-server with some custom middleware. (Do you like to hear more about json-server? ask me in the comments)

Did you see, that it is possible to just JSON.stringify the subscription? The json object will look like the following and that comes handy when we want to send the notification:

{
    "endpoint":"https://fcm.googleapis.com/fcm/send/fG81-cUNZp4:APA91bHblrMhOHGa7cxel5Lq4uZuhAj-58uh5fTnfxoLIx0kVvaWDEpFpYcZw_aHUmsazWA6JKFPycamYS-wQk79H2SofDkFRErNCZiW063PB3nCMwwsgFcrqeFV69DTzcqJDX-RNz6b",
    "expirationTime":null,
    "keys":{
        "p256dh": "BKlhdwZu2NDJ4KDb7EUwgcZ6SI3Z5yb3tlT8_VgGjfHskrFzrnZwDyoMee9TQCy7HkW8hAokSbEb-vFlYMQRHKQ",
        "auth": "16FA4eEGvd5GmDl0yStkJw"
    }
};

Now let's take a look at the implementation of the ServiceWorker:

// we only use the service worker now for web push notifications so we only listen on the push event.
self.addEventListener('push', event => {
    // we have to pass a promise to this waitUntil method, otherwise the worker get put back to sleep during our asyncronous operations.
    event.waitUntil((async ()=>{
    // as before we get the subscription
    const subscription = await self.registration.pushManager.getSubscription()
      if (!subscription) {
          throw new Error('User not subscribed');
      }
      const endpoint = subscription.endpoint;
      // the payload we can send via the push
      // message is quite limited, but we can
      // load what ever you need from the
      // server.
      // How exactly this is implemented
      // is up to you.
      const payload = await(await fetch('yourAPI/notificationText?endpoint=' + endpoint)).text();

      // Instead of loading content from our
      // server, you can also get the text
      // directly from the notification, like so:
      // const payload = event.data.text();

      // you see, showing the notification on
      // the users operating system, outside the
      // browser is actually not part of the
      // web-push standard, but a separate
      // browser feature, that is available in
      // the browser window and on the
      // registration inside the worker.
      self.registration.showNotification(
          'your app notification title',
          { body: payload }
      );
    })());
});

Generating a keyPair is actually so easy using the web-push package from npm, I will not comment it line by line. You see, it is a single function call then write the keys into their files.

const webpush = require('web-push');
const fs = require('fs');
const {publicKey, privateKey} = webpush.generateVAPIDKeys();

fs.writeFileSync(__dirname+'/../source/vapidPublicKey', publicKey);
fs.writeFileSync(__dirname+'/../vapidPrivateKey', privateKey);

So that is basically it, to setup the notification subscription. Then on your server side (your server, or a cloud function) when something happens, (a new post, product, any event) you can send a message to the user like this, the code is an example how notifications could be send via script from my local PC, after I publish a new article on my personal website. My website is a static site (#ssg) using hexo.

// the same module we used to generate the apps
// key pair, is used to send notifications.
const webpush = require('web-push');
const fs = require('fs');

// In this script we also need to set the keyPair
// to the lib.
webpush.setVapidDetails(
    'mailto:[e-mail]',
    fs.readFileSync(__dirname+'/../source/vapidPublicKey').toString(),
    fs.readFileSync(__dirname+'/../vapidPrivateKey').toString()
);

// We have before said to store the subscriptions
// into a json-server db.json file.
// Here we load the subscriptions.
const  { webPushSubscriptions } = JSON.parse(fs.readFileSync('./db.json').toString());

webPushSubscriptions.forEach(subscription => {
    // for every subscription we can now send
    // the notification.
    // remember every subscription is one user.
    webpush.sendNotification(subscription, 'Hallo my Web')
        .catch(err=>console.log(err));
});

What do you think? sending notifications is quite easy, right? For a small blog like mine, this small solution is enough.

For a serious news website, more sophisticated logic is needed. Like:

• When sending notifications, you should handle the error. on a 403 error the subscription has expired and can be removed from your db.
• Store in a proper db like mongo or sql.
• To improve privacy, you can generate more then one vapidKeyPair even go for one per user.
• Use the service workers subscription change event to resubscribe when a subscription expires.
• Depending on the use case send more personalized notifications and link the subscriptions with a user in your db.

You see, much more work and thought can go into the development of a good notification setup. That is also, why there are many providers, offering to maintain the subscriptions for you including different ways to manage the subscriptions or to have a unified API when pushing notifications not only to web, but also to native systems, such providers are very useful for sites that are statically generated using for example eleventy (#11ty).

Who is involved?

The rfc standard is developed mainly by mozilla and google, working together at the tc39. There is this github repository where you can ask questions or make suggestions.

Of cause there are all the publisher and app developer who want to push notifications to,... who guess it,... you, the user.

Then there are the browser vendors. Of cause there are Mozilla firefox, Chrome and all the developers of browsers. But browsers also often get installed bundled with the operating system. That is common for SmartPhones. While samsung for example leave this setting on default at google, are Chinese Phone manufacturers more likely to change that setting away from google, because google services are blocked in China.

Then there are the previously mentioned notification service providers: AdPush, OneSignal, PushEngage, CleverPush, SendPulse. Some also offer native, email, sms or extra management features. They get chosen for their service by the app or site developer, not by the end user.

Criticism

Criticism comes for various reasons. Mostly:

• monopoly
• protocol
• and functionality

Monopoly

First is the monopoly position that the device integrated push notification provider have. There are today mostly just two provider used. Mozilla and google. Microsoft could serve an other one for its edge users. Apply could support the web push protocol for ios and Safari users. The problem is, that as Chrome user (because it has the best dev-tools) My web push notifications have to go through google servers. And users of other platform do not have the choice to select an other service. Maybe a edge browser user would be happy to use the mozilla service, but he can't.

The monopolization get multiplied, when operators of push notification services also offer APIs as push notification provider and also as App developer with a large user base.

From the Monopoly position also arises the question what happen to all the data. The content might me encrypted, but the service still see who is receiving messages from what app or site. Does google know that I have a tesla because tesla notify all customer simultaneously about a new version of self driving on rails and the message also gets to me? Or I have some problem because my insurance provider send me a bunch of reminders? When I am interested in two competing product, can the notification publisher know to be treated equally? or does some competitor, big client of adWords, get extra treatment?

I don't want to hate on google, I am using its awesome services and trust the company with lots of my data. But being in this infrastructure critical position, that defines when people receive a message, a little time delay or a few dropped messages could shift sales could shift the market in highly competitive environments, such as car sales.

Protocol

Second the reliance on a proprietary protocol between the browser and the integrated push notification service. You can again take a look at the animation above. The colors are not chosen randomly. Green is for self implemented, what ever is developed by the site developer. Blue rely in standard. It can be proprietary software, but it follows the rules described in official standards and end user or site developer have an option to choose alternative. Red is not relying on any standards and as app developer or user we don't know what is going on, And do not even have the option for an alternative.

Firefox uses websockets, but there is no information about the protocol, on the connection. Is there JSON? Protobuffer? anything else? Also the service itself is closed source. We don't know if they collect all the notifications, until a computer is found to be fast enough to decrypt the messages. So as the protocol is unknown and the network communication is https encrypted, there is no way for us, to know if they store each transaction not only with a device identification, but also user app cookies and domain. Such data can greatly be used to find out more information about the user.

The proprietary protocol is not only a problem with privacy concern, but also the main point, that should be addressed, to allow third party services to host such service and the user to have the choice of the provider. It is often argued, that it is hard to provide a reliable notification service for the users and quality of service is most important. However, I rather loose a notification due to bad quality, than to a decision that something get kept from me. Or even just the feeling that it could be kept from me.

For personal use, I think big providers are more advantageous than, for example when I subscribe my games over the PNS of my employer. Or my p**n over my Churches server.

Functionality

Third there is the problem with how much you trust the apps that are sending notifications to me. Did you see the implementation of the service worker and how it did the avoidable API call? This is a great loophole to test if the users PC is on. I the service worker, you don't even need to show a notification to the user. The App can check silent when the user turn on his PC in the morning.

Also, I think browser developers do a great job, when asking the user before allowing an app to subscribe for notifications. Some websites get very creative of getting to yes. Requiring the allowance, before some feature like a download or a video get available. Such tricks are mostly used for spam messages, and can lead to more dangerous sites.

It is also not clear how long a subscription is valid or how long the push service will keep the notifications for the client. On a free market web push notifications, services would compete on the best features and quality.

Conclusion

Even dough I can understand this criticism, I don't think the situation is that bad. By showing up this criticism I do not want to stop you from offering push notifications on your website. After reading this article I hope that more developers want to offer push notifications on their website. I hope that the standard will develop into a more democratized direction. With more choice and better service. Opening up the standard, can allow more use cases to be covered, such as webmentions and other communication between services.

And who knows, someone of you can find the right words to convince the cs39. To add the PNS server API specification to the standard and add a configuration for web push notifications to the browser and operation system.