Paul Walker

Posted on Nov 23, 2021 • Originally published at solarwinter.net on Jul 17, 2021

Live reload in Hapi

#hapi #hapijs #programming #javascript

Something I’ve been meaning to address for a while is the lack of a livereload plugin for Hapi. Template changes are visible on manual reload if you turn off caching, but I’ve become used to automatic reloading when developing with Vue.

livereload architecture

The full livereload protocol is available for everyone to view. In brief, it requires two parts.

Server

The livereload server accepts HTTP and WebSocket connections (on the same port). It’s responsible for serving the livereload client code, and for monitoring the filesystem for changes. It also listens for messages from the client indicating URL changes.

When changes happen, it sends a message to the client (over WebSocket) indicating which file has changed so it can be reloaded if needed.

Client

The client runs in the browser - it makes a connection to the server and waits for a message. When it receives a message indicating a reload is required, it does it.

(This is a rather abbreviated explanation…)

I had no intention of actually re-implementing the client - I just wanted to automatically insert the fragment which causes the client code to be loaded and start listening.

Plugin structure

A Hapi plugin seemed to be the best place to start. I knew from previous work that the Hapi server has “extension points” in the request/response lifecycle, which can be hooked into and used to monitor or modify the request or response.

The basic structure of a Hapi plugin is simple - it’s an object which exports a structure named plugin containing a function named register() (and a bit of metadata). The register function receives a server object and an (optional) options object. It does what it needs to do, and then returns or throws an exception.

A very basic plugin is below. It exports the structure Hapi is looking for in a plugin. The register function creates a route which says hello to the user, or the whole world if the user doesn’t specify who to greet.

'use strict';

async function register(server, options = {}) {
  server.route({
    path: "/hello",
    method: "GET",
    handler: function () {
      const name = options.name || "world";
      return `Hello ${name}\n`;
    }
  });
}

module.exports = {
  name: 'hapi-reload',
  version: '0.1.0',
  register: register
};

There are multiple ways of declaring your plugin - the Hapi docs cover it all well. For now we’ll keep it simple, and put our plugin code in a file named plugin.js.

We also need to get a server up and running, so we have something to plug in to.

const Hapi = require("@hapi/hapi");
const hapiLivereload = require("./plugin");

async function run() {
  const serverOpts = {
    port: process.env.PORT || 4000,
    host: '0.0.0.0'
  };
  const server = Hapi.server(serverOpts);

  await server.register({
    plugin: hapiLivereload
  });
  return server.start();
}

run()
  .then(() => console.log("Server up"))
  .catch(err => console.error("Couldn't run", err));

And - it works.

$ node dev/server.js
Server up


$ curl http://localhost:4000/hello
Hello world

We can add the name option to customise the response.

  await server.register({
    plugin: hapiLivereload,
    options: {
      name: "Paul"
    }
  });

And now it knows me!

$ curl http://localhost:4000/hello
Hello Paul

Which hook to use?

So that’s a basic plugin, but it doesn’t do anything particularly useful - you can probably think of easier ways to say “hello world”.

Let’s start looking at the request lifecycle again. Looking through the list of steps the onPreResponse looks good - the response has been generated but not yet transmitted, and better still:

the response contained in request.response may be modified (but not assigned a new value). To return a different response type (for example, replace an error with an HTML response), return a new response value.

Let’s try a new approach - rather than a specific route to greet the user, let’s just make all the responses do that. Much more friendly.

async function register(server, options = {}) {
  server.ext({
    type: 'onPreResponse',
    method: function(request, h) {
      const name = options.name || "world";
      console.log(`Replacing "${request.response.source}"`);
      return `Hello ${name}\n`;
    }
  });
}

The console.log shows us what we’re replacing (useful for debugging); and we just return the new response text we want, which the docs say should work.

To test it, we’re going to need a route which doesn’t return Hello world. We can add that to run() in our server code.

  server.route({
    path: "/",
    method: "GET",
    handler: () => "I should be an index page\n"
  })

And if we try it?

Server up
Replacing "I should be an index page
"


$ curl http://localhost:4000/
Hello Paul

Excellent!

livereload register

The register function for the livereload plugin is basically the same. The differences are specific to the plugin, working out the location of the client code.

const logger = require("util").debuglog("hapi-livereload");

let host, port, src;

async function register(server, options = {}) {
  host = options.host || "localhost";
  port = options.port || 35729;
  src = options.src || '//' + host + ':' + port + '/livereload.js?snipver=1';

  server.ext({
    type: 'onPreResponse',
    method: addSnippet
  });
}

The bit we haven’t covered yet is that addSnippet function, which is where things got a little more involved - actually modifying the webpage being returned.

We also add a logger function using Node’s standard util.debuglog functionality, so we can report back to the user if they want us to.

Modifying the HTML

I spent some time looking for an HTML parser. It was complicated by the fact that the plugin not only needs to parse the outgoing HTML page, but we also have to be able to modify it and emit it back out as text.

Luckily, I eventually stumbled upon Himalaya, which is perfect for the job. It converts the HTML document to a JSON object - so this document:

<div class='post post-featured'>
  <p>Himalaya parsed me...</p>
  <!-- ...and I liked it. -->
</div>

becomes this JSON:

[{
  type: 'element',
  tagName: 'div',
  attributes: [{
    key: 'class',
    value: 'post post-featured'
  }],
  children: [{
    type: 'element',
    tagName: 'p',
    attributes: [],
    children: [{
      type: 'text',
      content: 'Himalaya parsed me...'
    }]
  }, {
    type: 'comment',
    content: ' ...and I liked it. '
  }]
}]

(Example taken from the Github page.)

As well as parse there’s also a stringify method, which converts the JSON back to HTML. Perfect.

This is a bigger problem than I realised when I started on it. The best thing I’ve found for big problems is to break them down, like below.

function addSnippet(request, h) {
  // Is it HTML? If not, we can't work with it.
  // Can we parse it?
  // If we can parse it, does it have a HTML tag as it should?
  // And is there a 'head' element we can attach our script to?
  // Did someone already add a livereload script to it?
  // Add the script to the 'head' element
  // Convert back to HTML and return it
}

That looks a manageable breakdown, and all the individual pieces don’t seem as challenging. Let’s work through them.

Can we work with the response?

We’ll go with the simple approach here - we’ll assume the MIME type is correct and check for text/html.

function isHtml(response) {
  return response.contentType?.indexOf("text/html") >= 0;
}

Can we parse it, and does it have the right nodes?

We end up searching nodes quite a bit, so let’s have a convenience function for that.

function findNode(nodes, name) {
  return nodes.find(node => node.tagName === name);
}

With that, our parsing/checking code can be written as below. We try to parse it; if we can parse it we look for the html element. If that exists, we look for a child of that which is a head element.

Note - we’re using h.continue here. That indicates to Hapi that we don’t have anything to say about this response, move it on to the next hook.

  const doc = parse(response.source);
  if (!doc || doc.length == 0) {
    logger("Couldn't parse", response.source);
    return h.continue;
  }

  const htmlNode = findNode(doc, "html")
  if (!htmlNode) {
    logger("Couldn't find HTML tag", response.source);
    return h.continue;
  }

  const headNode = findNode(htmlNode.children, "head");
  if (!headNode) {
    logger("Couldn't find HEAD tag", response.source);
    return h.continue;
  }

Is there a script already?

Technically, the script could be anywhere in the page. We don’t want to just search the whole body of the webpage - there could be a large number of nodes in that. So we’ll simplify our problem and only check for the script in the head, which is where it’s most likely to be anyway.

To do this, we need to:

find all the children of the head node,
which are script nodes,
with a src attribute
and check the src value for livereload.js

Step 1 is simple - head.children. The remaining 3 steps can be rewritten as - for each node, if it’s a script node, check the src attribute and src value. If the src value contains livereload.js, then the script was already added.

function reloadExists(nodes) {
  let found = false;

  nodes.forEach(node => {
    if ((node.type === 'element') && (node.tagName === 'script')) {
      const reloadAttribs =
        node.attributes
            .filter(node => node.key === 'src')
            .filter(attr => attr.value?.indexOf('/livereload.js') >= 0);
      found = reloadAttribs.length > 0;
    }
  })

  return found;
}

Add the script

We need to add a script node which loads the client script. We’ll make it async so it doesn’t slow the rest of the site loading.

Since we’re working with JSON, this is now relatively easy - we push a new node onto the existing children of head.

  headNode.children.push({
    type: 'element',
    tagName: 'script',
    attributes: [{ key: 'src', value: src }, { key: "async", value: null }],
    children: []
  })

Convert back to HTML

As the last step, we return the new HTML from the hook so that Hapi will use it as the response. Himalaya makes this easy:

  return stringify(doc);

Put it all together

We can now flesh out our addSnippet skeleton from above, replacing the comments with actual code.

function addSnippet(request, h) {
  logger("Processing response for", request.info.id);

  // Less typing.
  const response = request.response;

  if (!isHtml(response)) {
    logger("Response isn't HTML, skip it.");
    return h.continue;
  }

  const doc = parse(response.source);
  if (!doc || doc.length == 0) {
    logger("Couldn't parse", response.source);
    return h.continue;
  }

  const htmlNode = findNode(doc, "html")
  if (!htmlNode) {
    logger("Couldn't find HTML tag", response.source);
    return h.continue;
  }

  const headNode = findNode(htmlNode.children, "head");
  if (!headNode) {
    logger("Couldn't find HEAD tag", response.source);
    return h.continue;
  }

  if (reloadExists(headNode.children)) {
    logger("Snippet already exists, skip.");
    return h.continue;
  }

  headNode.children.push({
    type: 'element',
    tagName: 'script',
    attributes: [{ key: 'src', value: src }, { key: "async", value: null }, { key: "defer", value: null }],
    children: []
  })

  logger("Processing done for", request.info.id);
  return stringify(doc);
}

Trying it out

To test it out, we need a route which actually serves an HTML file - nothing fancy, so we can easily see if the code has been added.

<!DOCTYPE html>
<html lang="en">
<head>
    <title>Home Page</title>
</head>
<body>
<header><h1>Hello from Hapi!</h1></header>
<main>
    <p>This is a paragraph</p>
    <p>This is still also a paragraph.</p></main>
</body>
</html>

index.html

We’ll change our / route to serve that. Assuming the html variable contains the HTML page:

  server.route({
    path: "/",
    method: "GET",
    handler: (request, h) => h.response(html)
  })

Try it out:

<!doctype html>
<html lang='en'>
<head>
    <title>Home Page</title>
<script src='//localhost:35729/livereload.js?snipver=1' async defer></script></head>
<body>
<header><h1>Hello from Hapi!</h1></header>
<main>
    <p>This is a paragraph</p>
    <p>This is still also a paragraph.</p></main>
</body>
</html>

And there it is! You can see the HTML doesn’t look totally the same as before - the case of doctype has been changed, for example - but it has the same semantic meaning.

Conclusions

The good…

I haven’t heavily tested it but this approach seems to work for serving plain HTML files.

It wasn’t hard to implement - Hapi makes available everything we need for integrating ourselves into the request/response lifecycle, and the Himalaya library provided exactly what we needed for the HTML processing (it would have been a lot harder without that).

… and the ‘oops’

However… when I went to integrate it with the project I’m actually working on, I found it doesn’t work with Vision. The source element isn’t a rendered page, it’s a data structure, presumably one that’s meaningful to Vision.

At this point I did what should have occurred to me before - I put the JavaScript in the header partial being rendered by Vision. Once I’d done that, I had livereload working without the need for the plugin.

So from the “getting livereload working” perspective, this was actually a fairly pointless job. It was still a useful experience, though, and I learned from it. So it’s not wasted as such, just a bit of a “doh!” moment.

DEV Community