DEV Community: Paul Walker

Making a subset of Font Awesome

Paul Walker — Tue, 17 May 2022 21:07:00 +0000

Font Awesome is, well, awesome - but there's no getting away from the fact that if you only want a few of the icons then it's not a lightweight package.

Looking around for something to solve this with I found Logan Graham has written a nice JavaScript package to solve this, called - appropriately enough - fontawesome-subset.

That will do the work of extracting the icons you want and repackaging them back into font files, but you still need the CSS files to use them.

It only works with Font Awesome 5.x - if you want to use 6 then it looks like you need to use the Font Awesome Pro kits, or pull up the issue on Github and have a crack at it. :-)

After a couple of times trying to deploy this on a website and forgetting to copy or run something I decided that it was probably time to invest in setting up a task runner.

Which runner?

I’m sure there are more, but the ones I’ve seen are Gulp and Grunt. I needed some way to decide which to go for, and the GitHub repos actually made that fairly simple.

Gulp

Gulp hasn’t been updated since mid-2021, the CI test is failing, and it doesn’t support Node >= 14.

Grunt

Grunt was last updated 3 days ago (at the time of writing), and has passing CI tests.

So, Grunt it is.

Demo webpage

Just to note this post isn’t going to walk you through actually using FA - that’s not the focus here. So if the classes below don’t make sense to you, head over to the FA docs.

Here’s the page:

<html>
  <head>
  </head>
  <body>
    <form>
      <label for="email">
        <i class="fas fa-envelope"></i>
        Email
      </label>
      <input type="email" id="email"/>
      <label for="password">
        <i class="fas fa-lock"></i>
        Password
      </label>
      <input type="password" id="password"/>
      <button type="submit">Click me</button>
    </form>
  </body>
</html>

Right now we don’t see any icons - we haven’t done the subsetting yet, and we don’t have any CSS files included.

Before we load any CSS, no icons to be seen

We’ll use the livereloadx package for serving the content, since we don’t have to write any code to use it. We’ll add the command to the scripts section in package.json, to make it easy to run.

"scripts": {
    "serve": "node ./node_modules/.bin/livereloadx -s ."
}

How to subset the files

After all, if we can't work this out there's not much point in doing the rest. Luckily, the usage section of the README makes it fairly easy to work out.

Import the fontawesomeSubset function:

const { fontawesomeSubset } = require("fontawesome-subset");

Then we call it with two arguments. The first is a name, or array of names, for the icons that we want (without the fa- prefix). The second is where it should write the font files it's going to produce.

fontawesomeSubset([
  'envelope',
  'lock',
],
'static/webfonts');

And indeed, if we run that we can see it works, or at least produces some output. The output files are a fraction of the size of the source files, which is encouraging.

$ node fa_subset.js
$ fd . -t f static/
static/webfonts/fa-solid-900.eot
static/webfonts/fa-solid-900.svg
static/webfonts/fa-solid-900.ttf
static/webfonts/fa-solid-900.woff
static/webfonts/fa-solid-900.woff2
$ du -hs static/webfonts/
 20K    static/webfonts/

Getting Grunt setup

First let's get Grunt to the point where it runs successfully, even if it doesn’t do anything.

Grunt config can live in various places, but I’m choosing to put it in Gruntfile.js to keep it separated. The simplest config is as below:

module.exports = function (grunt) {
  grunt.initConfig({
    pkg: grunt.file.readJSON("package.json")
  })

  // We'll load any plugins here.

  // 'default' is run if you run Grunt without arguments.
  grunt.registerTask('default', []);
}

Note the ‘default’ task being registered even though it doesn’t do anything. As well as being a good placeholder, it also means if you just run grunt with no arguments it won’t produce an error. We’ll come back to that later.

And with that, it's alive - though admittedly it doesn’t say much.

$ grunt

Done.

Copying the files

To use the web fonts, we’ll use the CSS files from the Font Awesome package. They should automatically load the webfont files that we’ve extracted. Just a reminder - this only works with Font Awesome 5.x, not 6.x.

There’s a grunt-contrib-copy package which teaches Grunt how to copy files. The first thing after installing it is to load it in the Grunt setup function.

grunt.loadNpmTasks('grunt-contrib-copy');

From the Grunt docs we can see that we need:

the expand option, to enable the other options
the flatten option, to flatten the destination results to a single level.
src, to list the source files
dest, to tell Grunt where to put the copied files.

From that we end up with a config object like this.

grunt.initConfig({
  pkg: grunt.file.readJSON("package.json"),
  copy: {
    main: {
      files: [
        {
          expand: true,
          flatten: true,
          src: [
            'node_modules/@fortawesome/fontawesome-free/css/fontawesome.min.css',
            'node_modules/@fortawesome/fontawesome-free/css/solid.min.css'
          ],
          dest: 'static/css/'
        },
      ]
    }
  }
});

We can run that, and it says it's copied two files. Let's just check it's the right two files, and they're where we expect.

$ grunt copy
Running "copy:main" (copy) task
Copied 2 files

Done.
$ fd . -t f static
static/css/fontawesome.min.css
static/css/solid.min.css

Yep, that looks good.

Running the subset task

Now let's get Grunt to actually do the subsetting. For this we'll use the registerTask() function, which provides a general purpose “run this function” task. That has the following prototype:

grunt.registerTask(name, description, function)

So, let’s try just plugging in the function we wrote earlier.

grunt.registerTask('fasubset', 'Subset the FA icons.', function() {
  fontawesomeSubset([
      'envelope',
      'lock'
    ],
    'static/webfonts');
});

And … it works. That was almost too easy!

$ rm -rf static
$ grunt fasubset
Running "fasubset" task

Done.
$ fd . -t f static
static/webfonts/fa-solid-900.eot
static/webfonts/fa-solid-900.svg
static/webfonts/fa-solid-900.ttf
static/webfonts/fa-solid-900.woff
static/webfonts/fa-solid-900.woff2

Running tasks by default

For the final bit of Grunt config, let's change the default task to run the copy and subset, so we can just run grunt in future without any arguments.

grunt.registerTask('default', ['fasubset', 'copy']);

Add the CSS files to the webpage

Now we need to start using the CSS files, otherwise nothing will happen.

<html>
  <head>
    <link rel="stylesheet" href="/static/css/fontawesome.min.css">
    <link rel="stylesheet" href="/static/css/solid.min.css">
  </head>
[...]

Check the results

Finally, let's check the webpage again - we should see that our icons have popped into existence.

After loading CSS, with icons visible

That's all there is to it! I have to admit I was expecting more difficulty when I started, but actually Grunt is designed nicely enough to make sense, and fontawesome-subset Just Works. I like packages like that.

How about you? Got any useful packages you'd like to share in the comments?

mbsync, SASL and Gmail

Paul Walker — Thu, 17 Mar 2022 16:30:06 +0000

Slightly more esoteric than usual; I'm posting this for my benefit as much as anyone else's!

I was just trying to set up mbsync to work with Gmail. I'd got most of the config copied from another provider, but kept getting an error about an invalid parameter:

Error performing SASL authentication step: SASL(-7): invalid parameter supplied: Parameter Error in /System/Volumes/Data/SWE/macOS/BuildRoots/533514bb11/Library/Caches/com.apple.xbs/Sources/passwordserver_saslplugins/passwordserver_saslplugins-194/plain_clienttoken.c near line 195

I eventually figured out (from reading Jakub Kadlčík's clear post) that you need to tell mbsync which authentication method it should use - for some reason it doesn't correctly autodetect. I added in the AuthMechs line:

AuthMechs LOGIN

Now it's able to log in fine.

One Small Step...

Paul Walker — Sat, 26 Feb 2022 16:24:03 +0000

If there is anyone following me, you might have noticed that I haven’t really posted anything lately.

The reason is simple — there’s been so much going on at home that I haven’t had any brain space to write. There have been days when I could manage to watch a music video and that was about it.

And yet things like emptying the rubbish, cleaning the bathroom, they all still need to happen.

The best way I’ve found for this is to kind of sneak up on things. Rather than tell myself I’m going to clean the bathroom, I’m just getting this brush and cleaning these tiles. Rather than clearing all the stuff out of the kitchen, I’m just bringing in the bags that I’ll put the rubbish in, that’s all.

Somehow, when you do it like that, things seem to still get done. The bags are in, so you might as well put some of the stuff in them. These tiles are done, might as well do those ones.

The craziness isn’t showing any signs of going anyway just yet. But I know I still want to keep writing. So I’m going to keep doing it the same way as everything else at the moment — one step at a time.

This post? It's one small step.

Photo by Hayley Seibel on Unsplash

My Apple purchase regrets

Paul Walker — Thu, 16 Dec 2021 23:02:03 +0000

I read an interesting article by Mark Ellis the other day which got me thinking about my biggest Apple purchasing regret.

To be clear up-front, I’m very happy with most of their devices, and quite snug in their ecosystem. While they do have their faults I’m happier to have them making my phone than Google or any of the interchangeable Android OEMs.

Runner-up

The runner-up has to be my previous laptop, the MacBook Pro 2012.

To be clear, the MacBook itself didn’t really do anything wrong. What really let it down was the keyboard, and the freaking Touch Bar.

The keyboard had no travel to speak of. It was horrible to type on. I really should have trusted my instincts when I was in the shop, rather than thinking I’d get used to it. It was never good for typing - and as someone who writes code as well as starting to dabble in writing text, that was a major drawback.

The Touch Bar also was not a good thing. It looked good, all fancy and stuff, but having things move around on you destroys muscle memory. It’s much better now I can rely on shift-F6 being in the same place the whole time, rather than having to think “what’s the Touch Bar showing at this moment? Am I going to refactor my code, or launch iTunes?”, and it's much more productive.

For reference - I’m currently happily typing away on an M1 MacBook Air - light, responsive, great battery life, and the keyboard is a pleasure to use.

Winner

However — the winner has to be the Apple Watch.

I got hooked on the idea of it, and it looked all nice and sleek when I got it. It was only after a while that I realised that the UI was clunky, and it was really useful for only a couple of things:

subtly letting me know of notifications
measuring your heartbeat, and maybe your steps.

A Fitbit can do the second two much, much more cheaply. Setting my phone to vibrate on the notifications I cared about approximated the first one.

I’ve also never really liked big watches on my wrist - I thought I’d get used to the Apple Watch, but I never did. (It has that in common with the MacBook - I really must learn to listen to my instincts more.)

It’s spent pretty much the whole of lockdown on the charger, and honestly, I haven’t missed it.

What has taken its place is the Pulse HR from Withings. Great battery life, light, and does what I need much more cheaply.

How about you folks out there? Any tales of “buyer’s remorse” to share, be it Apple gadgets or otherwise?

Photo by Danie Franco on Unsplash.

Things I wish I’d known sooner about… JavaScript modules

Paul Walker — Wed, 10 Nov 2021 22:57:58 +0000

I have to confess it took me a while to get my head around JavaScript modules. This is at least partly because there are two different popular module systems, CommonJS and ECMAScript 2015 (usually shortened to ES2015).

The ECMAScript 2015 standard is newer than CommonJS. Personally I find it more straightforward, and I use it in preference where I can. Quite a lot of the modules on NPM still use the old style, however.

One important note - browsers only support ES2015, not CommonJS.

CommonJS

CommonJS was started in 2009 by a member of the Mozilla team, as an attempt to unify JavaScript functionality. CommonJS covers more than just modules - check the website.

Exports

In CommonJS modules, symbols are exported in an object (understandably named exports). Each property of the object is an exported symbol.

The usual object assignment rules apply. This means you can easily export a symbol with a different name than it has internal to the file.

function sayHello(name) {
  console.log("Hello", name);
}

module.exports = {
  sayHello,
  anotherSayHello: sayHello
};

Require

You can then require either the whole module or some symbols from that module.

const testModule = require("./module_name");
const { sayHello } = require("./module_name");

testModule.sayHello("world");
sayHello("second world");

NB : importing named symbols from a CJS-style module was only introduced in Node 14, and doesn't work perfectly - it relies on some heuristics within Node.

ES2015

Export

Exporting is easier in ES2015; an item (function, constant, variable etc) can be exported simply by adding the export keyword.

export function sayHello(name) {
  console.log("Hello", name);
}

export const myName = "Rumpelstiltskin";

Alternatively, you can export the items en-masse at the end of the file.

export { sayHello, myName };

Import

Like many other languages - Java, Python - you then import the parts of the module that you want to use.

import { sayHello } from "./moduleName";

sayHello("world");

Or if you want to keep them namespaced:

import * as moduleStuff from "./moduleName";

moduleStuff.sayHello(moduleStuff.myName);

Default exports

In most of the examples above, we've imported a specific named symbol from module:

import { sayHello } from "./module_name";

A module can also have a default export, which is what you then get if you don't specify what you want to import from the module.

CommonJS

CJS modules actually only export one single item - in that sense, they only have a default export. It's why the items we want to export need to be properties of an object.

ES2015

With this system you do get more flexibility. More than one thing can be exported, and you're able to mark which one you want to be the default.

export default const theAnswer = 42;

User beware

This isn't always a good thing; you're never totally sure what you're importing, and it can be imported with a completely different (and misleading) name to the original.

For example, take the trivial module basicMaths:

export default sum(x, y) { return x + y; }

That default export can be imported as anything.

import divide from "./basicMaths";

// This will not do what you expect.
divide(2, 4);

The advantage of named exports is that you import exactly what you want, no surprises.

import { divide } from "./basicMaths";

// divide is empty, so this crashes (or doesn't build, depending on how
// good your tools are). Either way it's better than using the wrong 
// function.
divide(2, 4);

JavaScript pattern - the “singleton file”

Paul Walker — Sun, 10 Oct 2021 21:26:40 +0000

The other day I came across the solution to a number of problems I’d had in the past.

I imagine it’s probably obvious to anyone who’s been doing JavaScript for a while, but since it took a while to dawn on me I thought I’d try and save someone else the wait, even if it’s only one. And it’ll provide the rest of you with a good laugh too.

I don’t know what this pattern is officially called, so I’ve dubbed it the ‘singleton file’ - you can import it everywhere, but it only creates a single object which everything ends up using.

The problem

The problem I was having was with services like logging, which need to be available before most of the modules are started. At first I tried doing this with an initLogging function, similar to most services. However, when the system was starting up and all the modules were being imported some of the module code was being run before the logging setup was done.

The solution

The solution isn’t really that cunning, but it works very well (for me at least): you make the actual logger ‘object’ an export from the module.

Defining the logging module

I’m using the Winston logging system in most of my things. I like the look of the Serilog/structured logging stuff but it’s been too much hassle to get working for now. The logging module I use currently looks a bit like the code below. Warning - it’s a bit long.

import { createLogger, transports, format } from 'winston';

// Allow the logging level to be controlled at start time
let level = process.env.LOG_LEVEL;
if (process.env.NODE_ENV !== "test") {
  console.info(`Logging at ${level} level.`)
}

// This is the format for lines to be printed to console. The format varies
// depending on the environment.
const consoleLineFormat = format.printf(({ level, message, label, timestamp }) => {
  if (process.env.NODE_ENV === "production") {
    return `[${label}] ${level}: ${message}`;
  } else {
    return `${timestamp} [${label}] ${level}: ${message}`;
  }
});

let consoleFormat;
if (process.env.NODE_ENV === "production") {
  // Heroku captures info like timestamp anyway, don't duplicate it.
  consoleFormat = format.combine(
    format.splat(),
    consoleLineFormat
  );
} else {
  // Put more information in the dev logs
  consoleFormat = format.combine(
    format.splat(),
    format.colorize(),
    format.timestamp(),
    consoleLineFormat
  );
}

// For file logs, use JSON
const fileFormat = format.combine(
  format.splat(),
  format.json(),
  format.timestamp()
)

// Create the base logger; everything else comes from this.
// We always log to console; exceptions are always logged, regardless
// of the level that's set.
const base_logger = createLogger({
                              level: 'info',
                              transports: [
                                new transports.Console({ level: level, format: consoleFormat })
                              ],
                              exceptionHandlers: [
                                new transports.Console({ format: consoleFormat })
                              ]
                            });

// For development, also log to files. No point in that for Heroku.
if (process.env.NODE_ENV === "development") {
  base_logger
    .add(new transports.File({ filename: 'combined.log', level: 'debug', format: fileFormat }))
    .add(new transports.File({ filename: 'errors.log', level: 'error', format: fileFormat }));
}

// 'parentLogger' is exported to the other modules. They use the 'child'
// function to create a local logger, tagged with the module name.
export const parentLogger = Object();
type childOptions = {
  module: string;
}
parentLogger.child = function(opts: childOptions) {
  return base_logger.child({ label: opts.module });
}

Apologies for the length of that - I did try to take bits out, but all of it’s there for a reason and I thought it might be useful to people.

Using the logging module

Luckily, actually using it is simple.

import { parentLogger } from "./logger";
const logger = parentLogger.child({ module: "events" })

That’s it. Now the local code simply uses logger.info, or whatever level you want. It’s logger in every module, so I don’t have to remember any other name.

Why it works

If I’m honest, I haven’t investigated this in detail. I imagine it’s because Node doesn’t let modules start running until the modules they’re importing have completed the initial run-through - otherwise it would be a nightmare to use dependencies. If anyone knows for sure, or knows different, please do let me know.

That’s it. Hope that helps someone!

Oh - and in case you're wondering, the image is totally unrelated to the story. I couldn't find a suitable post image, so I went with "cute puppy" instead.

Live reload in Hapi

Paul Walker — Sat, 17 Jul 2021 16:00:47 +0000

Something I’ve been meaning to address for a while is the lack of a livereload plugin for Hapi. Template changes are visible on manual reload if you turn off caching, but I’ve become used to automatic reloading when developing with Vue.

livereload architecture

The full livereload protocol is available for everyone to view. In brief, it requires two parts.

Server

The livereload server accepts HTTP and WebSocket connections (on the same port). It’s responsible for serving the livereload client code, and for monitoring the filesystem for changes. It also listens for messages from the client indicating URL changes.

When changes happen, it sends a message to the client (over WebSocket) indicating which file has changed so it can be reloaded if needed.

Client

The client runs in the browser - it makes a connection to the server and waits for a message. When it receives a message indicating a reload is required, it does it.

(This is a rather abbreviated explanation…)

I had no intention of actually re-implementing the client - I just wanted to automatically insert the fragment which causes the client code to be loaded and start listening.

Plugin structure

A Hapi plugin seemed to be the best place to start. I knew from previous work that the Hapi server has “extension points” in the request/response lifecycle, which can be hooked into and used to monitor or modify the request or response.

The basic structure of a Hapi plugin is simple - it’s an object which exports a structure named plugin containing a function named register() (and a bit of metadata). The register function receives a server object and an (optional) options object. It does what it needs to do, and then returns or throws an exception.

A very basic plugin is below. It exports the structure Hapi is looking for in a plugin. The register function creates a route which says hello to the user, or the whole world if the user doesn’t specify who to greet.

'use strict';

async function register(server, options = {}) {
  server.route({
    path: "/hello",
    method: "GET",
    handler: function () {
      const name = options.name || "world";
      return `Hello ${name}\n`;
    }
  });
}

module.exports = {
  name: 'hapi-reload',
  version: '0.1.0',
  register: register
};

There are multiple ways of declaring your plugin - the Hapi docs cover it all well. For now we’ll keep it simple, and put our plugin code in a file named plugin.js.

We also need to get a server up and running, so we have something to plug in to.

const Hapi = require("@hapi/hapi");
const hapiLivereload = require("./plugin");

async function run() {
  const serverOpts = {
    port: process.env.PORT || 4000,
    host: '0.0.0.0'
  };
  const server = Hapi.server(serverOpts);

  await server.register({
    plugin: hapiLivereload
  });
  return server.start();
}

run()
  .then(() => console.log("Server up"))
  .catch(err => console.error("Couldn't run", err));

And - it works.

$ node dev/server.js
Server up


$ curl http://localhost:4000/hello
Hello world

We can add the name option to customise the response.

  await server.register({
    plugin: hapiLivereload,
    options: {
      name: "Paul"
    }
  });

And now it knows me!

$ curl http://localhost:4000/hello
Hello Paul

Which hook to use?

So that’s a basic plugin, but it doesn’t do anything particularly useful - you can probably think of easier ways to say “hello world”.

Let’s start looking at the request lifecycle again. Looking through the list of steps the onPreResponse looks good - the response has been generated but not yet transmitted, and better still:

the response contained in request.response may be modified (but not assigned a new value). To return a different response type (for example, replace an error with an HTML response), return a new response value.

Let’s try a new approach - rather than a specific route to greet the user, let’s just make all the responses do that. Much more friendly.

async function register(server, options = {}) {
  server.ext({
    type: 'onPreResponse',
    method: function(request, h) {
      const name = options.name || "world";
      console.log(`Replacing "${request.response.source}"`);
      return `Hello ${name}\n`;
    }
  });
}

The console.log shows us what we’re replacing (useful for debugging); and we just return the new response text we want, which the docs say should work.

To test it, we’re going to need a route which doesn’t return Hello world. We can add that to run() in our server code.

  server.route({
    path: "/",
    method: "GET",
    handler: () => "I should be an index page\n"
  })

And if we try it?

Server up
Replacing "I should be an index page
"


$ curl http://localhost:4000/
Hello Paul

Excellent!

livereload register

The register function for the livereload plugin is basically the same. The differences are specific to the plugin, working out the location of the client code.

const logger = require("util").debuglog("hapi-livereload");

let host, port, src;

async function register(server, options = {}) {
  host = options.host || "localhost";
  port = options.port || 35729;
  src = options.src || '//' + host + ':' + port + '/livereload.js?snipver=1';

  server.ext({
    type: 'onPreResponse',
    method: addSnippet
  });
}

The bit we haven’t covered yet is that addSnippet function, which is where things got a little more involved - actually modifying the webpage being returned.

We also add a logger function using Node’s standard util.debuglog functionality, so we can report back to the user if they want us to.

Modifying the HTML

I spent some time looking for an HTML parser. It was complicated by the fact that the plugin not only needs to parse the outgoing HTML page, but we also have to be able to modify it and emit it back out as text.

Luckily, I eventually stumbled upon Himalaya, which is perfect for the job. It converts the HTML document to a JSON object - so this document:

<div class='post post-featured'>
  <p>Himalaya parsed me...</p>
  <!-- ...and I liked it. -->
</div>

becomes this JSON:

[{
  type: 'element',
  tagName: 'div',
  attributes: [{
    key: 'class',
    value: 'post post-featured'
  }],
  children: [{
    type: 'element',
    tagName: 'p',
    attributes: [],
    children: [{
      type: 'text',
      content: 'Himalaya parsed me...'
    }]
  }, {
    type: 'comment',
    content: ' ...and I liked it. '
  }]
}]

(Example taken from the Github page.)

As well as parse there’s also a stringify method, which converts the JSON back to HTML. Perfect.

This is a bigger problem than I realised when I started on it. The best thing I’ve found for big problems is to break them down, like below.

function addSnippet(request, h) {
  // Is it HTML? If not, we can't work with it.
  // Can we parse it?
  // If we can parse it, does it have a HTML tag as it should?
  // And is there a 'head' element we can attach our script to?
  // Did someone already add a livereload script to it?
  // Add the script to the 'head' element
  // Convert back to HTML and return it
}

That looks a manageable breakdown, and all the individual pieces don’t seem as challenging. Let’s work through them.

Can we work with the response?

We’ll go with the simple approach here - we’ll assume the MIME type is correct and check for text/html.

function isHtml(response) {
  return response.contentType?.indexOf("text/html") >= 0;
}

Can we parse it, and does it have the right nodes?

We end up searching nodes quite a bit, so let’s have a convenience function for that.

function findNode(nodes, name) {
  return nodes.find(node => node.tagName === name);
}

With that, our parsing/checking code can be written as below. We try to parse it; if we can parse it we look for the html element. If that exists, we look for a child of that which is a head element.

Note - we’re using h.continue here. That indicates to Hapi that we don’t have anything to say about this response, move it on to the next hook.

  const doc = parse(response.source);
  if (!doc || doc.length == 0) {
    logger("Couldn't parse", response.source);
    return h.continue;
  }

  const htmlNode = findNode(doc, "html")
  if (!htmlNode) {
    logger("Couldn't find HTML tag", response.source);
    return h.continue;
  }

  const headNode = findNode(htmlNode.children, "head");
  if (!headNode) {
    logger("Couldn't find HEAD tag", response.source);
    return h.continue;
  }

Is there a script already?

Technically, the script could be anywhere in the page. We don’t want to just search the whole body of the webpage - there could be a large number of nodes in that. So we’ll simplify our problem and only check for the script in the head, which is where it’s most likely to be anyway.

To do this, we need to:

find all the children of the head node,
which are script nodes,
with a src attribute
and check the src value for livereload.js

Step 1 is simple - head.children. The remaining 3 steps can be rewritten as - for each node, if it’s a script node, check the src attribute and src value. If the src value contains livereload.js, then the script was already added.

function reloadExists(nodes) {
  let found = false;

  nodes.forEach(node => {
    if ((node.type === 'element') && (node.tagName === 'script')) {
      const reloadAttribs =
        node.attributes
            .filter(node => node.key === 'src')
            .filter(attr => attr.value?.indexOf('/livereload.js') >= 0);
      found = reloadAttribs.length > 0;
    }
  })

  return found;
}

Add the script

We need to add a script node which loads the client script. We’ll make it async so it doesn’t slow the rest of the site loading.

Since we’re working with JSON, this is now relatively easy - we push a new node onto the existing children of head.

  headNode.children.push({
    type: 'element',
    tagName: 'script',
    attributes: [{ key: 'src', value: src }, { key: "async", value: null }],
    children: []
  })

Convert back to HTML

As the last step, we return the new HTML from the hook so that Hapi will use it as the response. Himalaya makes this easy:

  return stringify(doc);

Put it all together

We can now flesh out our addSnippet skeleton from above, replacing the comments with actual code.

function addSnippet(request, h) {
  logger("Processing response for", request.info.id);

  // Less typing.
  const response = request.response;

  if (!isHtml(response)) {
    logger("Response isn't HTML, skip it.");
    return h.continue;
  }

  const doc = parse(response.source);
  if (!doc || doc.length == 0) {
    logger("Couldn't parse", response.source);
    return h.continue;
  }

  const htmlNode = findNode(doc, "html")
  if (!htmlNode) {
    logger("Couldn't find HTML tag", response.source);
    return h.continue;
  }

  const headNode = findNode(htmlNode.children, "head");
  if (!headNode) {
    logger("Couldn't find HEAD tag", response.source);
    return h.continue;
  }

  if (reloadExists(headNode.children)) {
    logger("Snippet already exists, skip.");
    return h.continue;
  }

  headNode.children.push({
    type: 'element',
    tagName: 'script',
    attributes: [{ key: 'src', value: src }, { key: "async", value: null }, { key: "defer", value: null }],
    children: []
  })

  logger("Processing done for", request.info.id);
  return stringify(doc);
}

Trying it out

To test it out, we need a route which actually serves an HTML file - nothing fancy, so we can easily see if the code has been added.

<!DOCTYPE html>
<html lang="en">
<head>
    <title>Home Page</title>
</head>
<body>
<header><h1>Hello from Hapi!</h1></header>
<main>
    <p>This is a paragraph</p>
    <p>This is still also a paragraph.</p></main>
</body>
</html>

index.html

We’ll change our / route to serve that. Assuming the html variable contains the HTML page:

  server.route({
    path: "/",
    method: "GET",
    handler: (request, h) => h.response(html)
  })

Try it out:

<!doctype html>
<html lang='en'>
<head>
    <title>Home Page</title>
<script src='//localhost:35729/livereload.js?snipver=1' async defer></script></head>
<body>
<header><h1>Hello from Hapi!</h1></header>
<main>
    <p>This is a paragraph</p>
    <p>This is still also a paragraph.</p></main>
</body>
</html>

And there it is! You can see the HTML doesn’t look totally the same as before - the case of doctype has been changed, for example - but it has the same semantic meaning.

Conclusions

The good…

I haven’t heavily tested it but this approach seems to work for serving plain HTML files.

It wasn’t hard to implement - Hapi makes available everything we need for integrating ourselves into the request/response lifecycle, and the Himalaya library provided exactly what we needed for the HTML processing (it would have been a lot harder without that).

… and the ‘oops’

However… when I went to integrate it with the project I’m actually working on, I found it doesn’t work with Vision. The source element isn’t a rendered page, it’s a data structure, presumably one that’s meaningful to Vision.

At this point I did what should have occurred to me before - I put the JavaScript in the header partial being rendered by Vision. Once I’d done that, I had livereload working without the need for the plugin.

So from the “getting livereload working” perspective, this was actually a fairly pointless job. It was still a useful experience, though, and I learned from it. So it’s not wasted as such, just a bit of a “doh!” moment.

My 11 favourite Vim plugins

Paul Walker — Thu, 03 Jun 2021 21:11:49 +0000

While I’m spending most of my JavaScript writing time in WebStorm these days, I still use Vim a lot. Over time I’ve found some great plugins, extending Vim to cope with an amazing amount of stuff.

I should note that these are all essentially general purpose plugins. Some of the language-specific ones are great to use, but unless you’re working with that particular language they’re not going to be that useful.

Any screenshots are taken from the plugin website, rather than grabbed by me, unless explicitly noted.

Plug

This is where it all starts. It’s very simple to use - I don’t think I’ve ever had to change a setting.

Once you’ve added the plugin section to your .vimrc adding a new plugin is simple, especially if it’s on Github:

call plug#begin("~/.vim/plugged")
    Plug 'foo/bar'
call plug#end()

(It will handle arbitrary Git servers as well.)

Plug takes care of downloading and installing the plugin. If a plugin needs a few options then that’s easily taken care of too.

Plug 'junegunn/fzf', { 'dir': '~/.fzf', 'do': './install --all' }

Once you’ve got it set up just :PlugInstall to install plugins, and :PlugUpgrade to upgrade. Simples, just the way I like it.

Get Plug here.

General functionality.

These plugins cover Vim “as a whole” - they’ll improve your whole Vim experience.

startify

Startify has a simple goal - on startup it presents a list of most-recently-used files and sessions.

I started using this fairly recently, and it’s surprisingly useful - I’d probably use it for the SSave and SLoad functions alone (storing Vim sessions in a dedicated session directory). As an added bonus it even recognises if there’s an existing Session.vim file in the directory you’ve launched it from and offers that as an extra option.

Get Startify here.

NERDTree

NERDTree is a better file sidebar. It makes it easy to delete/add/rename files and directories, change the Vim working directory (useful for launching tools such as grep), and easy to navigate around the file structure. If you rename a file or directory and you’ve got buffers open with the old name it offers to reopen the buffers with the new name, which is a very nice touch.

It offers integrated help - I’ve not had to resort to reading the fine manual. The only niggle I have is that the help is accessed via ?, which is entirely reasonable but breaks backward search in the tree.

(It is quite possible the manual gives a way to change that, of course. Maybe I should read it…)

Get NerdTree here.

Obsession

One of Tim Pope’s plugins, which should be all you need to know about the quality of it. Once you start it in an editing session Obsession will automatically save any window/buffer/layout changes to the session file.

When used together with Startify it makes for a very smooth flow. Save the session file once and turn on Obsession - after that you can load it via the Startify menu buffer and everything’s set up right the way you left it last time you exited.

(And no more problems forgetting to save the session before that automatic :qa…)

Get Obsession here.

markdown-preview.nvim

To be honest - this is way better than I expected a preview in Vim to be. Once you’ve started it opens a browser window, then automatically reloads the contents as you edit the document.

Get markdown-preview.nvim.

Coding

These plugins help with coding, rather than appearance or general text editing.

CoC

Integrates support for Language Servers, as used by VS Code, IntelliJ, Sublime Text and co. Language servers are what let the editors quickly and accurately auto-complete, suggesting type-compatible completions.

Basically - if you write code with Vim, you need this plugin.

Get CoC.

Vimagit

This is a port of (or ‘rewrite inspired by’) the Emacs magit extension. While it only does a subset of magit - it doesn’t do remote operations like push/pull/fetch, for example - it makes adding and committing to the repo really easy. I particularly like the fact you can stage whole files or individual hunks.

Get Vimagit here.

ALE

ALE is the Asynchronous Lint Engine. It provides background linting, syntax checking and semantic errors in recent versions of Vim, so the errors are flagged while you edit. This lets you fix them while the code is still (really!) fresh in your mind, rather than waiting for the next time you run lint or the static checker.

Get ALE here.

Appearance

Finally, we’re on to the plugins which change how Vim looks.

Airline

Airline adds a status line to the bottom of each Vim window. Without any tweaking it’ll display:

mode (normal/visual, that kind of thing)
VCS info
filename, filetype
file encoding

It’s very configurable - you can probably get it to display just about anything that you’re interested in. It also plays well with others - for example, ALE above can be integrated into Airline to flag up warnings/errors in the status line.

Give it a try - just don’t blame me if you then spend the next day tweaking it to your exact satisfaction.

Get Airline here.

Themes

I find that I switch between themes depending on my mood, what I’m doing, and what the environment around me is like - for example, often light-background Papercolor is more readable than Solarized.

Luckily, this being Vim, it’s easy to write a couple of macros to switch between them and to set the background between light and dark.

For the themes I’m just going to add a screenshot from their homepage - there’s really not much I can say about these, other than I find them a good balance of colours.

Papercolor

Papercolor in light mode:

Get Papercolor here.

Solarized

Solarized8 in light mode:

Get Solarized8 here.

Dracula

Get Dracula here.

Useful key mappings

Themes and background

nnoremap <leader>tp :colorscheme PaperColor<CR>
nnoremap <leader>ts :colorscheme solarized8<CR>
nnoremap <leader>td :colorscheme dracula<CR>

nnoremap <leader>bd :set background=dark<CR>
nnoremap <leader>bl :set background=light<CR>

Other stuff

Toggle NERDTree on and off; I have to admit I mostly just turn it on these days, but it’s nice to have the easy option.

nnoremap <leader>n :NERDTreeToggle<cr>

These I use more than I should, since I keep fiddling with my .vimrc. (Faux productivity for the win!)

nnoremap ,v :edit $MYVIMRC<CR>
nnoremap ,u :source $MYVIMRC<CR>

Using Closures with Axios

Paul Walker — Sun, 30 May 2021 01:00:00 +0000

Recently I’ve been working on integrating with a subscription/payment gateway. (It hasn’t been straightforward, but that’s a whole other post…)

I wanted to be able to test my web-hook code without repeatedly triggering events from the gateway. I stored the incoming events in JSON format, which was fine - but then of course I needed to take the stored events and do something with them.

I thought it might be interesting to make a note of where I started from and how I got to the end. I’ve included the mistakes I made along the way, so if you read a bit and think “that won’t work!” - I probably found that out in the next paragraph. :-)

Starting out

Starting simple - read the file into an array of objects, and then loop through printing a couple of details from each event, so we know it loaded okay.

As this is test code I’m going to use the Sync version of readFile to keep the code simple - no callbacks, and we can feed the result of readFileSync straight into JSON.parse, like so:

const fs = require('fs');

function run() {
    const json = JSON.parse(fs.readFileSync(__dirname + "/events.json"))

    for (const event of json) {
        console.log("event: ", event.id, event.event_type);
    }
}

run()

Sure enough, we get what we expect.

$ node post-events.js
event: 1 Hello
event: 2 World

It works, but the loop is going to post the events really quickly. I’d prefer to space them out - it makes it easier to watch the receiving code that way, and I’m not trying to stress test it at this point.

Sending them gradually

setTimeout works nicely for queuing a function to be executed in the future. The simplest thing for the waiting time is to use the position in the array. The for...of construct doesn’t give us the index, so we’ll have to use a different method.

forEach can give us both the item and index, so let’s use that. It’s just the loop which changes - the file-reading and JSON-parsing stays the same, so I won’t repeat it.

json.forEach((event, index) => {
    console.log(`Event ${event.id}: ${event.event_type}`);
    console.log(`Will delay ${(index + 1) * 1000} ms`);
})

And yep, we’re good:

$ node post-events.js
Event 1: Hello
Would delay 1000 ms
Event 2: World
Would delay 2000 ms

Scheduling

Now we just need something to schedule. Let’s try the simplest thing first - for each event, queue a function taking the event as a parameter to print out the event id.

json.forEach((event, index) => {
    const timeout = (index + 1) * 1000;
    console.log(`Event ${event.id}: ${event.event_type}`);
    console.log(`Will delay ${timeout} ms`);
    setTimeout(event => console.log("Posting", event.id), timeout);
})

And:

$ node post-events.js
Event 1: Hello
Will delay 1000 ms
Event 2: World
Will delay 2000 ms
post-events.js:10
        setTimeout(event => console.log("Posting", event.id), timeout);
                                                         ^
TypeError: Cannot read property 'id' of undefined
    at Timeout._onTimeout (post-events.js:10:52)
    at listOnTimeout (node:internal/timers:557:17)
    at processTimers (node:internal/timers:500:7)

After thinking about it that makes sense, and I really should have known better.

The event parameter is read when the function runs. Because of the timeouts the functions run after the loop has finished - at which point event is no longer defined, which is what we’re seeing.

Closure

What we can do is create what’s known as a closure. A closure is essentially a function together with the environment present when it was created. Luckily JavaScript makes that easy.

function makeFunc(event) {
    console.log("Making func for", event);
    return async function() {
        console.log("Posting", event.event_type);
    }
}

Yet another version of our loop:

json.forEach((event, index) => {
    const timeout = (index + 1) * 1000;
    console.log(`Setting timeout for Event ${event.id}; delay ${timeout} ms.`);
    setTimeout(event => makeFunc(event), timeout);
})


$ node post-events.js
Setting timeout for Event 1; delay 1000 ms.
Setting timeout for Event 2; delay 2000 ms.
Making func for undefined
Making func for undefined

Well … something has gone wrong there. What’s happened is that because we wrote event => makeFunc(event), the call to makeFunc hasn’t happened straight away, but has been delayed - which gives us the same problem as before. Let’s make it an immediate call:

json.forEach((event, index) => {
    const timeout = (index + 1) * 1000;
    console.log(`Setting timeout for Event ${event.id}; delay ${timeout} ms.`);
    setTimeout(makeFunc(event), timeout);
})

And see how that does:

$ node post-events.js
Setting timeout for Event 1; delay 1000 ms.
Making func for { id: 1, event_type: 'Hello' }
Setting timeout for Event 2; delay 2000 ms.
Making func for { id: 2, event_type: 'World' }
Posting Hello
Posting World

The POST request

That’s more like it. We’ll use axios for doing the POST to the HTTP endpoint.

const fs = require('fs');
const axios = require("axios");

const client = axios.create()

function makeFunc(event) {
    return async function() {
        console.log("Posting", event.event_type);
        const res = await client.post("http://localhost:8000/", event);
        if (res.isAxiosError) {
            console.error("Error posting");
        }
    }
}

function run() {
    const json = JSON.parse(fs.readFileSync(__dirname + "/events.json"))

    json.forEach((event, index) => {
        const timeout = (index + 1) * 1000;
        console.log(`Setting timeout for Event ${event.id}; delay ${timeout} ms.`);
        setTimeout(makeFunc(event), timeout);
    })
}

run()

Looking at the output

You can use a service like requestbin as an easy way to check what POSTs look like. For this I’ve decided to use fiatjaf’s requestbin - it’s small and simple.

And here we are - correct data, and spaced a second apart as we expected.

$ ./requestbin -port 8000
Listening for requests at 0.0.0.0:8000

=== 18:00:00 ===
POST / HTTP/1.1
Host: localhost:8000
User-Agent: axios/0.21.1
Content-Length: 29
Accept: application/json, text/plain, */*
Connection: close
Content-Type: application/json;charset=utf-8

{"id":1,"event_type":"Hello"}

=== 18:00:01 ===
POST / HTTP/1.1
Host: localhost:8000
User-Agent: axios/0.21.1
Content-Length: 29
Accept: application/json, text/plain, */*
Connection: close
Content-Type: application/json;charset=utf-8

{"id":2,"event_type":"World"}

I hope that helps someone, even if it’s just that we ran into the same ‘oops’ and we made mistakes together. :-)

Things I wish I knew about… JavaScript functions

Paul Walker — Sat, 29 May 2021 23:00:28 +0000

Especially coming from a C/Python/Elixir background, there were some things about JavaScript functions that I really didn’t get to start with. I thought I’d write them down in the hope they’ll help someone else on their journey.

I should note this is probably part one - there’s bound to be more things I learn about JavaScript functions as I keep using the language.

When one is async, all are async

I didn’t really understand how JavaScript does async when I started using it, so I spent quite some time trying to work out how a function could get a result from an asynchronous call and return it without the function caller having to be async itself.

If you’re aiming for the same thing, I’ll save you the bother - you can’t do it. I initially had high hopes for something like the construction below.

async function iAmAsync(num) {
  return num * 2;
}

function iUseThen(num) {
  return iAmAsync(num).then(res => res + 1);
}

console.log("iUseThen(3) =>", iUseThen(3));

What I didn’t realise was that iAmAsync(3).then(...) will implicitly return a Promise, meaning the whole of iUseThen will return a Promise.

iUseThen(3) => Promise { <pending> }

One approach I did find for using async functions in short scripts is to declare an anonymous async function and immediately invoke it:

(async function() {
    const result = await somethingNetwork();
    console.log("Result", result);
}) ()

What’s the difference between `function` and `=>`?

In JavaScript, => is called a ‘fat arrow’. Fat arrows are a shorthand way to create functions (with some restrictions as below):

function anonymous(name) {
  console.log("Hello", name);
}

you can use:

name => console.log("Hello", name);

Apart from anything it saves coming up with lots of different names for anonymous functions.

Limitations of `=>`

As handy as this is, there are some limitations of the fat arrow form.

No this

A function defined with => doesn’t have a this to reference. A (somewhat contrived) example - this works:

withFunction = {
  answer: 42,
  ask: function () {
    console.log("The answer is:", this.answer);
  }
};
withFunction.ask();

Producing:

The answer is: 42

This one doesn’t:

withArrow = {
  answer: 42,
  ask: () => {
    console.log("The answer is:", this.answer)
  }
}
withArrow.ask();


The answer is: undefined

A more real-world example of this can be seen with Vuex - if you’re defining a mutation or an action and you use a fat arrow function, it probably won’t work as you expect.

As an implication of this — because there’s no this, you can’t use super either.

Can’t be used as constructors.

If you’re defining a class, you must use the full function foo(bar) {} form.

Can’t use yield

I have to admit this hasn’t been a problem for me, I haven’t yet had a cause to use generators.

When to use `(foo) =>` and when to use `foo =>`?

The foo => ... form accepts one and only one parameter, and even then only if it’s a simple form.

If you need to indicate no parameters, bracket are required.

() => console.log("I'm not listening to you");

If you need to pass two, (foo, bar) => ... then it needs brackets. So this is fine:

foo => console.log("I'm a valid construction");

And this:

(foo, bar) => console.log("You gave me", foo, "and", bar);

But this is not:

foo, bar => console.log("While I won't run!");

Important note

If you need to do anything at all with that single parameter, you need brackets. For example - need to add a TypeScript type? Brackets. Need to destructure? Brackets. Want to supply a default parameter? Brackets. And so on.

What this boils down to - just because you can_do something, doesn’t mean you _should. For reference, see Kyle Simpson’s wonderful flowchart.

Brackets flowchart

When to use `foo => {bar; return baz}` and when `foo => bar`?

If the function body is a single statement, you can omit the braces. Otherwise, the braces are required.

One statement:

foo => foo + 1

More than one statement:

foo => {
    console.log("You gave me", foo);
    return foo + 1;
}

Closures

Kind of a hybrid - part data, part function. I’ve come across closures before, but JavaScript makes them easier to use than the other languages I’ve spent much time with.

A closure is essentially a function together with the environment present when it was created.

function makeClosure(outerArgument) {
  // Return a function which adds 'outerArgument' to
  // whatever argument it's given.
  return function(innerArgument) {
    return outerArgument + innerArgument;
  }
}

addOne = makeClosure(1)
console.log("Created addOne", addOne);
console.log("Two plus one is", addOne(2));

Which outputs:

$ node closure.js
Created addOne [Function (anonymous)]
Two plus one is 3

Note that the function returned is anonymous, simply because we didn’t name It in makeClosure. It’s quite possible to name it, though I haven’t found it to serve much purpose (so far).

That’s a trivia example of a closure - for a more useful one see my other blog post on using Axios.

Conclusion

I hope that’s been a useful introduction for someone — wish I’d known these when I started with JavaScript!

Building 'delta' on Alpine Linux

Paul Walker — Fri, 14 May 2021 15:24:07 +0000

Short post for a change this time. :-)

I've just built delta on an Alpine Linux image. There were a couple of dependencies I had to install before it would build, and one of them puzzled me for a bit.

Cargo was building and failed on crti.o:

 = note: /usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find crti.o: No such file or directory
collect2: error: ld returned 1 exit status

apk search didn't show anything useful. So, for myself in the future (or anyone else who happens to be searching for this) - the package that provides crti.o for Alpine is musl-dev.

How to force TypeScript’s tsc to rebuild your files

Paul Walker — Sun, 09 May 2021 12:21:32 +0000

I’ve noticed when using tsc in watch mode that sometimes - especially if a file has been renamed - it doesn’t realise the file needs recompiling. This state persists across reboots, restarting the watcher, all the usual stuff.

The last time I hit this was when I needed to re-order the seed files I’m using for this project. A file that had been last needed to be first, everything had to move around as I use numbers in the filenames to ensure the seeding order is clear. Mostly it works really well.

Renaming the files wasn’t a problem, but suddenly the lib/seeds directory has two sets of seeds in. The results from that - as you’d expect - were not good.

rm lib/seeds/*, re-run and … nothing. Hmm.

Maybe it needs prompting to recompile them? touch src/seeds/* and still nothing.

Last time this happened I rebooted the laptop and everything. This time, thankfully, I remembered before I got that far.

The only solution I’ve found? Delete tsconfig.tsbuildinfo and rebuild. From the filename I assume it seems caches the build info in that file, which is entirely reasonable. It just doesn’t always figure out when the cache is out of date.

Hope that saves someone a few reboots and some time.

DEV Community: Paul Walker

Making a subset of Font Awesome

Which runner?

Gulp

Grunt

Demo webpage

How to subset the files

Getting Grunt setup

Copying the files

Running the subset task

Running tasks by default

Add the CSS files to the webpage

Check the results

mbsync, SASL and Gmail

One Small Step...

My Apple purchase regrets

Runner-up

Winner

Things I wish I’d known sooner about… JavaScript modules

CommonJS

Exports

Require

ES2015

Export

Import

CommonJS

ES2015

User beware

Other reading

JavaScript pattern - the “singleton file”

The problem

The solution

Defining the logging module

Using the logging module

Why it works

Live reload in Hapi

livereload architecture

Server

Client

Plugin structure

Which hook to use?

livereload register

Modifying the HTML

Can we work with the response?

Can we parse it, and does it have the right nodes?

Is there a script already?

Add the script

Convert back to HTML

Put it all together

Trying it out

Conclusions

The good…

… and the ‘oops’

My 11 favourite Vim plugins

Plug

General functionality.

startify

NERDTree

Obsession

markdown-preview.nvim

Coding

CoC

Vimagit

ALE

Appearance

Airline

Themes

Papercolor

Solarized

Dracula

Useful key mappings

Themes and background

Other stuff

Using Closures with Axios

Starting out

Sending them gradually

Scheduling

Closure

The POST request

Looking at the output

What’s the difference between `function` and `=>`?

Limitations of `=>`

When to use `(foo) =>` and when to use `foo =>`?

When to use `foo => {bar; return baz}` and when `foo => bar`?