Tangible software development

#webpack #react #carpentry #sculpture

When I first started writing software professionally, I was puzzled by the fixation on tools. A few years later, I’ve come to realize that the tools we use drastically affect both the efficiency of our work and how fulfilling it is. For comparison, imagine a carpenter framing a house without power tools. It can be done, but requires much more time and effort.

At Cascade Energy, we’re a small dev-ops team with a lot of work to do. This makes our choice of tools important. Over the past year, we’ve introduced automated deployments, static analysis and re-formatting (with CodePipeline, eslint and prettier, respectively). This post will focus on one slightly more experimental tool that is redefining our workflow: hot reloading.

Recently I was tasked with adding a new feature to our customer-facing React application. Here is the process I went through, repeatedly:

Decide on a small unit of change I could do at a time
Read through the code and click through the production application to figure out how it currently works
Read tests related to this component
Make a small change to the code itself
Save the file
Switch to my web browser
Refresh the page
Select a sample customer from a dropdown menu to display data on the page
Scroll down the page to the component I'm working on
Click on the component
See if my change worked
If not, repeat

Many of these steps are unavoidable. For instance, most developers will tell you they spend more time reading code than writing it. However, we can consolidate steps 6-10 using hot reloading. With hot reloading configured, every small change I make automatically registers in the web browser, with the surrounding context preserved. There’s a lot of plumbing to make this happen, but once it’s set up, it’s magical.

These days, when creating a new frontend application, you can use a pre-configured starter pack that already has hot reloading and other productivity features out of the box (vue-cli, create-react-app, etc). In this case, we couldn't lean on these tools since this was an existing application with some custom configuration.

Our setup is a Node backend layer which handles connections to our other services and serves up our frontend React application. We use webpack as our frontend build system.

The webpack team maintains the pre-packaged webpack-dev-server module, but it wouldn’t work for our purposes, since the backend and frontend of our application are intertwined. If the backend of our application were built using express, then we could configure the server to use webpack-dev-middleware (used by webpack-dev-server under the hood) directly. However, we’re using hapi, which doesn’t support Express-style middleware.

Only slightly discouraged, I took a deep breath and pushed on. It was still possible to write a wrapper around webpack-dev-middleware. Fortunately, I found an article that got me started-- a tutorial for writing a hapi middleware adaptor for webpack.

I borrowed the basic premise of hooking into hapi’s onRequest and onPreResponse lifecycle extension points to intercept requests and pipe them to webpack so it can handle hot reloading. However, I didn’t find the author’s suggestion of webpack-dashboard to be any more helpful than webpack’s built-in logging capabilities, and it obscured our API logs which normally get routed to the same console.

With a bit more tinkering, I was able to get webpack-dev-middleware hooked up to hapi. Here’s roughly where that got us.

// Install dev middleware
server.ext("onRequest", (request, reply) => {
  if (passThroughRequest(request)) {
    return reply.continue();
  }

  devMiddleware(request.raw.req, request.raw.res, err => {
    if (err) {
      return reply(err);
    }
    return reply.continue();
  });
});

// Install hot middleware (for module reloading without reloading the page)
  server.ext("onPostAuth", (request, reply) => {
    if (passThroughRequest(request)) {
      return reply.continue();
    }

    hotMiddleware(request.raw.req, request.raw.res, err => {
      if (err) {
        return reply(err);
      }
      return reply.continue();
    });
  });

  // Make sure react-router can handle our actual routing
  server.ext("onPreResponse", (request, reply) => {
    if (passThroughRequest(request)) {
      return reply.continue();
    }

    return reply.file("public/index.html");
  });

(passThroughRequest ignores a few paths that need to skip webpack and go straight to the backend.)

With this set up, I tried saving a change to a module. However, instead of a hot reload, I got a warning in the console:

Ignored an update to unaccepted module ./client/components/project/tileView/ProjectTile.js -

…

process-update.js?e135:104 [HMR] The following modules couldn't be hot updated: (Full reload needed)
This is usually because the modules which have changed (and their parents) do not know how to hot reload themselves. See https://webpack.js.org/concepts/hot-module-replacement/ for more details.
…

It turns out that not only do we need to wire up webpack for hot reloading, we also have to teach our frontend components to hot reload themselves. This way, when webpack’s watch process notices the components have changed, it can inject just the changed bit of code and not reload the whole page. Each framework has a different approach to this. React has react-hot-loader, a pet project of Dan Abramov that, despite being quite experimental, is well supported and active. Abramov has written extensively about the concepts behind it, and this article is well worth a read. Essentially, you have to mark your top-level App component as hot-exported:

App.js
import React from "react";
import { hot } from "react-hot-loader/root";

import Routes from "./Routes";
import CascadeTheme from "./theme/Cascade";
import { AppContainer } from "./sharedComponents";

const App = () => (
  <CascadeTheme>
    <>
      <AppContainer>
        <Routes />
      </AppContainer>
    </>
  </CascadeTheme>
);

export default hot(App);

We also had to make some changes to webpack config to load both react-hot-loader and the webpack-hot-middleware client. This is the relevant section:

if (process.env.npm_lifecycle_event
 === "start") {
  config = merge(common, {
    devtool: "cheap-module-eval-source-map",
    plugins: [new webpack.HotModuleReplacementPlugin()],
    module: {
      rules: [
        {
          // Mark everything matching this rule as "cold" (e.g. not hot exported)
          // This will allow hot reloading to work as expected for the rest of the
          // application
          test: /\.js?$/,
          include: /node_modules/,
          exclude: /node_modules\/@sensei\/shared-components\/src/,
          use: ["react-hot-loader/webpack"],
        },
      ],
    },
  });

  config.entry.app = ["webpack-hot-middleware/client", ...common.entry.app];
}

As you can see, this config only applies when the app is run via npm start (aka, in development).

So I got that working, PR’d and merged. Then one of our other frontend developers noticed a bizarre occurrence—logout functionality was broken while using hot reloading in development. The app was still visible to logged-out users but in a broken state, with all calls to the backend failing. I realized that all of our webpack dev/hot middleware calls were getting through, regardless if the user was authenticated or not.

I had nagging feeling that there was a fundamental security flaw in my hot reloading implementation, and that I'd have to just make peace with manually refreshing the page every time I made a change. Regardless, I pressed on.

I tried debugging the extension points and checking the auth object, but it seemed that cookies had not been parsed yet. My first instinct was not a helpful one— I tried importing our hapi authorization plugin and re-injecting it into the new server request extension event methods. This led to numerous forays into the internals of hapi which started to develop a bad code smell.

So I took a deep breath and a step back and re-read hapi’s documentation. The most useful bit was the request lifecycle.

It turns out that in the example posted above, the author relied on the onRequest extension points, which come before authentication in the request lifecycle. The solution was to use the onPostAuth extension point to hook up webpack-dev-middleware and webpack-hot-middleware, so that our normal authentication middleware still processed each request. However, I still needed the onPreResponse handler to serve the HTML file, so we needed to check authorization at that point, too.

Here is what we ended up with:

/* eslint-disable consistent-return */
const config = require("config");
const webpack = require("webpack");
const webpackDevMiddleware = require("webpack-dev-middleware");
const webpackHotMiddleware = require("webpack-hot-middleware");

const webpackConfig = require("./webpack.config");

const IGNORE_PATTERN = /^\/(api|styles|img)\//;

function installWebpackHapiMiddleware(server) {
  // If we're not running under `npm start`, bail
  if (process.env.npm_lifecycle_event !== "start") {
    return server;
  }

  console.log(
    "You appear to be running a development server. Initializing webpack dev/hot middleware..."
  );

  const compiler = webpack(webpackConfig);

  const devMiddleware = webpackDevMiddleware(compiler, {
    port: config.port,
    historyApiFallback: true,
    publicPath: webpackConfig.output.publicPath,
  });

  const hotMiddleware = webpackHotMiddleware(compiler);

  // Install dev middleware
  server.ext("onPostAuth", (request, reply) => {
    if (passThroughRequest(request)) {
      return reply.continue();
    }

    devMiddleware(request.raw.req, request.raw.res, err => {
      if (err) {
        return reply(err);
      }
      return reply.continue();
    });
  });

  // Install hot middleware (for module reloading without reloading the page)
  server.ext("onPostAuth", (request, reply) => {
    if (passThroughRequest(request)) {
      return reply.continue();
    }

    hotMiddleware(request.raw.req, request.raw.res, err => {
      if (err) {
        return reply(err);
      }
      return reply.continue();
    });
  });

  // Make sure react-router can handle our actual routing
  server.ext("onPreResponse", (request, reply) => {
    if (passThroughRequest(request)) {
      return reply.continue();
    }

    return reply.file("public/index.html");
  });

  return server;
}

function passThroughRequest(request) {
  const isNotAuthenticated = request.auth.mode === "required" && !request.auth.isAuthenticated;
  return isNotAuthenticated || IGNORE_PATTERN.test(request.path);
}

module.exports = installWebpackHapiMiddleware;

Software development can often feel intangible, since the code we write looks very different from the direct machine instructions it eventually becomes, through complex processes that few of us fully understand. Ultimately, the immediacy of hot reloading brings our daily workflow closer to that of a carpenter working with physical tools-- make a change, and see it reflected immediately. I celebrate tangible software development and the tools that make it possible!