DEV Community: Andrew Walpole

Through the pipeline: An exploration of front-end bundlers

Andrew Walpole — Sat, 13 Feb 2021 06:46:36 +0000

This last week I spent a lot of time buried deep in the documentation of parcel, rollup, and esbuild. I've been on a quest. Mainly one that starts from a place of being fed up with webpack being slow and unwieldy to configure and manage. And I've been hearing and even playing around with these other bundlers a bit, well enough to know a happier path should exist.

My bundling needs

Let me lay out my specific need, because it is specific enough that no bundler was giving me everything I needed out of the box. And this is a good time to preface my forthcoming feedback with the notion that in playing with parcel, rollup, and esbuild, they ALL are wonderful, easy to quick-start bundlers if your use-case fits their default model or even handful of command-line argument options they provide.

I'm building and bundling .js and .scss files for Wordpress theme development.
I need it to handle multiple main entry points (four that eventually turned into two):
- js/main.js – is the front-end site entry point for javascript
- css/style.scss – is the front-end site entry point for css styles
- js/editor.js – is Gutenberg-specific javascript (not often used, but handy when you need it!)
- css/editor.scss – you guessed it, another Gutenberg-specific set of css for occasional visual block and block editor tweaks.
These four entry files should bundle into to four build files: build/main.min.js, build/main.min.css, build/editor.min.js, and build/editor.min.css.
While developing, I want any imported dependency .js and .scss file changes to trigger a rebuild and I want all those files plus .php file changes to trigger livereload in the browser.
I need to be able to install, import, and bundle 3rd party libraries via npm
I need it to be faster than webpack has been (around 5-10 second development build times)
Finally, I need all the usual good stuff: minification, sourcemaps, transpilation, etc.

Ok, so that lays out what I was looking for, and hopefully is priming googlebot for sending you here in the first place. From here on out, I'm going to get a lot terser, and jump directly to my pros, cons, and learning moments with each of the bundlers I checked out.

Parcel 1.x (and a little bit of 2.x)

Pros:

I started with parcel, I've used it in other (more modern stack) projects and it really does often just work, with minimal to no configuration.
It's pretty fast and has a very pleasant set of command-line and node API configuration options that are well documented.
It felt a bit awkward at first, but I ended up being fine with the

Cons:

1.x is no longer being actively developed, in favor of 2.x (in beta as of this writing), and while there's a good ecosystem here, a handful of paths end in finding an old Github issue with something like, "this is not currently possible in parcel 1.x, but will be available in parcel 2.0"
Assets, like fonts and images referenced in your css get bundled into your build directory automatically. I did not find a solution to stop this from happening.
I ended my journey with 1.x when I found that changing css files was causing js files to stop working until I resaved them, still a big mystery as to what was going on.
Also, in using multiple entry points, the editor build files always got a hash in the filename even when trying to turn it off, while the front-end files worked perfectly.
I tried out 2.x for a bit in hopes to find solutions, but while promising to be more capable and extendable, I found many of the new features to be tough to use, and the documentation to be hard to follow.
My 2.x journey ended when I took to tweeting at the creator for some guidance, only to find the thing I was trying to do was not possible. (I was trying to create a plugin to remove the hashing issue from the editor filenames, but it turns out you can't use a plugin without publishing it to something like npm, which I had no interest in doing.)

Learnings:

I'm certain that I will reach for parcel for other projects, especially with the parcel-bundler node API knowledge I got, I think it's a great API to tap into.
It felt a bit awkward at first, but I did eventually give in to importing my scss into my respective js files, as that feature seems to be common across bundlers. This cut my entry points down from four to two, and now I was only dealing with javascript entry points.
Here's my parcel build script before I moved on:

const Bundler = require("parcel-bundler");

//entry files
const entries = ["./js/main.js", "./js/editor.js"];

const baseOptions = {
  outDir: "./build",
  minify: true,
  sourceMaps: true,
  watch: process.argv.includes("--watch"),
  publicUrl: "./",
  logLevel: 4,
  contentHash: false,
};

(async () => {
  const bundler = new Bundler(entries, { ...baseOptions });
  return await bundler.bundle();
})();

Rollup.js

I felt a bit defeated, but remained positive that I might find the answers in rollup!

Pros

Similarly to parcel, rollup has a nice command-line and node API interface that I was able to get up and running with quickly.
The core features are a bit limited, but it makes up for it with many available plugins that are easy to install and use.
I was able to get rollup to do everything I wanted, but the build times started to stack up.

Cons

My biggest con is how long it did take me to get to where I had everything I wanted working. I sifted through github issues and google searches for far too long.
In the end, I decided to move on from rollup because I wasn't seeing a huge gain in bundling speed over webpack and it was a few seconds slower than I had gotten with parcel.

Learnings

I'll let my rollup.config.js do the talking:

// rollup.config.js
import resolve from "rollup-plugin-node-resolve";
import commonJS from "rollup-plugin-commonjs";
import scss from "rollup-plugin-scss";
import { babel } from "@rollup/plugin-babel";
import { terser } from "rollup-plugin-terser";
import livereload from "rollup-plugin-livereload";
import fg from "fast-glob";

//custom watching plugin to get rollup to rebuild when other files get changed
const watcher = {
  name: "watch-external",
  async buildStart() {
    const files = await fg(["assets/js/**/*.js", "assets/css/**/*.scss", "**/*.php"]);
    for (let file of await files) {
      this.addWatchFile(file);
    }
  },
};

//setup the plugins for both builds
const plugins = [
  watcher, //my custom file watcher
  resolve(), //related to resolving node_module references, I think.
  commonJS({ //this allowed things like: import jquery from 'jquery' to work
    include: "node_modules/**",
  }),
  babel({ babelHelpers: "bundled" }), //transpile!
  scss({ outputStyle: "compressed" }), //minified scss
  terser(), //for minifcation
  livereload(), //for livereload
];

//notice, two entry points here, handled like a champ!
export default [
  {
    input: "js/main.js", //the style.scss is imported in this file
    output: {
      file: "build/main.min.js",
      format: "iife", //immediately invoked function expression - browser solution
      inlineDynamicImports: true,
    },
    plugins: plugins,
  },
  {
    input: "js/editor.js", //the editor scss is imported in this file, also
    output: {
      file: "build/editor.min.js",
      format: "iife",
      inlineDynamicImports: true,
    },
    plugins: plugins,
  },
];

esbuild

While rollup worked fine, I've been hearing about the new kid on the block, esbuild, and its ridiculous build times, so it was time to dive in.

Pros

Yet again, good documentation, good command-line and node experiences here.
HOLY CATS, THIS THING SCREAMS! 150ms folks! From 8-10 seconds to 150ms! I honestly thought it was broken, but there they were, my build files were pristine.

Cons

With multiple entry points, I had to concede being able to name the build files specifically. I had found workarounds in rollup for this, but I seem to be out of luck with esbuild. So instead of build/main.min.js I'm left with just build/main.js. I can deal.
esbuild is young, and while it's working well, the ecosystem around it is a bit scrappy feeling. I have a hunch that had I not just done prior battle with parcel and rollup, I may have been scared off by esbuild at first glance.

Learnings

esbuild just (like 12 days ago) got a watchFiles flag, and it works, but it doesn't seem to let you watch files outside of the entry points
I was able to cobble together chokidar and browserSync to create the watching and livereload solution I wanted.
Here's my final esbuild build script:

//esbuilder.js
const esbuild = require("esbuild");
const sassPlugin = require("esbuild-plugin-sass");
const chokidar = require("chokidar");
const browserSync = require("browser-sync").create();

const build = async () => {
  console.log("Building...");
  const service = await esbuild.startService();
  try {

    const timerStart = Date.now();

    // Build code
    await service.build({
      entryPoints: ["js/main.js", "js/editor.js"],
      format: "iife",
      bundle: true,
      minify: true,
      outdir: "build/",
      plugins: [sassPlugin()],
      sourcemap: true,
      //this stops esbuild from trying to resolve these things in css, may need to add more types
      external: ["*.svg", "*.woff", "*.css", "*.jpg", "*.png"],
    });

    const timerEnd = Date.now();
    console.log(`Done! Built in ${timerEnd - timerStart}ms.`);

  } catch (error) {
    console.log(error);
  } finally {
    service.stop();
  }
};

//watch it?
if (process.argv.includes("--watch")) {
  //chokidar will watch theme files for changes to trigger rebuild
  const watcher = chokidar.watch(["js/**/*.js", "css/**/*.scss", "**/*.php"]);
  console.log("Watching files... \n");

  //first build
  build();
  //build on changes
  watcher.on("change", () => {
    build();
  });

  //browserSync will trigger livereload when build files are updated
  browserSync.init({
    //TODO: make these values passed in by `npm run dev`
    port: 3334,
    proxy: "localhost:3333",
    files: ["assets/build/*"],
  });
} else {
  //no watch flag, just build it and be done
  build();
}

Bonus! estrella

Estrella is somewhat of a wrapper around esbuild. And while I found esbuild pretty easy to work with, estrella cranks up the simplicity-factor to 11. I liked working with estrella, and I think if you're interested in esbuild but are finding it a bit intimidating, it could be the starting point you need!

Conclusion

The conclusion is that I enjoyed learning about all of these bundlers, they work well, and under much less specific circumstances, all would do pretty well as a bundler choice. In the end, the speed of esbuild is significant enough that it truly changes the development experience in a way that maybe we weren't complaining about before, but might begin to in the future now that we know what's possible.

The Future of Computation is Robust

Andrew Walpole — Mon, 20 May 2019 20:42:23 +0000

Most of the progress made in computer science and technology stands on the platform of correctness and efficiency. Correctness, hardware and software determinism; the idea that if we want to live in a digital world, the hardware can’t fail, the software can’t fail. No bugs in the hardware. Period. And when there’s a bug in the code, fix it quick, or you might as well scrap it all. Efficiency; go faster, do more, squeeze out every single trivial compute cycle. And we adorn you with adulation for being the elite among us.

Centralized processing — the CPU — necessary to manage and maintain correctness; computation only happens with the perfect linear path of execution, and we make sure it’s efficient. Random-access memory, the ability to organize information with teleportation — incredibly efficient.

But today, Moore’s law is straining under the weight of its own progress, hardware is pushed to the very limits of material science, making it more fragile than ever, software still has bugs, and the security repercussions of the entire system is having increasingly devastating consequences on our socio-economic world. And yet, we still want to go faster, do more.

The history here is rich and complex, so excuse me for leaving any semblance of that out, but getting to the point is important, and if need be, digging into the details later is not a fruitless effort, I encourage it. I, myself, need to do more of it.

This architectural foundation was established long ago, and it was quick to deliver in capability and growth. So much so, that many stopped questioning it. But in not questioning it, we miss the opportunity to ask if there’s another way to compute. And there is.

There’s an uncharted world of computer science and we’ve found a door to it.

A place where centralized processing doesn’t exist. It’s replaced with distributed processing that removes the bottle-neck and allows for indefinite hardware and software scalability. This alone is a concept that unlocks a path worthy of traveling down. It’s a place with no concept of RAM. Instead, it’s much like our own physical world — spatially architected — limiting (from the perspective of efficiency), but importantly so. A centralized manager of process execution is replaced by distributed, randomly-executing actors, that compute by way of cause and effect. And if their cause doesn’t affect, perhaps they just try again; their best-effort actions working together to achieve a localized computational goal. A world where bugs are constant, and yet it does little to stifle progress and execution. Where hardware can blow up, and the system perseveres.

This is the world of robust-first and best-effort computing. A place where efficiency and correctness still play a role, but not the role. A place where robustness, durability, thick-skin — call it what you will — enters into the platform as a first-class concept, bringing with it incredible sturdiness to our ability to compute and to grow our technology into the future, much further and safer than ever thought possible.

When you see the software that runs here, as rudimentary as it may be today, it looks much less like any construct of software you have built in your head, instead, it looks like life. And in fact, living systems principles play a major foundational role in how you make computation happen robustly in this system. For any AI enthusiasts out there, whether you’re an expert in building complex neural networks or just a fan of these concepts that you may one day hope to understand, this world feels like a step in a new direction that, in its simplicity, unlocks a door to understanding our own world and the living systems within it.

There’s more — a lot more — to it all. But, this post is not a comprehensive explainer. It’s barely a teaser. And in fact, herein lies the last point, maybe the most important point to make. The answers to your questions, many of them don’t even exist yet. And so, there is an incredible opportunity for us all, one that is rare. To journey into a world of computer science that is undeclared and undocumented. A place where the simplest observations matter, where experimentation is the only work to do in order to have a chance of colonizing this new world.

Right now there is a small group of us, led by the person who has spent the most time in this new world, Dr. of Computer Science and Associate Professor Emeritus, David Ackley. And, perhaps like you, we’re not the most intellectually equipped to go there, but there’s something about it that we can’t stop thinking about. Like an adventure to be seized that fuels our excitement for the possible. Please consider joining us, please consider asking about possibly joining us. If any of these ideas resonate or intrigue you, you’re qualified, you’re hired, we would love to have you.

So what now? The following is a link dump that can help you learn more about the Robust-First Computing movement.

We congregate in our Gitter Chat Room — come say hi, ask questions!
The T2 Tile Project is Dr. Ackley’s weekly vlog and journey into building an indefinitely scalable hardware platform on top of robust-first concepts. A long, but incredibly worth-while rabbit-hole.
A very notable video from the T2 Tile project, affectionately known as the Christmas rant, does a great job of pointing out what it is and why this new type of computing may be very worthwhile from many perspectives.
You can also find many useful talks diving in into the depths of the Robust-First ecosystem and concepts on his personal Youtube Channel
A few notable starting points are, Robust-First Computing: Beyond Efficiency and Demon Horde Sort
If you want to see what this all looks like. My biggest contribution to the project thus far has been the website, mfm.rocks which is a loosely-interpreted simulation of Dr. Ackley’s Moveable Feast Machine. It doesn’t follow all of the the rules perfectly, but is enough to show you how computation might happen in this decentralized, actor-based world. This project is open-source and I welcome contributions!
The Robust-First Wiki dives into the constructs of the current software and some hardware of this new architecture. It can also lead you to running the MFM on your own machine.
You can find more research and links at Dr. Ackley’s website

If you got this far, thanks for reading. If you have any thoughts or questions, I would love to see them show up in the comments! I’ll leave you with the Robust-First Computing Team Creed: