Why I developed a static Website Builder 🤦🏻‍♂️

Dirk Holtwick — Wed, 02 Sep 2020 11:46:57 +0000

I love Vue.js from Evan You and I like static websites. Of course there are already solutions to combine these two passions like VuePress or Nuxt. But would I be a programmer if I would choose this simple way?

Of course, I wanted to get to the bleeding edge and was quickly inspired by Evans newest coup: vite. It throws the ballast of the webpack overboard and does everything right. First I tried my luck with it and vitepress, but unfortunately that was not quite what I was looking for.

So I took a step back and looked at the classics of static website generation: Gatsby, Hugo, Jekyll and 11ty. They did everything right too, but everything didn't come off the shelf as I would like it to. Especially since I had already built my own solution for SeaSite, with which all my websites were generated.

What do I want?

But what was it that I wanted? I have found out the following points for me:

Speed: I want to make changes in the code like I did with Vue.js and see the result immediately in the browser.
Flexibility: I would like to be able to influence every aspect of the code myself and be able to program. Preferably in Javascript.
Post processing: I would like to be able to easily adjust content after it has already been calculated. This was the core principle of SeaSite, which allowed me to optimize images and videos afterwards, but also to run translations of text passages for different language versions.

How do I do it?

Well, at point 1 I had already discovered esbuild in vite. It is so incredibly fast that I could not believe it. The result is also reliable and exactly as it should be. esbuild was set as a tool that I wanted to use.

So I first built a small Node.js script that transpiled a Javascript file. I also built a small library to register routes. The generation of the content should be done on-demand when the website is requested by a simple Express.js webserver. To generate the static pages I would simply generate and save the content for all registered routes. This worked great and took only milliseconds.

Quickly I wanted to have the comfort of vite, i.e. when files change, the browser reloads immediately. With Chokidar, I could watch the folder with the JS files and recompile everything via esbuild. With a little trick, the import cache of Node.js could be bypassed and the new JS could be loaded and executed. With socket.io a reload mechanism for the browser was quickly assembled.

Now everything should become more beautiful!

I had now finally caught fire and there was no turning back. Then it could also become more beautiful :) Unfortunately I didn't succeed in integrating Vue.js at the first go, but I also doubted if this would make sense at all. In SeaSite I had already used JSX and JSDOM. For another project I had already written a DOM abstraction, which is very lean. I now extended it in a way that HTML and XML could be generated easily with JSX.

This made it possible to manipulate the content with simple DOM actions. But how much nicer it would be, if the corresponding nodes could be found by CSS selectors. So I also implemented the css-parse and it worked fine.

Also, a markdown parser was already available from SeaSite and was only extended to provide metadata for the registration of routes while maintaining the pleasant speed.

Open Source!

So now everything was on board that was needed and it was time to create a simple unified structure to publish the project. A first goal was to describe the routes with simple data structures to get maximum flexibility. For common formats like HTML, XML, JSON, text and assets convenient methods were created.

Since everything had the appearance of a web server anyway, which can also spit out static pages, it was obvious to adopt the smart middleware pattern of Koa.js. This way, templates and plugins are easy to realize. A copy of the data structure mentioned serves then as context and the result is expected in the property ctx.body.

Here it is now, the final project. I would be very happy about help and ideas. Maybe it is not the greatest tool to create static websites, but maybe it is the basis for an even smarter solution that builds on it.

github.com/holtwick/hostic
npmjs.com/package/hostic

In the coming posts I will further explore some of the issues that arise when creating a website and how they can be solved with Hostic. The list of current ideas on topics:

Building a simple static website with Hostic
Building a blog with Markdown
Building a multilingual website and localization
Optimizations for search engines and accessibility
Hosting: Beaker Browser, see...

These websites are already driven by Hostic:

How to transpile with light speed 💫

Dirk Holtwick — Thu, 20 Aug 2020 19:56:22 +0000

For every developer there comes the moment where speed matters. It saves you a relevant amount of time and keeps the flow going.

esbuild is definitely fast and reduces the built time significantly. And it is nice and simple too, when it comes to set up.

Build

It can be started from command line or nicely being integrated in a node.js script like this:

const esbuild = require('esbuild')

const options = {
  target: 'node12',
  platform: 'node',
  jsxFactory: 'h',
  jsxFragment: 'hh',
  bundle: true,
  outfile: 'out.js',
  sourcemap: 'inline',
  loader: {
    '.js': 'jsx',
    '.css': 'text',
  },
  entryPoints: [sitePath + '/index.js'],
}

await service.build(options)

This will build a single JS file containing everything that is needed to run. It also translates JSX and uses the function h to create elements. It also loads files ending on .css as plain text. A source map will be written as well. All this is done in a fragment of a second! This is because esbuild is written in Go instead of Javascript, because speed matters sometimes.

Sitemaps

Speaking of source maps the same author of esbuild also wrote a module to support them on node: node-source-map-support.

Testing

Now the setup is almost complete, but how about testing? I usually use jest for testing and therefore I wanted to get it working here as well. The solutions available did not fit my case, therefore I wrote my own transform:

First make sure to tell Jest what to do in a package.json section:

"jest": {
  "transform": {
    "^.+\\.jsx?$": "./src/jest-transform.js"
  },
  "testEnvironment": "node",
  "testPathIgnorePatterns": [
    "node_modules/",
    "dist/"
  ]
}

The transformer looks like this:

// Inspired by https://github.com/aelbore/esbuild-jest#readme

const fs = require('fs')
const pkg = require('../package.json')
const esbuild = require('esbuild')

const external = [
  ...Object.keys(pkg.dependencies ?? {}),
  ...Object.keys(pkg.devDependencies ?? {}),
  ...Object.keys(pkg.peerDependencies ?? {}),
]

module.exports = {

  getCacheKey() { // Forces to ignore Jest cache
    return Math.random().toString()
  },

  process(content, filename) {
    esbuild.buildSync({
      target: 'node14',
      platform: 'node',
      jsxFactory: 'h',
      jsxFragment: 'h',
      bundle: true,
      outfile: 'out-jest.js',
      sourcemap: 'inline',
      loader: {
        '.js': 'jsx',
        '.css': 'text',
      },
      entryPoints: [filename],
      external,
    })
    let js = fs.readFileSync(file, 'utf-8')
    fs.unlinkSync(file)
    return js
  },
}

Competition

Why would you want to you use esbuild and not webpack, babel, rollup, etc.? Well, because it is fast and easy to use. The other solutions are blown up and become pretty complex after a while. They have many 3rd party dependencies, which can cause troubles as well.

If you want to experience the blatant acceleration, then try esbuild.

Photo by Traf on Unsplash

DEV Community: Dirk Holtwick