DEV Community

Cover image for One thing led to another and I built my own static site generator today
Ben Halpern
Ben Halpern

Posted on

One thing led to another and I built my own static site generator today

#webdev #javascript #ruby #html

I started by building a static site as a small side project for my brother—but then I wanted partials... and regression tests. I thought partials which could help me inline the CSS and JS tags while breaking the code up into different files for organizational purposes in development. I like to inline the assets to avoid render-blocking latency on simple landing pages which will likely be served over unreliable network conditions.

At first I really didn't think I needed a generator at all, but one thing led to another and I kind built a basic one myself.

It consists of a build.rb file that looks like this...

prod_build = ARGV[0] == "for_prod"

# Read files
meta_html =       File.open("workspace/meta.partial.html").read
style_css =       File.open("workspace/style.partial.css").read
body_html =       File.open("workspace/body.partial.html").read
json_data =       File.open("workspace/data.json").read
scaffold_js =     File.open("workspace/scaffold.partial.js").read
dynamic_js =      File.open("workspace/dynamic.partial.js").read
analytics_html =  File.open("workspace/analytics.partial.html").read
base_html =       File.open("workspace/base.html").read
test_html = ""

unless prod_build
  test_html = File.open("workspace/test.dev.html").read
end

# Create built page
build_string = base_html
  .gsub("{{ meta }}", meta_html)
  .gsub("{{ style }}", style_css)
  .gsub("{{ html }}", body_html)
  .gsub("{{ data }}", json_data)
  .gsub("{{ scaffold_script }}", scaffold_js)
  .gsub("{{ dynamic_script }}", dynamic_js)
  .gsub("{{ analytics }}", analytics_html)
  .gsub("{{ test }}", test_html)

# Write to target page

if prod_build
  puts "Production build.... index.html"
  File.write("index.html", build_string)
else
  puts "Development build.... wip-index.html"
  File.write("wip-index.html", build_string)
end
Enter fullscreen mode Exit fullscreen mode

I could DRY up this code, but I prefer it to be dumb and super explicit at this stage.

As you can see, this is just basic string find and replace. {{ could just as easily have been 💩💩 or [cromulent >>. It's completely arbitrary, but {{}} looked fancy.

base.html looks like this...

<html lang="en">
  <head>
    {{ meta }}
    <style>
      {{ style }}
    </style>
  </head>
  <body>
    {{ html }}
    <script>
      // Data
      var data = {{ data }}

      // Code
      {{ scaffold_script }}
      {{ dynamic_script }}
    </script>
    {{ analytics }}
    {{ test }}
  </body>
</html>
Enter fullscreen mode Exit fullscreen mode

...I even wrote my own dependency-free JavaScript test suite. I'll share more once it's further along.

I probably should have reached for an existing static site generator instead of doing this from scratch, so why did I take this approach?

In all seriousness, I generally like to avoid dependencies when doing projects like this so it's easier to hop in for a quick change in the future without having to install a bunch of old dependencies. Building a whole toolchain myself is sort of silly, but fun!

If you don't want to be like me, you may want to check out this great thread...

ben

What is the simplest static site framework?

Ben Halpern ・ Feb 7 '20

#discuss #html #webdev

Happy coding!

Top comments (28)

Collapse
 
hyftar profile image
Simon Landry •

I like this approach because, even though you're most likely reinventing the wheel, you often learn new stuff along the road and learning is the most important thing in our industry. Hats off to you.

Collapse
 
ben profile image
Ben Halpern •

😊

Collapse
 
dtinth profile image
Thai Pangsakulyanont • • Edited

I agree about keeping dependencies low in small projects! When I hop between multiple projects, I can feel the context switching cost if there are setups involved.

By the way, this looks like a great use-case for…

<html lang="en">
  <head>
    <?php include 'workspace/meta.partial.html'; ?>
    <style>
      <?php include 'workspace/style.partial.css'; ?>
    </style>
  </head>
  <body>
    <?php include 'workspace/body.partial.html'; ?>
    <script>
      // Data
      var data = <?php include 'workspace/data.json'; ?>;

      // Code
      <?php include 'workspace/scaffold.partial.js'; ?>;
      <?php include 'workspace/dynamic.partial.js'; ?>;
    </script>
    <?php include 'workspace/analytics.partial.html'; ?>
    <?php if (isset($_GET['test'])) include 'workspace/test.dev.html'; ?>
  </body>
</html>

# Launch a development web server
$ php -S 0.0.0.0:1234

# View production build
$ open http://localhost:1234

# View test build
$ open http://localhost:1234/?test

# Build static site
$ php index.php > index.html
Collapse
 
ben profile image
Ben Halpern •

I haven’t used PHP in years... and this seems very appealing!

Collapse
 
thanasismpalatsoukas profile image
Sakis bal •

What stack do you primarily use? Thanks for your response beforehand.

Collapse
 
sirseanofloxley profile image
Sean Allin Newell •

Blasphemy! Beautiful blasphemy!

Collapse
 
dtinth profile image
Thai Pangsakulyanont •

BlasPHPemy!

Collapse
 
fifo profile image
Filipe Herculano •

That’s awesome!!! 👏

I’ve been there too, I once was tinkering with markdown parsing when I built this small library (link below) and it turned out that the live demo I made of using it could be seen as a somewhat static site generator from markdown files

I also felt kinda silly initially but now I want to revisit it as it was kinda fun too (and also try building a dependency free one from scratch like yours)

GitHub logo this-fifo / use-marked-hook

A react hook for parsing markdown with marked and sanitize-html

useMarked() hook

A react hook for parsing markdown with marked and sanitize-html

NPM JavaScript Style Guide

Live Demo

The app located at /example demonstrates how it could be used, see the live result at this-fifo.github.io/use-marked-hook/

Install

yarn add use-marked-hook

Usage

import React from "react";
import { useMarked } from "use-marked-hook";

const App = () => {
  const markdown = `**bold content**`;
  const html = useMarked(markdown);
  // html -> <p></strong>bold content</strong></p>
  return <div dangerouslySetInnerHTML={{ __html: html }} />;
};

License

MIT © Filipe Herculano


View on GitHub


Collapse
 
ben profile image
Ben Halpern •

Neat!

Collapse
 
giorgosk profile image
Giorgos Kontopoulos 👀 • • Edited

I had built a simple static site generator back before CMSs were a thing and SSGs had a name for themselves. I believe in those days everyone working on web development had something similar for organizing site development right ?

Collapse
 
phlash profile image
Phil Ashby •

v2 of my personal website was templated using Apache Velocity and some home-brew wrapper code to pick up resources from disk... I thought that was a silly idea back in about '05 and replaced it with a MoinMoin wiki. Now I'm back to Hugo :-/

Collapse
 
andrewbrown profile image
Andrew Brown 🇨🇦 • • Edited

This is mine. I decided to use erb since its not really much different than using handlebars.
Best part is if you don't use any ruby's gems other than standard easy to put on a lambda and use CodeBuild and CodePipeline to automatically deploy changes to S3 Static Website Hosting.

For my use case I have to compile at least a thousand pages and doing it this way is under a 1 minute

require 'erb'
require 'json'
require 'fileutils'

class Namespace
  def initialize(hash)
    hash.each do |key, value|
      singleton_class.send(:define_method, key) { value }
    end
  end

  def get_binding
    binding
  end
end

class Generator
  def self.render src_path, data={}
    html = ERB.new File.read(src_path), nil, '-'
    ns = Namespace.new data
    html.result(ns.get_binding)
  end

  def self.render_file src_path, build_path, data={}
    html = ERB.new File.read(src_path), nil, '-'
    ns = Namespace.new data
    File.open build_path, 'w' do |f|
      f.write html.result(ns.get_binding)
    end
  end

  def self.src_path name
    cwd = File.dirname __FILE__
    File.join cwd, 'src', "erb/#{name}.html.erb"
  end

  def self.build_path name
    puts "+ build/#{name}.html"
    cwd = File.dirname __FILE__
    path = File.join cwd, 'build', "#{name}.html"
    if File.exists?(path)
      File.delete path
    else
      FileUtils.mkdir_p File.dirname(path)
    end
    path
  end

  def self.json_data name
    cwd = File.dirname __FILE__
    path = File.join cwd, 'data', "#{name}.json"
    json = File.read(path)
    JSON.parse(json)
  end

  # when dealing with a file where each line is json
  def self.json_array_data name
    cwd = File.dirname __FILE__
    path = File.join cwd, 'data', "#{name}.json"
    data = []
    File.foreach(path).with_index do |line, line_num|
      data << JSON.parse(line)
    end
    data
  end
end

class PrepAnywhereGenerator < Generator
  def self.head
    src_path   = self.src_path('head')
    data       = {}
    self.render src_path, data
  end

  def self.header
    src_path   = self.src_path('header')
    data       = {}
    self.render src_path, data
  end

  def self.footer
    src_path   = self.src_path('footer')
    data       = {}
    self.render src_path, data
  end

  # /
  def self.homepage
    src_path   = self.src_path('homepage')
    build_path = self.build_path('index')
    data       = {
      head: self.head,
      header: self.header,
      footer: self.footer,
      data: self.json_data('homepage')
    }
    self.render_file src_path, build_path, data
  end

  # /textbooks
  # /textbooks/us
  # /textbooks/canada
  def self.textbooks
    src_path = self.src_path('textbooks')

    textbooks_all   = self.json_data('textbooks')
    textbooks_ca    = self.json_data('textbooks-ca')
    textbooks_us    = self.json_data('textbooks-us')

    # Canada Books
    build_path_all = self.build_path('textbooks/index')
    build_path_ca  = self.build_path('textbooks/ca')
    build_path_us  = self.build_path('textbooks/us')
    data = {
      head: self.head,
      header: self.header,
      footer: self.footer,
    }

    self.render_file src_path, build_path_all, data.merge({body_class: 'textbooks-all', data: textbooks_all})
    self.render_file src_path, build_path_us , data.merge({body_class: 'textbooks-us', data: textbooks_us })
    self.render_file src_path, build_path_ca , data.merge({body_class: 'textbooks-ca', data: textbooks_ca })
  end

  # /textbooks/:book
  def self.textbook data_key='textbook'
    src_path = self.src_path('textbook')
    books = self.json_array_data(data_key)
    books.each do |book|
      path = [
        'textbooks',
        book['permalink'],
        'index'
      ].join('/')
      build_path = self.build_path path
      data = {
        head: self.head,
        header: self.header,
        footer: self.footer,
        book: book
      }
      self.render_file src_path, build_path, data
    end
  end

  # /textbooks/:book/chapters/:chapter/materials/:material
  def self.material data_key='material'
    src_path = self.src_path('material')
    materials = self.json_array_data(data_key)
    materials.each do |material|
      build_path = self.build_path(self.material_path(material))
      data = {
        head: self.head,
        header: self.header,
        footer: self.footer,
        material: material
      }
      self.render_file src_path, build_path, data
    end
  end

  def self.material_path material
    [
      'textbooks',
      material['textbook_permalink'],
      'chapters',
      material['chapter_permalink'],
      'materials',
      material['permalink'],
      'index'
    ].join('/')
  end

  # /textbooks/:book/chapters/:chapter/materials/:material/videos/:video
  def self.video data_key='video'
    src_path = self.src_path('video')
    videos = self.json_array_data(data_key)
    videos.each do |video|
      build_path = self.build_path(self.video_path(video))
      data = {
        head: self.head,
        header: self.header,
        footer: self.footer,
        video: video
      }
      self.render_file src_path, build_path, data
    end
  end


  def self.video_path video
    [
      'textbooks',
      video['textbook_permalink'],
      'chapters',
      video['chapter_permalink'],
      'materials',
      video['material_permalink'],
      'videos',
      video['permalink'],
    ].join('/')
  end
end
Collapse
 
dividedbynil profile image
Kane Ong • • Edited

I had built one from scratch as well. Mine is mainly used for minimizing page size and optimizing page rendering, hot-reloading is included for development.

var gulp = require('gulp')
, minify = require('gulp-htmlmin')
, inlinesource = require('gulp-inline-source')
, browserSync = require('browser-sync').create()
, reload = browserSync.reload
, exec = require('child_process').exec
, jsonminify = require('gulp-jsonminify')
;

//  default hot-reloading task is `watch`
gulp.task('default', ['watch']);

// hot-reloading config
gulp.task('browserSync', function () {
    browserSync.init({
        server: {
          baseDir: 'public'
        },
    })
})

// here is the `watch` task
gulp.task('watch', ['inlinesource', 'minifies', 'browserSync'], function () {
    gulp.watch(
        ['develop/js/*.js', 'develop/*.html', 'develop/css/*.css'], 
        ['minify-index', reload]
    );
    gulp.watch('develop/html/*.html', ['minify-html', reload]);
});

gulp.task('minify-index', ['inlinesource'], function() {
    return gulp.src('public/*.html')
        .pipe(minify({
            collapseWhitespace: true,
            removeComments: true,
            removeAttributeQuotes: true,
            removeStyleLinkTypeAttributes: true
        }))
        .pipe(gulp.dest('public/'))
});

gulp.task('minify-html', function() {
    return gulp.src('develop/html/*.html')
        .pipe(minify({
            collapseWhitespace: true,
            removeComments: true,
            removeAttributeQuotes: true,
            removeStyleLinkTypeAttributes: true
        }))
        .pipe(gulp.dest('public/html/'))
});

gulp.task('inlinesource', function () {
    return gulp.src('develop/*.html')
        .pipe(inlinesource())
        .pipe(gulp.dest('public/'))
});

gulp.task('minify-json', function () {
    return gulp.src(['develop/*.json'])
        .pipe(jsonminify())
        .pipe(gulp.dest('public/'));
});

gulp.task('minifies', ['minify-html', 'minify-index', 'minify-json']);

gulp.task('deploy', ['minifies'], function (cb) {
    return exec('npm run deploy', function (err, stdout, stderr) {
        console.log(stdout);
        console.error(stderr);
        cb(err);
    });
});

The code above is about 3 years old, feel free to run it at your own risk.

Collapse
 
jsn1nj4 profile image
Elliot Derhay • • Edited

Lol I love that we're greeted with a :shrug: banner as we open the article.

Also, yeah, if you have time to build an interesting side project while building a side project, go for it. If I'm reaching for a static site generator, I like to use Nuxt at the moment—although I'm already partial to Vue and I haven't done anything crazy with it yet.

Collapse
 
leob profile image
leob •

The advantage is that you only get what you need, no less no more, minimal overhead (both mental and memory/cpu/LOC).

The disadvantage is that, when you find out that you need more, you'll run into limitations, and at that point you'll have to decide - add more features (and risk losing the charm & the simplicity), or migrate to a "real" SSG.

But if you know that your requirements will stay simple then it may be a good choice!

Collapse
 
v6 profile image
🦄N B🛡 •

Could you abbreviate it to bi.rb? For the memes?

Collapse
 
andi1984 profile image
Andreas Sander •

I can totally understand that you did it, as I did the same few months ago and I still like to take some holidays from the nowadays generators like Gatsby, NextJS, Hugo etc. and build my own thing.

I don't expect that this becomes popular or something. It just fits my purpose and helps me learning new things along the development road.

andi1984.dev/posts/my-static-site-...

Collapse
 
matthewbdaly profile image
Matthew Daly •

That's how I wound up with my current site. In late 2014 I was using Octopress, which was fine, but I don't really use Ruby professionally and so I thought ideally I'd be using a Node.js solution. On a whim I rolled a very simple proof of concept for a Grunt plugin to convert Markdown and Handlebars templates into HTML, and put together a Yeoman generator to set it up with some other plugins. It went so well that in early 2015 I switched over to it and have been using it since, though lately I have been considering switching to Gatsby.

Collapse
 
jaymeedwards profile image
Jayme Edwards 🍃💻 •

Cool learning exercise. Have you checked out Svelte? It’s got a similar tooling stack to react and angular but imho much lighter weight and less quirky you can definitely build static sites with it.

Just throwing it out there. Hope you’re doing well Ben.

svelte.dev/

Collapse
 
sharadcodes profile image
Sharad Raj (He/Him) • • Edited

Same here I also made two of the static site generators both are featured on staticgen.

Python based : my_py_site

github.com/sharadcodes/my_py_site

C++ based: sudo_site

github.com/sharadcodes/sudo_site

my_py_site is capable of doing more then rendering templates, it is able to generate multiple blogs without configuration and it also parses and separates the YML front matter and the Markdown content converted to HTML. You can access that YML front matter anywhere in the templates and moreover the Meta data of pages and posts is also accessible in the templates.

You can use different layouts for any post or page as well. Just specify layout in YML front matter.

All is done under 100 lines of code

Collapse
 
stealthmusic profile image
Jan Wedel •

Can you explain what you mean by “partial”. I mean, I see the files named .partial.* but what do want achieve with that?

Collapse
 
devvyd profile image
Dāvis Naglis • • Edited

Re-usability, for the most part. You don't want to just copy paste the same exact code in every page. You make a partial, for example, that damned footer you want to put in the same page - now you don't have to copy & paste manually all the time. Just add an include to it.

Another thing - change something in the partial file -> it reflects everywhere you do it. You don't want to hunt down that exact code in every damn page.

Collapse
 
corentinbettiol profile image
Corentin Bettiol •

Did the same thing... but in C, in 2018.

It's good to realize a static site generator only concatenate some files into some other files, it's really this simple.

View full discussion (29 comments)

Read next

husain01 profile image

Securing Next.js Applications with Robust CSP Headers

Husain Bhagat -

vishnusatheesh profile image

Controlled and Uncontrolled components in react

Vishnu Satheesh -

per-starke-642 profile image

Finding Clients as a (Web Development) Freelancer

Per Starke -

yashmahalwal profile image

Optimising React Apps

yashmahalwal -