DEV Community: Wei Ji

On your markdown, hypertext, now!

Wei Ji — Tue, 06 Nov 2018 02:35:51 +0000

Originally posted at letsjustwander.now.sh

    # language: en
    Feature: Free and minimal static blog
      In order to have a long-lasting blog
      As a blogger,
      I want a system that is portable

      Scenario: Blog host goes down
        Given a backup copy of my blog's content
         When my old blog host becomes unviable
         Then I can rehost my blog and content somewhere else

Bit of background, I've been blogging on and off since I was a teenager maybe 10 odd years ago. As a kid, spending money on hosting and domain names was something I avoided like the plague, and keeping things free is still something I'm obsessed with.

A problem I faced was keeping my blog content safe from blogging platforms going down, becoming obsolete, or getting sucked in to very corporate locked-in platforms. I have only a faint idea on where my first Windows Live Spaces posts are, stored in some proprietary format I may never unlock. The backups from my golden b2evolution years (late 2000s to early 2010s) are likewise stuck in some mysql database which is marginally 'better'.

Around 2014 after graduating from my degree (while travelling in Chile actually) I restarted blogging using Ghost, the beta version. It was a bridge connecting me from the old school PHP based Content Management Systems I've used before to something akin to 'modern' JAMstack.

But when Ghost 1.0 got released, I didn't bother migrating to the new API. Technically the posts were stored in JSON and I could parse it if I wanted to. Read: still a lot of effort to go on a trip down memory lane.

Really, it all seemed too much to maintain. Yes I could pay, and have someone manage the hosting, upgrades, backup and whatnot.

But stubborn ol' me thought the world must have gotten better than that. It's 2018 already for goodness sake! Was it too much just to have a few kilobytes or megabytes in the cloud to host some blog?

Nuclear disaster proof blog

Ok, I'm probably exaggerating. If you've read this far, you're probably hoping I've got a solution.

I'm not a web developer, but I do program and can follow instructions. The blogging workflow below will not be Wordpress WYSISYG friendly, nor is it reliant on some passing Javascript framework that may or may not exist in the next 5 years.

Because I'm paranoid, there should be two and at least two ways to do any part of the pipeline below.

So what do we use?

Static plaintext

One word - plain-text.

I'm not chucking my words into some database no matter how open source it is. Nor am I going to put it into JSON, YAML, or some other FOUR letter acronym format.

Each post should standalone as one file. As a minimum, the blog folder will have an index page and a folder for all the individual posts.

_posts/
- hello-world.md
- post-two.md
index.md

I write my posts in markdown and although there's a hundred flavours out there, I can probably still open up a .md file in ten years and read it. If I had written my posts ten years ago in markdown, I would still be able to read it. Period. You could use some other markup language (LaTeX anyone?) if you fancy, but I highly recommend markdown.

To store all the metadata of a post (e.g. date, title, tags, etc), I put it into a short YAML frontmatter like so:

---
date: 2018-10-31T12:00:00+12:00
title: Hello World
---

This follows the Jekyll frontmatter convention. But we won't be using Jekyll (gasp!), though it is a popular static site generator. I feel like there's too much boilerplate, too many folders for css files, plugins and whatnot.

The same can be said of most other static site generators. Everyone has an opinion on what's better, faster, trendier. Popularity means nothing if we're going for longevity, remember Geocities? Instead, we're going for something opinionatedly unopinionated.

Text to Hypertext

The only reason we need this text-to-hypertext step is so that your blog doesn't look like some git repository. Or, yes you could upload all your files to an FTP site and call it a day. If that's you, skip down to the publishing/hosting part below.

Today, we're striving for minimalism, but we're not compromising on quality!

I'll jump straight to the chase and recommend no nonsense Eleventy. Alternatively, skip ahead for Plan B - pure scripting!

Plan A - Using Eleventy

Install eleventy with npm in your terminal as follows:

    npm install -g @11ty/eleventy

Now go to the top-level directory of the blog and type this:

    eleventy

Eleventy will process your .md markdown files into .html, by default into a _site folder. Our folder should now look like:

_posts/
- hello-world.md
- post-two.md
_site/
- hello-world/index.html
- post-two/index.html
- index.html
index.md

And that's all there is, seriously! Okay maybe read the docs or add some plugins if you want more configuration jazz. Otherwise, take your _site folder host it somewhere (see next section).

Plan B - Using pandoc

Credits to @gypsydave5 for coming up with this!

If you haven't got swiss-army knife pandoc, install it.

    sudo apt-get install pandoc #linux
    brew install pandoc         #mac
    choco install pandoc        #windows

I'm using Linux as my system, so next I would fire up my bash shell and type in this command:

    for POST in _posts/*.md; do pandoc --to=html5 --output=_site/$(basename ${POST%.md}).html --standalone $POST; done

This is basically a for-loop that converts every .md markdown file in the _posts folder to a standalone .html file.

The caveat with this method compared to (a properly configured) eleventy is that it will look horrible. There's no CSS styling, so you'll get something that's akin to a 90s era webpage.

Luckily pandoc does allow you to set a --css file or url to make it prettier (credits to Martin). You could point to some pretty.css file you made, or cheat like me below and chuck in a CDN hosted css.

    for POST in _posts/*.md; do pandoc --to=html5 --css https://cdn.rawgit.com/yegor256/tacit/gh-pages/tacit-css-1.3.1.min.css --output=_site/$(basename ${POST%.md}).html --standalone $POST; done

I used tacit css above, which is a class-less CSS framework. That means I'm not bothering with opinionated class names that goes with using some front-end framework like Bootstrap or newer players out there. Simply write proper HTML5 (whatever proper means) and it will do the trick, at least until HTML6 comes out.

You might need to tweak this pandoc method further to get it to look as nice as eleventy. There's probably a million ways to do this step so I'll leave that to your imagination. Share it in a comment if you have another one-liner script that does it!

Now, let's jump straight into the scary part - publishing.

Publishing/Hosting

So you've written your blog post (in markdown), you've converted it into nice hypertext (html), now it's time to put it up somewhere! Again, I'm aiming for free. That doesn't mean it can't be easy.

So far I've been through more dodgy free hosting providers than I can count. What I dislike is a free lunch that I'll get a stomachache from later.

I'm of the opinion that publishing in itself is a scary thing, especially for new writers. But the last thing you want is to have a blog post written up and having it stuck on your computer staring back at you an only you. The word deserves your opinion, trust me.

You need to push it out into the world with a click of your fingers! While you stress out on what the world will think of your post, do you really want to think about how hosting/ssl-certs/domain-names/database-configurations works? No!

But hey, we're using static files now (<1MB in size), not some behemoth database! No database means no funky configuration. With static files, shouldn't there simply be a way to upload my files and get a URL link?

It's 2018, what do we have out there?

I'm going to show you the Zeit Now way for its Zen-like simplicity. Alternatively, skip ahead for Plan B - P2P hosting!

Plan A - Zeit Now

Install now-cli with npm in your terminal as follows:

    npm install -g now

Now go to the top-level directory of the blog and type this:

    now _site

Type 'y' for yes to confirm. It should take a second or three depending on how big your files are, and you'll be given a url to your blog! Something that looks like https://site-abcdefghij.now.sh

Of course, you might like a nicer url, in which case you can set up an alias like so:

    now alias https://site-abcdefghij.now.sh https://example.now.sh

And that's easy static hosting on Zeit Now in a nutshell. You can even alias it to your own custom domain if you've got the cash. I'll also give a shout out to Netlify Drop as an alternative hosting provider, just zip up your _site folder and drag-and-drop it to the website!

Congrats, you've made it! Read on though if you'd like to know another way ;)

Plan B - P2P Dat

Credits to this guide.

Install dat with npm in your terminal as follows:

    npm install -g dat

Now go to the top-level directory of the blog and type this to share the files in your _site folder:

    dat _site

It should take a second or three depending on how big your files are, and you'll be given a url to your blog! Something that looks like dat://abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopqrstuvwxyz12.

Wait, what's dat://? That's the P2P protocol part, you need a special browser like BeakerBrowser to open it. That may be okay for you, or you might prefer a standard https:// url.

Enter hashbase. Hashbase wraps around your dat and gives a nicer url like https://example.hashbase.io. For example, you can also access the Beaker Browser website at https://beakerbrowser-com.hashbase.io.

Hashbase acts as a super peer that hosts your files so you don't have to leave your computer switched on... So sign up for it, or be a host yourself by forking their Github repo.

I've always wanted to go the self-hosting route, and this is probably the closest I've gotten so far. Maybe someday I'll get a raspberry pi mini-server set up, but this will do for now.

Caveat lector

I can't predict what will happen to the future. Like I've mentioned in the beginning, I've blogged on and off for ten years or so. What seems like a good idea back then isn't quite so now. There is an element of hindsight bias in this post you need to be aware of.

This $0 plain text -> hypertext converter -> publishing/hosting pipeline assumes the following:

That plain text editors will continue to exist (and can open my markdown files). Maybe UTF-8 won't be valid anymore? Perhaps my favourite Atom editor will become obsolete?!!
That I'll be able to convert markdown to html in the future. Will Eleventy still work in ten years time? What will I do if pandoc goes away too?!!
That the hosting services I use and recommend now will remain free. What if Zeit Now/Github Pages/Netlify/Hashbase starts charging for use? Can the dat protocol actually continue into the future?!!

The enlightened ones may notice a potential Plan C baked into each step. Can't I edit plain text with any editor? Why, isn't it possible to convert markdown to html with some Python scripting? Surely there will be another hosting provider that will do it for free?

Let me point out though, that the weakest link in the picture might be yourself.

You are version controlling your blog posts right?

Git is awesome, I didn't mention it in the guide above but it's there in the background, silently making sure my history is intact.

There are also little details I left out from this guide such as navigation links, making an archive page, etc. So for good measure, all my static blog templates are hosted on Github here. It is as ridiculously barebones as I can make it, and you're free to copy or build off the design.

Sure, there are definitely cons with using this free static blog system. Some people don't think second level domain names are professional (but I can live with that). It also takes some extra effort to get interactive stuff on the page like video embeds.

But hey, there's no javascript being served! That means there's no popup ads (can be a good or bad thing). My site also loads blazingly fast in milliseconds.

The thing that bugs me most is having all my hard work disappear from the web in a few months or years, and having to migrate to somewhere else, usually during some inconvenient time. This portable design aims to make that eventual migration as painless as possible.

For now, this is probably the best I can come up with.

    echo 'Hello World' > index.md  #write
    eleventy                       #convert
    now _site                      #publish

And there you have it, a free static blog guide for 2018 and beyond.

Developing behaviours by starting with why

Wei Ji — Fri, 02 Nov 2018 00:40:14 +0000

Originally posted at letsjustwander.now.sh

In order to warn everyone about catastrophic sea level rise,
As a scientist,
I want to openly share a better map of what lies below Antarctica.

So I've been rambling on in my last few posts in a Zen/Chan-like trance, on suddenness. Indeed, people really shouldn't take it a step at a time, because it's all in the mind and there are better ways to look at things.

Yet Chan is not just about emptiness, it is also about balance.

So today we'll take a step back, looking at the individual steps you can take to reach the holy grail.

Note that this is the culmination of the things I've been going through in the past two weeks or so. There is a saying that doing a PhD is like running your own little company. A PhD student is like the founder nurturing the initial spark of an idea, the worker who turns it into a tangible thing, and also the salesperson selling the benefits to humankind.

Let's first start with the biggest question - Why?

Why why why

Go watch this great video on the Golden circle:

In the video, Simon Sinek tells us to start with the "Why?" It is a compelling argument, driven not by psychology he says, but biology. Starting with "Why" taps into the limbic part of the brain, and is said to overcome our modern neocortex.

It is not just a business thing though, I went to a workshop on producing an effective PhD thesis two weeks ago. In it, one of the points made was to put all the good bits in the beginning of the sentence, at the start of the abstract, the first few chapters of your thesis! Don't leave all the juicy parts to the end of the text, people want to know why your work is original.

It is almost a running joke in academia that you just need to skim the abstract and conclusions of a paper to get at the core idea being put forward. Really though, you don't want to lose a reader by putting all your effort in the conclusion and writing your abstract at the last minute. Capture the audience, put your best foot forward - in the beginning!

Besides business and writing, you also apply this line of thought to other fields! In the personal finance world I sometimes blog about, you are often told to come up with a goal first. Yes, set a goal before coming up with a budget or finding a financial advisor!

Not just any goal though, you need to think hard about Why you want to reach that goal. The How you do it is simply maths and grit, and the What is the reason behind the dopamine rush when you reach that goal.

Did you know that there's even a formalized way to write it out? I discovered this as I started creating unit tests and fell down a rabbit hole to...

How - With a language called Gherkin

Gherkin - a pickled cucumber. It is typically used for getting people on the same page. Well, the Gherkin language rather, not the cucumber.

As a domain specific language (translated to over 70 languages), Gherkin is used for writing behaviour scenarios in '.feature' files with an intuitive syntax. These '.feature' files are codified templates that can be understood by non-technical people, and used by technical people to guide their coding in Behaviour-driven development. See also BDD101(Python-orientated).

This is an example I came up with this morning for my current project:

    # language: en
    Feature: High Resolution Bed Elevation Model of Antarctica
      In order to have a better map of Antarctica's bed
      As a glaciologist,
      We want a model that learns from many open datasets  

      Scenario: See high resolution bed
        Given low and high resolution images of a bed
         When the user selects some view of Antarctica
         Then a high resolution bed elevation map is returned

The brilliant thing about this feature file is that it is high-level and spells out what needs to be done!

The 'Feature:' is basically the title. Under it is context. You will see that I've used something akin to the Golden Circle by starting with the Why. Unlike the Golden Circle though, I put in the Who next, a hint at the importance of user-centric design. Finally the What is spelt out, which is either a model, an app, a fancy function, or ... you get the idea.

The 'Scenario:' lays out How the feature works. Under it is a Given-When-Then statement, a form of Specification by example. One way I think of it is as an input-processing-output pipeline. Given some body of data or knowledge, When you are faced with something (the app has to act on this), Then an output is returned.

Take this as an opinionated starting template:

    # language: en
    Feature: <title>
      In order to ... <Why>
      As a ... <Who>
      I/We want ... <What>

      Scenario: <example>
        Given ... <input>
         When ... <something happens>
         Then ... <output>

If you want more examples, I find that the Python behave tutorial is one of the funnier ones, with ninjas! It goes to show how people can use this in so many different ways besides its original use in Behavioural-Driven Development. Why not make a few cards for a new habit you want to take up?

    # language: en
    Feature: Healthy Eating
      In order to live longer
      As a healthy person,
      I want to eat a balanced diet

      Scenario: Eating fruit
        Given a fruit bowl full of juicy fruit
         When the healthy person hasn't been any fruit eaten today
         Then there should be less fruit in the fruit bowl
          And some fruit in the healthy person's tummy

Alright, I am sort of (ab)using gherkin for purposes it wasn't designed for, and maybe breaking some standards to boot! Go and read how to write good gherkin with proper grammar (e.g. the link recommends writing in third person instead of first person). Or maybe follow some good gherkin examples.

We've now ran through a method of formally writing down something we want to do using Gherkin language. So now what?

Did I hear "what's in it for me" you say?

What's in it for me

It's easy to for anyone to argue on how to do things (implementation). But it is much harder to argue about the why (principles).

By starting with the core Why, you deliberately make things as vulnerable/awkward as you possible could. If it's a strong why, it stays. If it's not good, people realize it immediately and you can correct the most fundamental problem before it's buried in layers and layers of subjective matter.

The Why should be made as clear as possible. Over time, different people will have different interpretations on What it means and How we can use the message laid out by the why. The use cases will change, technology will change, but a good Why should withstand the test of time.

For example, I went to a workshop on managing documents and data last Wednesday.

They started off with the Why of backing up - so that you don't lose your work in the event of some unexpected event like an earthquake. This is something I've covered in my geographical diversification post.

The How is the implementation. Here already, we differ on opinion. The workshop people recommended a particular cloud storage provider that is quite 'reliable', whereas I prefer something like git which is not tied to a provider.

The What is the benefits of using a proper backup solution, things like peace of mind, ensuring your work lives on, etc etc.

As I write this paragraph, the place where I publish my work (Github) is experiencing some login issues and people are fretting out about potential data loss. Yes it's annoying but I'm not too fussed because I've got my continuity plans in place. With two local copies of my work, I am able sync it to any other git provider if necessary due to git's fungible nature.

If I used what the people at the university recommended, and that provider were to fail, or if I graduate and get kicked out of the university system... Yes I might still have my two local copies, but I'd have lost my version control, and it would take lots of time and effort to find another backup provider that does the trick. During that time, who knows what might happen to your personal computer?! With fungible git, I can simply add a new remote and push it up in a minute like nothing has happened.

By starting with the "Why", you get to know the fundamental reason. "How" things are done can change, "Who" your users are can change, "What" they think about your idea/product can change, but if you have a compelling "Why", those differences won't matter.

Maybe ten years down the road, there will be an even better solution, a better How to do things. Yet the Why of having a peace of mind with backups won't really change. It might even be the case that What the new solution offers is even more comprehensive, more foolproof, something that would seem like magic to us today.

So who stands to benefit from this? Is this just some propaganda to change all of our language into a strict monotonous template of Gherkin-like syntax?

Who cares

Hackers

I'll be honest with you, I'm a guy who likes a good howto article that gets into the nuts and bolts. A blog post might have a How-What-Why structure and I won't bat an eye. Like, just tell me how it works already!! Perhaps the developers and engineers reading this will feel the same.

You could be a great engineer, or developer, or some hacker that brings a vision to life. There is however, a possibility of over-engineering something. You could go from bug-driven development to test-driven development to behavioural-driven development, and so on, but was all of that necessary? Could you have designed something much more simpler and easier to maintain that covers 80% of the use cases?

Testers

Strangely enough, the Gherkin syntax was borne from Behaviour Driven Development (BDD), itself an extension of Test Driven Development (TDD). Testers are akin to users, they are effectively the ones using the product, discovering What it does.

You could be so user-centric, focused on your customer, at the expense of losing your sanity. Sure, the customer is always right, but that doesn't mean the customer can call all the shots and break the law to get what they want! I've worked in the service industry, and you do realize that there are some privileged customers who can feel so entitled as to wreak havoc on your day. They might not be satisfied with what you offer, and you'll need to accept that.

Visionaries

Doesn't this Why-first syntax seem like some push from business orientated folk with an agenda? Why should we believe in their vision? Sure, come up with a vision and have us all adhere to that, building your empire and using your goods.

For the businessman, or teacher, or whatever inspirational leader out there. Sure, maybe it is too much of an ask for you to write in (good) Gherkin. It's all too easy to send out a tweet or have a witty one-liner quote sent out, while your minions scramble to work on it. I don't know what to say really. Perhaps be careful of the ideals you are bestowing on the world, because taking it to the extreme can be dangerous.

Hustle on!!!

It's easy to specialize into your own silos, I know. Although you get some great people who can cover all three, or maybe two out of the three questions (Why-How-What) in detail, it's a rarity.

As an individual, I do find it tricky to balance all three.

Coming back to my PhD... Some days or weeks I would just get so engrossed in coding, implementing cool bleeding edge stuff that few people on the planet have done before. When it comes to writing, it can be scary to go deep into the "Why?" question, because it means a lot of soul searching. Further down the track, I'll need to explain myself too, what I've done over my PhD, what it means for our world!

As I mentioned at the start, it's all about balance.

To strike that balance, you need communication. Be it communication between your team members, or communication with the different parts of your brain that does different things, we all need to figure out how to join the dots.

At the end of the day, you need to get to the same page. Start with the Why. Hopefully you find the gherkin spec helpful in formalizing your thoughts. As for what comes next, that's up to you to discover!

Get going!