DEV Community: Deborah D

Gatsby Up & Running: Creating a CD Pipeline

Deborah D — Sat, 26 Sep 2020 18:36:02 +0000

My last adventure was migrating my crufty Jekyll site to Gatsby. I used to deploy my Jekyll site manually by building it locally and pushing the generated static content to the gh-pages branch on GitHub. With my new site in place, I decided it was time to leave my old ways behind and set up an automated pipeline to deploy my site to GitHub pages.

Why GitHub Pages?

I've always used GitHub Pages to host my blog because it's lightweight and integrates seamlessly with GitHub. It is simple, free, and provides free goodies like enforcing HTTPS by default and also supports adding a custom domain.

Deploying Locally

Before automating the deployment of my new Gatsby site, I learned how to deploy it locally. To do this, I added an entry to the scripts tag in package.json which creates a production-ready build for my Gatsby site.

"scripts": {
  ...
  "build": "gatsby build",
}

npm run build

builds the site and places the generated assets in a folder called public. Copying this public folder to any HTTP server would deploy my site to that server.

I started an HTTP server using http-server to serve the assets in the public folder which effectively deployed my site locally.

➜  cd public
➜  http-server
Starting up http-server, serving ./
Available on:
  http://127.0.0.1:8082
  http://10.10.10.10:8082
  http://192.168.13.12:8082

Deploying Manually to Github Pages

Setting up Github Pages

I already had GitHub pages set up since I used it for my Jekyll site but I will go over the steps here for completeness.

I headed to my GitHub repository's settings page and scrolled down to the GitHub Pages section.

I did not want the files generated by the Gatsby build process to clutter up my master branch so I chose the gh-pages branch as the source branch for my GitHub Pages site. Any static site I pushed to this branch would then be published at deborah-digges.github.io. I also liked the fact that Enforce HTTPS was enabled by default. All the security!

Deploying Manually

I deployed the site manually to GitHub Pages by building the site locally and pushing only the public folder to the gh-pages branch on GitHub.

This required a rather awkward sequence of commands.

git checkout -b gh-pages

gatsby build

# Temporarily move the public folder
mv public /tmp

# Remove all other files
rm -r *

# Move the contents of the public folder back to the root of the directory
cp -r  /tmp/public/* .

git commit -m "Release new version of blog"

git push origin gh-pages

That's a mouthful and it's easy to go and accidentally nuke your whole computer with commands like rm -r *. It's a good thing the gh-pages package exists to save us from ourselves.

I added the dependency to my project to give it a spin.

yarn add gh-pages

To let gh-pages know where my repository was located, I added the homepage property in package.json.

{
  ...
  "homepage": "deborah-digges.github.io",
  ...
}

I added another script deploy to my package.json which does a build and then pushes the public folder to the gh-pages branch of my repository on Github.

"scripts": {
  "build": "gatsby build",
  "deploy": "npm run build && gh-pages -d public",
}

And voila! I was able to deploy my site manually from my computer using the npm run deploy script. My site was then up and running at deborah-digges.github.io

✨Continuously✨ Deploying with Travis CI

Great! I deployed my Gatsby site! However, I wouldn't want to be deploying a site manually from my laptop on a Friday afternoon.

I decided to continously deploy my site so that every new commit would automatically be deployed to my GitHub Pages site. This was exciting, but I had bypassed an important step which is having automated tests for my site to ensure that a bad commit did not bring down my entire blog. However, I decided to live dangerously and keep the testing of my Gatsby site for a future blog post.

Signing Up

I signed up on the Travis CI website with my GitHub account and consented to sharing my GitHub data with Travis.

Enabling the Repository

I then headed to the repositories page and enabled builds for the deborah-digges.github.io repository.

Adding the travis.yml file

I added a travis.yml file to the root of my repository to tell Travis what to do on every commit to master.

language: node_js
before_script:
  - npm install -g gatsby-cli
node_js:
  - "10"
script: git config --global user.email $GH_EMAIL 2> /dev/null &&
  git config --global user.name $GH_USERNAME 2> /dev/null &&
  git remote set-url origin "https://${GH_USERNAME}:${GH_TOKEN}@github.com/deborah-digges/deborah-digges.github.io.git" 2> /dev/null &&
  yarn install && yarn run deploy 2> /dev/null

The script runs the yarn run deploy step that I previously used to deploy my site locally. It is doing some extra steps to give Travis CI the right access to push to my GitHub repository.

It tells the git client installed on Travis CI who I am.

git config --global user.name $GH_USERNAME
git config --global user.email $GH_EMAIL

To provide the script push access to my GitHub repository, I embedded my Github Token environment variable in the remote URL.

git remote set-url origin "https://${GH_USERNAME}:${GH_TOKEN}@github.com/deborah-digges/

Where did these environment variables come from?

Configuring Travis Environment Variables

I headed to my repository settings and made the following environment variables available to my script.

It took me a few attempts to get this right, but I am proud to say that my site is now being continously deployed to GitHub Pages on every commit to the master branch of my repository.

This was a lot of work, and in my quest to search for a simpler solution, I will explore using a GitHub Action to continously deploy my site. Stay tuned for more dangerous living!

From Jekyll to Gatsby: 7 Simple Steps

Deborah D — Thu, 17 Sep 2020 03:03:01 +0000

Six years ago, during my final year in college, I was enrolled in a course on Open Source technology. To pass the course was to do two things: contribute to an open source project, and blog about it. Laughably, the only thing that propelled me to start writing publicly was the fear of failing the class.

I passed the course and managed to get some semblance of a blog together. However, I didn't put too much effort into building the site itself since much of my effort was expended in writing the content and trying to creep my way into open source. At the time, Jekyll was all the rage and I went with a themed Jekyll starter called Lanyon.

It served me well over the years, but my unfamiliarity with how it worked kept me from making any substantial changes to the structure of the site. I've finally taken the stand to move to a stack I'm more comfortable with.

Why Gatsby?

I considered refurbishing my blog with Gatsby and Next. I decided to go with Gatsby since it is a little more specialized towards static site generation.

Let's Get Migrating

Scaffolding

Getting a Gatsby project up and running is well-documented on the Gatsby site. Enhancing the project to generate a page from a markdown file is also clearly explained.
These are the steps I followed to get the initial project set up for the blog:

Used the gatsby-source-filesystem plugin to read markdown files into Gatsby. I added an entry to the plugins key in gatsby.config.js.

plugins: [{
   resolve: `gatsby-source-filesystem`,
   options: {
    name: `markdown-pages`,
    path: `${__dirname}/src/markdown-pages`
  }
}]

Parsed markdown files using the gatsby-transformer-remark plugin which extracted the frontmatter as data and the content as HTML.

plugins: [
  {
    resolve: `gatsby-source-filesystem`,
    options: {
      name: `markdown-pages`,
      path: `${__dirname}/src/markdown-pages`
    },
  },
  `gatsby-transformer-remark`
]

Created a template for blog posts using React to render a blog post's data as a React component.
Generated pages for all markdown files using the createPage Gatsby API.
Created an index page for all blog articles that displayed a list of links.

Migrating Content

I copied all the markdown files from my old website to the folder indicated in the gatsby-source-filesystem config entry: markdown-pages. I was now able to see all my old blogs being listed on the index page, but most of the posts themselves looked terribly broken.

Fixing All the Things

Links

To avoid breaking links, I used the same slug format that my old Jekyll site used: YYYY/MM/DD/Title. In my case, the slug was derived from the filename instead of the frontmatter, which required a little special handling.

To access the filename, I used createFilePath.
I extracted the slug and the date from the filename and passed them along as node fields, available to subsequent page queries.

const result = extractMetadataFromFilename(filePath);
date = result.date;
slug = result.slug;
createNodeField({ node, name: `date`, value: date });

Highlighting

In my old Jekyll site, I was using pygments for syntax highlighting of code snippets, which used syntax like this to create a highlight:

{% highlight C++ %}
{% raw %}
  void f(int x, int n)
  {
    Gadget * g = new Gadget{n}; // Look ! I’m a Java Programmer :)
    if(x < 100) throw std::runtime_error{"weird"}; //leak
    if(x < 200) return; //leak
    delete p;
  }
{% endraw %}
{% endhighlight %}

Since this no longer worked in Gatsby, I changed all highlights to be enclosed in three back-quotes which translates to a <pre> tag enclosing a <code> blog.

I would like to eventually embed gists directly into my posts, but that's an improvement for a later blog post.

Images

First, I copied over the images from my old site to src/images. To make Gatsby aware of where my images were located, I added another plugin entry for the gatsby-source-filesystem plugin.

{
  resolve: `gatsby-source-filesystem`,
  options: {
    name: `images`,
    path: `${__dirname}/src/images`,
  },
}

I then modified the plugin entry for gatsby-trasformer-remark to add the gatsby-remark-images plugin to it.

{
  resolve: `gatsby-transformer-remark`,
  options: {
    plugins: [
      {
        resolve: `gatsby-remark-images`,
        options: {
          maxWidth: 800,
        },
      },
    ],
  },
}

With this configuration, I was able to embed images inline using the familiar markdown format:

![Architecture](../images/NN.jpg)

It's important to use a relative path to the images folder.

- src
  - markdown-pages
  - images

For the directory structure above, ../images/image.png would be the path to an image referenced in a markdown file.

Gifs

gatsby-remark-images cannot handle gifs. I added the gatsby-remark-copy-linked-files plugin to properly handle any gifs that gatsby encountered.

{
  resolve: `gatsby-transformer-remark`,
  options: {
    plugins: [
      {
        resolve: `gatsby-remark-images`,
        options: {
          maxWidth: 800,
        },
      },
      `gatsby-remark-copy-linked-files`
    ],
  },
}

Favicon

Again, I invoked the mighty plugin system of Gatsby and added another plugin entry in gatsby-config.js.

{
  resolve: `gatsby-plugin-favicon`,
  options: {
    logo: `./src/favicon.ico`
  }
}

Styling

I lazily copied over the poole/lanyon theme to avoid me having to write any CSS. Let's not kid ourselves, I don't really know how to design a site.

Since these are global CSS styles, I needed to import them into the gatsby-browser.config.js file to make them apply to the generated HTML files.

import './src/styles/poole.css';
import './src/styles/lanyon.css';
import './src/styles/Calendas_Plus.otf';

Deploying

To test the site out and compare it with my old site, I deployed the site to Netlify. It was about as simple as going to the Netlify site, authorizing access to my GitHub repo, and providing a build command.

Testing that No Links are Broken

With my site deployed to Netlify, I tested that no links would be broken by ensuring that each link on the old site also existed on the new site:

links.forEach(link => {
  link = link.replace('deborah-digges.github.io', 'epic-mirzakhani-8e39a6.netlify.app');
  request(link, (err, res, body) => {
    if (err) { console.log('Page fetch failed', err); }
    console.log(link, res.statusCode);
  });
});

I ran the following script on each of the two pages of my old blog to extract all the links:

let links = document.querySelectorAll('a');
let siteLinks = []
for (let i=0; i < links.length; i++) {
  let linkText = links[i].textContent;
  linkText = linkText.replace(/\s+/g, ' ').trim();
  const link = links[i].href;
  siteLinks.push([linkText, link]);
}

for(let i=0; i < siteLinks.length; i++) {
  console.log(siteLinks[i][1]);
}

Admittedly, it's a little rudimentary, but hey! it did the job.

Switching over

Once I was ready to bid adieu to the shackles of my Jekyll site, I overwrote my Github User Pages repository with the shiny new Gatsby site. I will be going over the details of deploying a Gatsby site to GitHub Pages in a subsequent blog post!

Next Up

Now that I'm using a system I understand a little better, I want to soon:

Implement pagination
Create categories for pages and allow browsing the blog by category
Create a navigation element called Content in the header which expands to Blogs, Drawings, Book Reviews
Create a Reading List navigation element
Lean enough about CSS and design to create a better landing page (Maybe hire a designer?)

If you'd like to check it out in more detail, you can find the source code here.

Tune in again for more exciting updates to my Gatsby blog!