DEV Community: Niklas

Deno and GitHub Actions workflows

Niklas — Sat, 09 Mar 2024 10:12:15 +0000

Today I want to talk about Deno and GitHub Actions. For the last year or so, I have only built new projects with Deno. I really like the runtime, it's typescript out of the box config, formatting, linting etc. It is just a lot of fun to work with.

Since I still wanted to make sure that everything works when pushed to our Github repositories, I had to build some GitHub actions. Today I want to share a repository where I collect useful actions. At the time of writing, the repository contains 3 different workflows. You can find the repository at niklasmtj/deno-actions on GitHub.

CI

The CI workflow runs on every PR creation or commit. It checks the format, runs tests, and lints the whole project. Since these steps are usually really fast, I did not limit it to just the changed files.

name: "CI"

on:
  - pull_request

jobs:
  ci:
    runs-on: ubuntu-latest
    steps:
      - name: "Checkout"
        uses: actions/checkout@v4

      - name: "Setup Deno"
        uses: denoland/setup-deno@v1
        with:
          deno-version: "1.x"

      - name: "Run Tests"
        run: deno task test

      - name: "Run Lint"
        run: deno task lint

      - name: "Run Format"
        run: deno task format

To run this workflow you need to set up 3 tasks in the deno.json file. First set up test with the required permissions. Then lint and format. This is done in deno.json under the tasks key as you can see in the following block of code:

{
  "tasks": {
    "lint": "deno lint",
    "format": "deno fmt --check",
    "test": "deno test --allow-read --allow-write --allow-net",
  }
}

This example uses the `--allow-read --allow-write --allow-net' flags to run the tests. When setting up your workflow tasks, you can easily check the commands locally. Deno will help you figure out the permission flags you need.

Build Docker Container Images

The second action will build a container image using Docker. The workflow runs when new pushes are made to the `main' branch. This happens most often after merging a pull request.

Make sure that you have a Dockerfile describing your desired container image in the root directory of the repository. This is used by the build-and-push action as it's image recipe.

It is set up using buildx, which allows you to build multi-architecture images. For example, if you are building your containers locally on an M1 Mac, these are arm64 images. If your hosting provider does not support arm64 images, but rather uses Intel or AMD processors on their servers, you will need to build amd64 images. Using buildx and qemu we can make sure we're able to build both in the same build. This is why you see linux/amd64,linux/arm64 in the platforms fields in the following workflow file:

name: "Build Container Image"

on:
  push:
    branches:
      - main

jobs:
  ci:
    runs-on: ubuntu-latest
    steps:
      - name: "Checkout"
        uses: actions/checkout@v4

      - name: Set up QEMU
        uses: docker/setup-qemu-action@v3
        with:
          platforms: linux/amd64,linux/arm64

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
        with:
          platforms: linux/amd64,linux/arm64

      # Login to Docker Hub or any other registry here
      - name: Login to Docker Hub
        uses: docker/login-action@v3
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}

      - name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: niklasmtj/deno-image:latest
          cache-from: type=gha
          cache-to: type=gha,mode=max

The built image here will then be pushed to Docker Hub after login. Make sure your organization or repository already has the DOCKERHUB_USERNAME and DOCKERHUB_TOKEN secrets set. These are set in a repository under `Settings -> Secrets and Variables -> Actions -> Repository secrets'. How to obtain a Docker Hub token can be found in the Docker documentation.

Caching is also set up when the container image is built to ensure that the subsequent execution is as fast as possible.

Create a Test Coverage Website and Publish to GitHub Pages

Testing should be an integral part of application development. To make sure that every part of the application is covered with tests, Deno supports creating a coverage report right out of the box (🤝). The third action supports building just that. Create the coverage and push it to GitHub pages. To be able to publish, you need to make sure that GitHub Pages is enabled for the repository you want to build coverage for.

`yaml
name: Code Coverage

on:
push:
branches: [main]
workflow_dispatch:

permissions:
contents: read
pages: write
id-token: write

jobs:
ci:
runs-on: ubuntu-latest

steps:
  - name: Setup repo
    uses: actions/checkout@v4

  - name: Setup Deno
    uses: denoland/setup-deno@v1

  - name: Create lcov file
    run: deno task coverage

  - name: Setup Pages
    uses: actions/configure-pages@v4

  - name: Upload artifact
    uses: actions/upload-pages-artifact@v3
    with:
      path: './coverage/html'

  - name: Deploy to GitHub Pages
    id: deployment
    uses: actions/deploy-pages@v4

One thing you need to make sure is that GitHub Actions has permissions to publish the created website. Make sure you set it under the permissions key in the yaml file.

Conclusion

This repository is not set in stone. I will try to keep the dependencies up to date (shout out to Dependabot) and will add or change actions in the future. So make sure you keep an eye on the repo (click the watch button in the top right corner).

Thanks for reading and have a great day!

Conditional GitHub Actions job

Niklas — Tue, 14 Feb 2023 15:15:00 +0000

With GitHub Actions it is possible to trigger jobs under certain conditions. GitHub provides the if for this purpose, which can be called at job level. In the following you will see how this can be achieved.

Syntax

As mentioned above the syntax of the if can be found on the sub level of the job name.

jobs:
    example-job:
        if: github.ref_name == 'main'

Here it is possible to check for different conditions. GitHub offers various variables in the ¹, which can also be combined with the ² provided.

The Ifs can follow two different formats. Both are equivalent to each other in the case.

if: github.ref_name == 'main'
if : {{if: github.ref_name == 'main'}}

Note: When using literals in expressions it is needed to enclose them using single quotes. Double quotes will fail ³

Examples

On push to main branch

The following GitHub Action will return "This is the main branch." only when the push is done to the named branch. Otherwise it will be skipped and the action will not start.

name: check-main-branch
on: 
  push:
  workflow_dispatch:

jobs:
  check-main-branch:
    if: github.ref_name == 'main'
    runs-on: ubuntu-latest
    steps:
      - run: echo "This is the main branch."

Output

On Release

This GitHub Action will only run when a release was published which has the tag name including -beta. When the tag is named different it is not triggered.

Requirement: Use of semantic versioning⁴. Example tag: 1.0.0-beta

name: check-beta-tag
on: 
  release: 
    types: [published]

jobs:
  beta-deployment:
    if: contains(github.ref_name, '-beta')
    runs-on: ubuntu-latest
    steps:
      - run: echo "This is the beta tag of $GITHUB_REF_NAME"

Note: $GITHUB_REF_NAME is part of the exposed environment variables ⁵ of the runner.

Output

Following is the output of the actions which are triggered after a release. In the first picture you can also see that the main-branch action is triggered but skipped, since the tag name is not main.

Additional Resources

Use GitHub Container Registry (GHCR) to host your Helm Charts

Niklas — Sun, 08 May 2022 20:14:29 +0000

TLDR: Full commands can be found at the end of the post.

I recently started to check out Helm and thought about the combination of hosting the charts on GitHub Container Registry (GHCR) since the charts follow the regular OCI (Open Container Image) standard which is also used by Docker container images. Because of that, I tried to understand the steps necessary to host my charts on GHCR since public charts / containers are free of charge. The following post will describe the necessary steps that are required. So let’s dive right in.

Helm

First of all, make sure that you have Helm installed in a version >3.8 since OCI support before that is experimental. It is possible to use the feature before 3.8 with the environmental variable HELM_EXPERIMENTAL_OCI=1 set.
If you don’t have Helm installed yet you can do so via brew install helm on macOS. For different operating systems please check Helm’s docs for more installation options ¹.

For presentation purposes, I will use the default chart created by Helm which is the result of running helm create <chart-name>. For this post, I will call it helm create example-chart. The content of the created directory is not the subject of this post but more information can be found in the Helm docs ².

To package it all up and make it ready for pushing to GHCR Helm provides the helm package command which takes the argument of the directory to package. In the chart’s parent directory. So using helm package example-chart will create a new package with the name of example-chart-0.1.0.tgz. The version number can be changed in example-chart/Chart.yaml under the version attribute.

GitHub Container Registry Preparation

To push to GitHub Container Registry we need to authenticate with a Personal Access Token. If you never created one you can follow the steps described here ³. Make sure that your Token has the following permissions:

read:packages
write:packages
delete:packages

You can eather just export your Access Token for the terminal session and use export GHCR_PAT=ghp_... with your Token after the = sign or use your .zshrc or .bashrc. To do so, edit your file with e.g. nano .zshrc and add the export line in your file. The one you need depends on your current default shell. To see which one is your default use echo $SHELL in your terminal.
After saving and closing the editor via ctrl + x and confirming with y you have to reload your .zshrc with source .zshrc. After that, the Token should be available for use.

Logging in

To check if things are working use echo $GHCR_PAT | docker login ghcr.io -u <GITHUB-USERNAME> --password-stdin with your username following the -u flag.
If everything is configured correctly you should get the answer of Login Succeeded.

Push to GHCR

Pushing now is easy. Get the current chart version and use Helm to push to GHCR:

export CHART_VERSION=$(grep 'version:' ./example-chart/Chart.yaml | tail -n1 | awk '{ print $2}')
helm push example-chart-${CHART_VERSION}.tgz oci://ghcr.io/niklasmtj

You should get something similar like the following:

Pushed: ghcr.io/niklasmtj/example-chart:0.1.0
Digest: sha256:c13e9bc40b48460a7b3af6a5df78b4faeff6af7d0688333124884117478be18c

Now your package is pushed and you can check it out via your profile under the tab packages. For example, this example chart can be found under https://github.com/users/niklasmtj/packages/container/package/example-chart.

This is how you can host your Helm charts on GitHub’s Container Registry.

Thank you for reading,
Niklas

TLDR commands

helm create <chart-name>
helm package <chart-name>
echo $GHCR_PAT | docker login ghcr.io -u <GITHUB-USERNAME> --password-stdin
export CHART_VERSION=$(grep 'version:' ./path/to/Chart.yaml | tail -n1 | awk '{ print $2 }')
helm push <chart-name>-${CHART_VERSION}.tgz oci://ghcr.io/<GITHUB-USERNAME>

How to install Weave's Ignite for Firecracker VMs with simple script

Niklas — Sun, 20 Feb 2022 15:29:23 +0000

Since I want to get more into Firecracker MicroVMs I started playing around with Weave’s Ignite which gives a familiar interface to docker to interact with the VMs. I do this with DigitalOcean’s droplets (Affiliate link, get $100 in credits for 60 days for free) since they have KVM enabled and are pretty inexpensive. This script will also work on the $5/month Droplets. Since I set up a new droplet every time to save costs when testing Ignite out I wanted to keep it simple and "automate" the installation with a quick bash script. The steps are taken from the installation page from the Ignite docs).

First, you have to create the script on your VPS which can be done by opening it with an editor of choice. On a Digital Ocean droplet, you can use for example vim. To create the bash script file type vim install-ignite.sh in your terminal window.

The script

Now we go over to the more interesting part of the post. The script itself. It will install all the dependencies of Ignite (as of February 2022) and install Ignite afterwards. After the installation, it will check if ignite was installed successfully by checking the currently installed version. So now go forward and copy the following script in your open editor.

#! /usr/bin/bash

# Update apt-get repository and install dependencies
apt-get update && apt-get install -y --no-install-recommends dmsetup openssh-client git binutils

# Install containerd if it's not present -- prevents breaking docker-ce installations
which containerd || apt-get install -y --no-install-recommends containerd

# Installing CNI
# Current version from https://github.com/containernetworking/cni/releases
export CNI_VERSION=v1.0.1
ARCH=$([ "$(uname -m)" = "x86_64" ] && echo amd64 || echo arm64)
export ARCH
sudo mkdir -p /opt/cni/bin
curl -sSL "https://github.com/containernetworking/plugins/releases/download/${CNI_VERSION}/cni-plugins-linux-${ARCH}-${CNI_VERSION}.tgz" | sudo tar -xz -C /opt/cni/bin

# Installing Ignite
# Get the current version from https://github.com/weaveworks/ignite/releases
export VERSION=v0.10.0
GOARCH=$(go env GOARCH 2>/dev/null || echo "amd64")
export GOARCH

for binary in ignite ignited; do
    echo "Installing ${binary}..."
    curl -sfLo ${binary} "https://github.com/weaveworks/ignite/releases/download/${VERSION}/${binary}-${GOARCH}"
    chmod +x ${binary}
    sudo mv ${binary} /usr/local/bin
done

# Check if the installation was successful
ignite version

Make it executable

To use the script now we have to make it executable. This is easily done with chmod +x install-ignite.sh. Now we can start the script with ./install-ignite.sh from the directory you created the script in.

Have fun with Weave’s Ignite

The CLI feels very familiar with everyone who knows about docker commands. Starting a new VM can be done with ignite run weaveworks/ignite-ubuntu. From there have fun playing around with Ignite.

That's it for now but you can be sure that there will be more posts about Firecracker MicroVMs and how to use them in the future.

Thank you for reading,
Niklas

An alternative Docker installation with Multipass on macOS without using Docker for Mac

Niklas — Sat, 29 Jan 2022 18:44:56 +0000

Last week I received an email from the Docker Team which said that Docker for Mac (the software which also comes with a GUI) will be forbidden for commercial use when the company has more than 250 employees AND makes more than $10 million per year. To use it commercially the company has to get licenses for every developer using it, starting at $5/month. This made me think what an alternative could be for devs that don’t want to use Docker for Mac anymore, since I read a lot of posts that many devs don’t even need it. Most of them interact via CLI anyway. I stumbled across a nice article from Josh Gorneau where he uses multipass to host his Docker VM. In this case, it is a Ubuntu 20.04 installation. So a couple of commands will be similar to Josh’s article such as the VM configuration used in this post. So let’s start with setting up multipass.

Install Multipass

Multipass is a project made available by Canonical¹ also develops and publish the Ubuntu² Linux distribution. So install it either from the website or via homebrew.
If you don’t have the Docker CLI yet, install it too

brew install multipass docker

Creating an SSH Key

Since we will connect to our VM via SSH we need a new SSH key that will be imported while setting up the VM. To do so we can use ssh-keygen like in the following. I will create a new one since I use just one for every service I connect to. Follow the prompts to generate a key. Mine will be called docker-multipass. Since macOS and Ubuntu are pretty similar in creating SSH keys you can also follow the instructions over at DigitalOcean.

➜  ~ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/Users/user/.ssh/id_rsa): /Users/user/.ssh/docker-multipass
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /Users/user/.ssh/docker-multipass.
Your public key has been saved in /Users/user/.ssh/docker-multipass.pub.
The key fingerprint is:
SHA256:LjhUL0bZ8lXf9iIrOPF1EXjAJguNu3YbvUVH2tUHcak user@domain
The key's randomart image is:
+---[RSA 3072]----+
|         o .oo+o+|
|       oo o.+..==|
|      = .o.+ .++=|
|     o +...  Eoo+|
|    . o S. .o.o..|
|   . o oo+o..+.. |
|    o ..+.oo.o   |
|     . . ....    |
|                 |
+----[SHA256]-----+

To get the generated public key which will be imported to the Ubuntu VM on the setup you can use cat /Users/user/.ssh/docker-multipass.pub. Copy the output since you will need it in the next step.

The VM Config

In the following, we will create a config that will be used to set up the VM. This is done with a cloud-init configuration. Cloud-init³ is a configuration tool from Ubuntu that helps set up virtual machines. More Information about it can be found here: Using cloud-init with Multipass | Ubuntu. The config is taken from Josh’s article linked above.

Create a docker.yaml file in your current directory and paste in the following configuration:

---
users:
  - name: ubuntu
    sudo: ALL=(ALL) NOPASSWD:ALL
    ssh-authorized-keys:
      - ssh-rsa AAAAB3...

package_update: true
packages:
  - docker.io
  - avahi-daemon
  - apt-transport-https
  - ca-certificates
  - curl
  - gnupg
  - lsb-release
runcmd:
  - sudo curl -fsSL https://get.docker.com | sudo bash
  - sudo systemctl enable docker
  - sudo systemctl enable -s HUP ssh
  - sudo groupadd docker
  - sudo usermod -aG docker ubuntu

The configuration sections explained

users is an Array that defines what users should be set up while creating the VM. Here you will also paste your SSH key created in the last step.
package_update tells the Ubuntu VM to run apt-get update before installing any packages in the following step to make sure the newest packages are available for installation
packages as the name suggests tells Ubuntu which packages to install to proceed
runcmd will be run after installing the packages before. In this case, it will download Docker, enabling it via systemctl, creating the docker group and afterwards adding our newly created user to this group. This will make sure that our ubuntu is allowed to interact with Docker.

Launching the Ubuntu VM

multipass launch -c 4 -m 8G -d 20G -n docker 20.04 --cloud-init docker.yaml

-c this specifies how many CPUs the VM can use. In this case 4
-m defines the amount of RAM
-d is for setting the disk space used by the VM
-n sets the VM’s name

Handling possible permission problems with a non-root user

launch failed: multipass socket access denied
Please check that you have read/write permissions to '/var/run/multipass_socket'

When you use a macOS user which is not a local admin like I do you have to give this user the permissions to read/write to multipass’s socket to be able to function. So use the chmod command to give your user the permissions needed. Otherwise, you will not be able to start multipass instances.

sudo chmod a+rw /var/run/multipass_socket

After that, the socket should be usable for the non-root user.

Log into VM

After launching the multipass Ubuntu VM in the last step check if it is possible to ssh into the newly created VM.

ssh ubuntu@docker.local

Permission denied (publickey)

When the error ubuntu@docker.local: Permission denied (publickey). will be thrown check if you provide the right ssh key when connecting. Without specifying a key ssh will use your main ssh key which is most often id_rsa. To use the key created above add an entry to your .ssh/config like below.

Host docker.local
  HostName docker.local
  User ubuntu
  IdentityFile ~/.ssh/docker-multipass

If you can connect to your newly created VM close the connection with exit and proceed to the next step where we will now use Docker from our host machine to interact with the Docker installed in the virtual machine.

Time to use Docker

Now that everything is set up check if Docker is answering. Since we use a custom Docker host we have to provide the cli where we want to execute the commands. This is where DOCKER_HOST comes into play, so use it like in the example below.

DOCKER_HOST="ssh://ubuntu@docker.local" docker version

Save the Docker Host environment variable in your terminal session

Since always putting the DOCKER_HOST variable in front of the commands can be annoying you can export it in your terminal session or save it in your .bashrc or .zshrc to have it in every terminal session in the future.

export DOCKER_HOST="ssh://ubuntu@docker.local"

Have fun using Docker

When everything is set up and your Docker is usable from within your terminal you can now start any container you like.

docker run -p 9898:9898 stefanprodan/podinfo

Exposing Ports to the Host machine

Right now (January 22) multipass does not automatically port-forward the exposed ports from within the VM. To set this up for a port use ssh with the -L argument which will block the port on your host machine which is exposed on the connected machine.

ssh -L <Host-Port>:localhost:<VM-Port> ubuntu@docker.local

I hope this article helped you set up and use Docker without using the Docker for Mac software.

Thank you for reading,
Niklas

Resources

What to do when macOS keyboard writes wrong special characters

Niklas — Mon, 10 Jan 2022 20:31:19 +0000

At the moment I always have an external Keyboard connected to my MacBooks over a USB-C Dongle when I am working. From time to time the keyboard layout switches after waking the laptop up again after a couple of minutes. For example, it starts to write ^ where normally the < is. This is my current solution to fix this issue.

Check Keyboard settings

First of all, check the keyboard settings in Settings -> Keyboard -> Input Sources if anything changed. Sometimes this can change since macOS has the default shortcut of ^(control) + space set. I already had the scenario that I miss pressed this shortcut and changed it to a different input source.

Delete the Keyboard preferences file

Since this is most often not the case and my input source is on DE which is my default I try to reset the keyboard settings. When connecting a keyboard for the first time macOS asks which key is next to the left shift key on the connected keyboard. To toggle this dialogue again one has to delete a plist file. This is where macOS stores its’ settings.

The keyboardtype.plist of the current user can be found in ~/Library/Preferences/com.apple.keyboardtype.plist so removing that is possible with the following command

rm ~/Library/Preferences/com.apple.keyboardtype.plist

If this file is not available in the home Library there should be one in the root /Library of macOS. This can be found in /Library/Preferences/com.apple.keyboardtype.plist and can be removed with the following:

sudo rm /Library/Preferences/com.apple.keyboardtype.plist

After removing the file(s) remove the keyboard from the macOS device and restart it. After restarting the computer log in to the desired user and plug the keyboard back in. macOS should then show you the wanted keyboard setup screen again. After setting up the keyboard again the special keys should work as expected again.

Thank you for reading,
Niklas

GitHub Actions workflows in combination with GitHub Container Registry Package Visibility

Niklas — Mon, 18 Oct 2021 19:04:39 +0000

Last week my task was to set up a container image that we wanted to use to test the GitHub Container Registry (GHCR). We wanted to see if we could lower our building times for one of our CI jobs when using GitHub’s registry to pull from. The following is a description of an error we encountered and how we got rid of it. Additionally, I will talk about the different visibility types of container images or packages how they are called at GitHub which were a hurdle we had to take with the registry.

The workflow.yaml file we used looked something like the following:

name: CI

on:
  - pull_request

jobs:
  continous-integration:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Log in to the Container registry
        uses: docker/login-action@v1
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
    - name: Pull Image from ghcr.io
      run: docker pull ghcr.io/USERNAME/MY_PRIVATE_IMAGE
    ...

First, we check out the repository and log into the GitHub Container Registry afterwards with our github.actor as the username and the GITHUB_TOKEN as the password. After a successful login, we’ll pull our container image from the registry. The steps coming afterwards are not subject to the error we encountered so these are left out in this article.

The Error

After triggering the workflow with a Pull Request the workflow quickly answered with an Error response from daemon: unauthorized when getting into the docker pull ghcr.io/USERNAME/MY_PRIVATE_IMAGE step. After searching for an answer in the GitHub docs and the internet we found out that our problem lays inside of the visibility type of the container image. I will go into further detail in the sections below.

Package Settings and Visibility

The problem happened with an image built and saved in a GitHub organization account. For regular packages in GHCR, there are two types available – public and private. There is also a third visibility type of internal available in the GitHub Enterprise Cloud plan. After creating and pushing a container image to GitHub Packages the default visibility type after creation is always set to private. This can be changed in the package settings. In the following the 3 options will be described and how they influence the ability to work with these packages in GitHub Actions.

Public

Changing the package’s visibility to public is the easiest option of these 3 to work with. After making the package public it is visible to anyone and it can be pulled anonymously via ghcr.io. This is good for public images of applications like web apps or web servers like nginx. While working with GitHub Actions this is also the easiest option of visibility since we don’t need to login to the registry first. So using a public repository is as straightforward as it gets. However, this was not a viable option for us.

Caution: When making a package public it cannot be made private again

Private

The visibility option of private is the default one when a container image is created in the first place. Accessibility is restricted only to people who have been given the rights from the package owner in the package’s settings. When given access it is possible to pull or with advanced rights write/update or delete the package entirely. To pull or change the image via CLI or API one has to authenticate with the container registry first. Authentication to pull, update or delete the container image happens via a Personal Access Token (PAT). How to do that is described in the GitHub documentation. ¹ Without authenticating first one will see the error response unauthorized from the beginning of this post. For GitHub Actions workflows GitHub recommends using the GITHUB_TOKEN instead which is available in the workflow.

Anyway using a private package in a GitHub Action workflow did not work even when the account that triggered the workflow and was used to authenticate with the container registry. The login to the registry even succeeded but the following pull step failed due to the unauthorized error shown above. Even after giving the repository’s GitHub Actions workflow access to the package ², it was not possible to pull the image.

Internal

The solution came from a GitHub community answer ³. The internal visibility type is the answer to our problem here. With this setting, the container image is visible within the whole organization or all the organizations in the enterprise. Every member will have at least read/pull access to the package. Write and Delete rights are then given via the package’s settings.
The GitHub Actions documentation is pretty vague about what the internal visibility does differently than giving the user account permissions who triggers the workflow in the first place.

Conclusion

After setting the package’s visibility to internal it was possible for our workflow to successfully run. Anyway, we found out that the GitHub repository is not yet a better option to migrate to with our CI runs. Having the possibility to cache the container image would make a huge difference in speed. But since this is not (yet?) possible we will stay with our current setup.

Thank you for reading,
Niklas

My first DevOps job interview Part 3 of 3

Niklas — Thu, 29 Apr 2021 07:18:39 +0000

This article is part 3 of a 3 part series of my job interview experience as a DevOps Engineer.

Part 1
Part 2
Part 3

The last part was about setting up the Kubernetes Cluster and deploying the NodeJS app. Today I want to talk about the last exercise and my conclusion.

Learning about Auto Scaling

The last task dealt with the topic of resource limits and scaling strategies. I had to think about the factors I would use to determine the scaling strategy. I wasn’t ready to know about resource limits or scaling strategies within my Kubernetes course, so I fired up my search engine of choice DuckDuckGo and came across how resource limits and auto scaling works in Kubernetes.

Resource limits

Resources can be easily specified in the specification of a pod. To do this, one adds a resource object to the respective container, as follows:

...
containers:
    - name: exercise-app
    image: niklasmtj/exercise-app:v1
      resources:
        limits:
        memory: "128Mi"
        cpu: "500m"

Here you can see that the container may use a maximum of 128 MB Ram as well as half of a CPU. 500m stands for 500 millicpu. You can read more about this in the Kubernetes documentation: Managing Resources for Containers.

Auto Scaling strategy

As results to my search I found information about Horizontal Pod Autoscaler (HPA). I never heard of these through my course, as they weren’t explained until the end of Part 3, and I only finished that a few days after these assignments. However, HPAs seemed perfect for me to use for this, as Kubernetes does the auto-scaling of the pods. Below is my decision on the utility of HPAs, as well as the metrics by which the HPA scales the pods.

The Pods will be scaled horizontally via Kubernetes’ own Horizontal Pod Autoscaler (HPA). Checking the resources is a good indicator of how much an application has to handle at that moment. Because of that the auto scaling begins when the CPU usage of the Pod exceeds 50%. This will trigger Kubernetes’ HPA which then creates a new Pod to handle the traffic.
There are also Vertical Pod Autoscaler (VPA) that will increase the given resources to scale the Pod vertically.
In my opinion checking the number of requests as a scaling strategy can be possible. BUT: applications use resources very differently. A web application that only serves a simple static file or only text like the used NodeJS app that prints Hello World to the page can handle a lot more requests than a more complex one which has more side effects (e.g. getting data from an remote API). If a developer is really sure about the correlation of number of requests and resouce usage then it would be a good idea to use it as a scaling strategy.

To create a HPA with those 50% cpu restriction, a minimum of 1 Pod and a maximum of 10 pods one can use:

kubectl autoscale deployment exercise-app --namespace exercise-ns --cpu-percent=50 --min=1 --max=10 --dry-run=client -o yaml > manifests/hpa.yaml

Which will outputs the desired HPA definition to a yaml file in the manifests directory.
The file will then look like this:

apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  creationTimestamp: null
  name: exercise-app
    namespace: exercise-ns
spec:
  maxReplicas: 10
  minReplicas: 1
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: exercise-app
  targetCPUUtilizationPercentage: 50
status:
  currentReplicas: 0
  desiredReplicas: 0

Testing the scaling strategy

To test the scaling effects one could use Bombardier which creates multiple HTTP connections to check the response of a service. The test can be done with a temporary Docker container which can be run with:

docker run --ti --rm alpine/bombardier -c 500 -d 120s -l http://192.168.178.41:8081

http://192.168.178.41:8081 is in this example my local IP address where the app is accessible. The flags -c 500 -d 120s tells Bombardier to create 500 concurrent connections for 2 minutes.
To create a Kubernetes Pod which will be deleted after its’ job one can use:

kubectl run bombardier --image=alpine/bombardier --rm -it --restart=Never --command -- bombardier -c 500 -d 120s -l http://exercise-svc.exercise-ns

This will do the same but calls the exercise service in it’s exercise-ns namespace.
After a while starting Bombardier kubectl get hpa -n exercise-ns will show:

NAME             REFERENCE               TARGETS    MINPODS   MAXPODS   REPLICAS   AGE
rendertron-hpa   Deployment/rendertron   230%/50%   1         10        5          6m36s

This shows that 5 Replicas of the Pod definition are running because the CPU is at 230%. The Horizontal Autoscaler works because it dynamically created 4 additional Pods to manage the incoming traffic.

My conclusion

This was my experience and reflections on my first practical task on Kubernetes for a job. Despite the problems at the beginning, the tasks were fairly given and were quite solvable. Towards the end, it was really a lot of fun to solve the tasks and learn new parts of Kubernetes. Kubernetes takes so much of the configuration work out of your hands. Especially in the area of autoscaling, you as a developer or operations team have to worry much less about whether the upcoming traffic can be handled by the systems.
I am very glad to have started with the DevOps with Kubernetes course shortly before. It made me much more confident in setting up the cluster and at least I made quick progress there. Especially after the initial complications, it gave me more confidence. I definitely want to get further into Kubernetes and learn more in the DevOps area. This area still seems to offer so much opportunity to make developers and operations teams’ jobs easier.

Thank you for reading,
Niklas

The code from this post can also be found on GitHub: niklasmtj/kubernetes-exercise.
Additionally, I created an arm64 as well as an amd64 docker image for niklasmtj/exercise-app:v1. So the example app should be usable on other devices as well.

The series:

Part 1
Part 2
Part 3

My first DevOps job interview Part 2 of 3

Niklas — Thu, 22 Apr 2021 07:09:15 +0000

In the last part I introduced in the exercises and talked about the complications I had with building a Dockerimage with Chrome inside on an arm64 platform. Also the used NodeJS app and its’ Dockerfile was presented. This part will be more about Kubernetes, setting up the Cluster and deploying the NodeJS app.

Setting up a Kubernetes Cluster

After creating the Dockerfile it was about setting up a Kubernetes cluster. Which program I use for this was up to me. In my DevOps with Kubernetes course we use k3d, this implements K3s in Docker. During the course I had not experienced any problems with this solution, so I was sure to use k3d to solve the assignments.
Installing k3d on macOS is easy via Homebrew with brew install k3d. The cluster can then be created with k3d cluster create --port '8082:30080@agent[0]' -p 8081:80@loadbalancer --agents 2. The 8081:80@loadbalancer will allow our apps to be accessible to us via localhost:8081.

Deploying the app

With a running Kubernetes cluster, deployment was now on the agenda. I decided from the beginning to logically separate the app from the rest of the cluster and created a separate namespace. This is possible with kubectl create namespace exercise-ns in Kubernetes. The basic structure for deployment can be found in the Kubernetes documentation (Deployments | Kubernetes). So I created a deployment.yaml in my manifests folder with the following content:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: exercise-app
    namespace: exercise-ns
  labels:
    app: exercise-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: exercise-app
  template:
    metadata:
      labels:
        app: exercise-app
    spec:
      containers:
      - name: exercise-app
        image: niklasmtj/exercise-app:v1

To make the deployment discoverable I also created a service. The Kubernetes documentation for services (Deployments | Kubernetes) again provides the basic structure. The service definition can be found under manifests/service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: exercise-svc
    namespace: exercise-ns
spec:
    type: ClusterIP
  selector:
    app: exercise-app
  ports:
    - protocol: TCP
      port: 80
      targetPort: 3000

It is important to note here that the targetPort corresponds to the port exposed by the Docker container.

Ingress

After preparing the deployment, the task was now to be able to access it. Thus, the connection via an Ingress came into play. By default, K3s uses the Traefik Ingress Controller for Ingress routing, which I also used. The ingress configuration is quite simple. In an ingress.yaml file I used the following configuration:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: exercise-ingress
  namespace: exercise-ns
spec:
  rules:
  - http:
      paths:
      - path: /
        backend:
          serviceName: exercise-svc
          servicePort: 80

This way the / path is directly forwarded to the exercise service we defined in service.yaml on port 80.

IP-Whitelisting

Another task I had to solve with an Ingress was to limit the IP range that is accepted at all. Unfortunately, due to my previous knowledge, I did not had the possibility to implement the whole thing at that time. Nevertheless I read up on how I would implement it and documented it in my readme. During the task I took a closer look at the NGINX Ingress Controller, because I found the most results about it during my research. There it seems to work by a simple annotation in the ingress.yaml:

metadata:
  annotations:
    ingress.kubernetes.io/whitelist-source-range: "192.168.0.0/16"

This will make sure that only IPs from the 192.168.0.0/16 net is able to connect to the apps. So every IP starting at 192.168.0.1 to 196.168.255.254 are eligible to connect. NGINX will drop every request not coming from this IP range.

Also, attempts with Traefik as the Ingress controller, which is used by k3s by default, failed. This is an area I need to look at in more detail in the near future to understand exactly how Ingress works and at what level you can block IPs. Additionally, I need to look at whether it makes a difference that my k3s cluster is running inside Docker and how that affects incoming IPs.

Thank you for reading,
Niklas

The series:

Part 1
Part 2
Part 3 - coming April 29

My first DevOps job interview Part 1 of 3

Niklas — Thu, 15 Apr 2021 13:26:16 +0000

Introduction

In the middle of March I had my first interview for a DevOps Engineer job for the time after my studies. During the interview process I was given a task to demonstrate my Kubernetes skills. Basically, the task was to create a Docker image for a NodeJS app and then deploy it to a Kubernetes cluster. I had about 8 hours to complete the task. Below, I will go into detail about each of the tasks I was given. However, I will not formulate the task 1:1, but paraphrase what should be done in the task. Nevertheless, with this article I will try to give a feeling for how a practical task is structured in a DevOps job interview.

For information: I used a MacBook Pro with M1 chip for the following tasks. Why I mention this will become clear in a moment.
The NodeJS app I was supposed to roll out was built on puppeteer, which is a library for using a headless Chrome or Chromium browser. To create the Dockerfile I followed the documentation of puppeteer. This shows that Chrome can be downloaded directly via apt package management using the URL http://dl.google.com/linux/chrome/deb/. After I found out that Chrome does not currently provide an arm64 build, I used an amd64 Docker image of Node. However, this did not lead to the desired goal either. Finally, I downloaded and installed Chrome directly via wget. This led to a successfully built container image, but shortly after starting the container it threw Chrome errors and was not usable. After 2 1/2 hours I contacted the company to see if it was okay to use a simple NodeJS app for the remaining tasks, so I could at least solve the remaining tasks (there were still 5 tasks ahead of me).

At the end of the 3 articles is a link to the GitHub repository with all the files that can be used to deploy the following app in Kubernetes.

The rather unspectacular NodeJS app looks like this:

const express = require('express');
// Constants
const PORT = 3000;
const HOST = '0.0.0.0';

// App
const app = express();
app.get('/', (req, res) => {
  res.send('Hello World');
});

app.get('/health', (req, res) => {
  res.sendStatus(200);
});

app.listen(PORT, HOST, () => {
  console.log(`Running on http://${HOST}:${PORT}`);
});

The Dockerfile

There was still a requirement to create a Dockerfile for the NodeJS. The following Dockerfile is not 1:1 the same one I used during the task. I extended it by some best practices afterwards. It is important to make sure that the user running the program in the container is not root. For this the node images provide the node user. This can be set up by adding the line USER node just before the app is launched.
Another security for the container, but also for the user of the images, is to set a specific node image. This way you can ensure at any time that the dependencies you find are the ones you tested with.
This is done by using the SHA code of the respective image e.g. FROM node:14@sha256:00e90d6cbb499653cd2c74a3770f4fa5982699145b113e422bdffe31a7905117 for an arm64 build of node in version 14.
The same is true for the apt-get update command before installing new dependencies in a container. Since the sources can change, it is otherwise not possible to ensure that there are no problems when building new containers. Thus, it is better to test directly with a new build of the base image. More best practices for Docker images can be found at Docker Security Best Practices from the Dockerfile or Docker Security - OWASP Cheat Sheet Series.

The next part will be about setting up a Kubernetes Cluster and deploying the NodeJS app in the cluster.

Thank you for reading,
Niklas

The series:

Part 1
Part 2 - coming April 22
Part 3 - coming April 29

My DevOps learning path

Niklas — Fri, 26 Feb 2021 12:37:34 +0000

My goal for the time after my university is to find a job in the field of DevOps or Site Reliability Engineering. The Phoenix Project¹ and the DevOps Handbook² books by Gene Kim where both great books to get a glimpse about what matters in DevOps. Especially The Phoenix Project was great since it is written as a fictional story not as a "real" technical non-fiction book but still delivers in explaining the Three Ways³ which are the three principles of DevOps perfectly.

Although I have been increasingly involved in software development in recent years, I am drawn to the field of DevOps. Perhaps this has something to do with my background as a system integrator. I am fascinated by the complexity of designing systems reliably and rolling them out as automatically as possible. The knowledge I have built up in JavaScript and NodeJS could help in this process.

This is the knowledge and understanding of the topic that I have right now.
I will try to write a post about the knowledge I’m gaining about these when I’m going forward in the process.

Following are the steps I will take to learn those topics.

First I started with getting knowledge about Kubernetes. It seems to grow to be the state of the art orchestration tool for systems and pretty much every job posting requires at least basic knowledge of it.

Kubernetes LinkedIn course

To get a quick introduction to the terms of Kubernetes I used my LinkedIn Learning membership that I can use as a student of the University of Cologne. This course by Karthik Gaekwad gives an overview of the terms and techniques of Kubernetes.
Kubernetes (K8s) is used to orchestrate containers on one’s infrastructure. Orchestration in that case means deploying applications to infrastructure and managing the application’s state, replication status. K8s also has integrated health checks for Pods (the smallest possible entity, one Pod holds one or multiple containers). So when a Pod crashes Kubernetes will make sure that the crashed Pod will be restarted.
In some cases, it lacks a little bit of explanation of terms that are used in the current video. Anyways, overall it was a great video course to get into K8s.

Resources

Learning Kubernetes

DevOps with Kubernetes course by the University of Helsinki

This free course is developed by Jami Kousa who works at the University of Helsinki. The course has been recommended by a friend who used it to get the knowledge for his Certified Kubernetes Application Developer (CKAD) certificate. It uses k3d for setting up the local Cluster. The Google Cloud Platform also is introduced in the course especially the Google Kubernetes Engine which is used in one part of the course.
Kubernetes (K8s) is used to orchestrate containers.
The course introduces one to Kubernetes and shows one how networking and storage works. After that in Part 2 one will work with Clusters, StatefulSets and Jobs. Plus there is a chapter about monitoring the cluster. Part 3 looks to be about the Google Kubernetes Engine and building a Deployment Pipeline. Part 4 is newly added and will introduce GitOps. What it is and why it is important will be described in the next chapter. The last part of the course is about the internals of Kubernetes, Custom Resource definitions and Service Meshes.
I think that this course will be a great introduction to Kubernetes and a good way to get the hands dirty.

Resources:

DevOps with Kubernetes 2020

Gitops

GitOps is a topic that is newly added as a part of the DevOps with Kubernetes course. So I think this is a good opportunity to get more into the actual implementation and usage of Kubernetes „in the real world“.

When working with Kubernetes Clusters GitOps is a way to manage the clusters and automate the application delivery via git. Git in this case is the single source of truth. This means that the desired production environment is defined in a git repository which will be used to identify any drifts off of the running infrastructure. Using those container definitions helps to automate the continuous delivery of one’s applications.
Because the definitions are managed in a git repository it is easy to see the changes of an application definition. Plus it is way easier to rollback to a working definition of a system when problems happen, since the prior version can be found inside of the git history.
The Guide To GitOps by Weaveworks seems to be a great source of knowledge about GitOps. I heard about the company and their knowledge base in a recent Software Engineering Radio podcast with Alexis Richardson the founder and CEO of Weaveworks ⁴.
I will use the two sources below and the course section as my starting point for learning about GitOps.

Resources:

Ansible / Terraform

Ansible and Terraform or Infrastructure as Code tools in general are important things to know in the world of automating deployments.
Both applications seem to be fundamental tools in building automated infrastructure. Also in the world of GitOps, they look to have an important role since both of those tools help declaring the infrastructure.
Ansible is a tool that is mostly used to provision a server. This means that the definition contains the applications and configs that have to be present on the machine. It was acquired by Red Hat in 2015. Ansible uses so-called playbooks that are written in the YAML format. It is also possible to do Ad-hoc task executions for example to ping all the available hosts.
Terraform uses its’ own configuration language. It is called HashiCorp Configuration Language or HCL for short. Terraform is mostly used to configure cloud resources and helps for example to define the availability zones or instance types of a resource. It can also help to organize infrastructure across multiple clouds.

Resources:

So these are the topics I will learn in my semester break and possibly in the months after that. Going through these will probably bring up a lot more topics that will follow for example command-line tools like jq or yq and awk. I will keep you informed about the current state.

Thank you for reading,
Niklas

Simple Analytics iOS widget with Scriptable app

Niklas — Mon, 22 Feb 2021 19:37:27 +0000

Today I quickly build an iOS widget to see my page views and visitors on my iPhone. Since I don't always want to start my laptop to see my current website stats in Simple Analytics I created this widget based on an example widget I found in the Scriptable app. The great thing about Scriptable is that uses the JavaScript Core of iOS. This means that a lot of features of ES6 are available to use. You can find a documentation in the Scriptable app or at docs.scriptable.app.
Since it would be very tedious to write code on the small touchscreen, I wrote the code on my laptop and then pasted it on my phone.

Without further ado here are the installation steps.

How to install

Load the scriptable app for your device
Copy the content of the getSimpleAnalyticsStats.js file on GitHub and fill in the first 3 lines of the file with your apiKey, userId and the domain you want to track. You can get your keys in your account settings https://simpleanalytics.com/account
Try to run the script and see if everything works.
Go back to your home screen and trigger the "jiggle-mode"
Now add scriptable and choose your newly created script as the script to run
Have fun with your Simple Analytics iOS widget

The GitHub repository can be found here.

Thank you for reading,
Niklas

This post and more you can find on niklasmtj.de