DEV Community: Simon Hofmann

Migrating a non-trivial Travis CI pipeline to GitHub Actions

Simon Hofmann — Fri, 08 Jan 2021 23:46:07 +0000

Image source: https://unsplash.com/photos/Grg6bwZuBMs

Avid readers of my blog know that I’m really appreciating a proper CI / CD setup. Until a few weeks ago, my go-to solution for such setups has been Travis CI. Unfortunately, Travis slightly changed its business model which drastically limits its use for open-source projects. For me, this was a huge setback, since I’m relying on Travis CI to provide nut.js. And since I’m not making any money out of this project, it’s hard to reason paying at least 69$ per month for CI.
As I had already started using GitHub Actions in other repos, I decided to migrate nut.js to GitHub Actions as well.

The existing pipeline

So, what do we have to migrate?

A nut.js build consists of two to three stages:

sonar
test
deploy (only triggered on tags and on commits to the develop branch)

The sonar stage run on Linux with a fixed node version and executes all tests.
That is:

all platform independent tests
all platform dependent tests
all tests requiring a GUI
all tests running in a Docker container

Generated coverage reports are merged and reported to sonarcloud.

The following test stage is a build matrix, running tests on both macOS and Linux with varying node versions.
On Linux, the test stage runs the exact same set of tests as the sonar stage while on macOS tests running in Docker containers are left out.

The optional deploy stage will either publish a new @next snapshot release for build on the develop branch, or a new @latest release for tags.

In summary, the requirements for a successful migration to GitHub Actions are the following:

Build stages
Build matrices
Docker support
Running GUI tests
Sonar reports

Let’s migrate!

The final setup is split into three distinct files, one for general branch and pull request build, one for snapshot releases and one for stable releases.
Let’s look at branch build first.

Sonar stage

Or workflow should run on pushes to all branches except develop and any release branch under release/** as well as pull requests.

name: Run CI
on:
  push:
    branches-ignore:
      - develop
      - release/**
  pull_request:

Travis stages are resembled by jobs on GitHub Actions.

jobs:
  sonar:
    runs-on: ubuntu-latest
    steps:
      - name: Set up Git repository
        uses: actions/checkout@v2
      - name: Set up node
        uses: actions/setup-node@v2
        with:
          node-version: 14

Our job runs on Linux (ubuntu-latest defaults Ubuntu 18.04, soon upgrading to Ubuntu 20.04), checks out our repo and installs node v14.x.x.
On Linux, Docker is available right out of the box, so we’re able to start a daemonized container:

- name: Setup Docker
  run: |
    docker pull s1hofmann/nut-ci:latest
    docker run -it -d --name nut-ci --shm-size 4gb --user $(id -u):$(id -g) -v ${PWD}:${PWD}:rw s1hofmann/nut-ci:latest bash

What follows are some steps to initialise the project:

- name: Install
  run: npm ci
- name: Compile
  run: npm run compile
- name: Init e2e test subpackage
  run: npm --prefix e2e/tests ci
- name: Clean coverage report
  run: npm run coverage:clean

Once set up, we now want to run tests which require a GUI.
A common way of running headless GUI tests on Linux is using a virtual framebuffer.
Travis provides a simple to use service to enable a virtual framebuffer in a build, so let’s find a proper replacement for GitHub Actions.
GabrielBB/xvfb-action provides a nice wrapper which will start a virtual framebuffer where possible.
When running on Linux, a virtual framebuffer will be set up before running your action, while on other platforms it will be executed right away.
No additional configuration required when running in a cross-platform setup (like we do in our matrix build)!

Following the GUI and unit tests, let’s execute E2E tests running in a Docker container and merge all coverage reports:

- name: Run Docker E2E tests
  run: docker exec nut-ci bash -c "bash $PWD/.build/build.sh ${PWD} 14"
- name: Merge coverage reports
  run: |
    npm run coverage:merge
    npm run coverage:merge-report

Travis provides an add-on for sonarcloud to easily run sonarcloud analyses.
Luckily, there’s an official SonarCloud Action to run scans on GitHub Actions.
Running sonar-scanner with an existing sonar-project.properties config becomes as easy as

- name: Send results to SonarCloud
  uses: SonarSource/sonarcloud-github-action@v1.4
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

Instead of storing the encrypted Sonar token in our workflow file like on Travis, we’re storing and referencing it as a repo secret and we’re good to go!

Test stage

Following our sonar stage we want to run a build matrix with multiple platforms and node versions.
By default, jobs run in parallel, but since we only want to run our test job in case the previous sonar stage has been successful, we’re setting a requirement to the previous job:

test:
  needs:
    - sonar

Now let’s configure the build matrix:

strategy:
  matrix:
    os: [ ubuntu-latest, windows-latest, macos-latest ]
    node: [ 10, 12, 14 ]
    exclude:
      - os: ubuntu-latest
        node: 14
    runs-on: ${{matrix.os}}
    steps:
      - name: Set up Git repository
        uses: actions/checkout@v2
      - name: Set up node
        uses: actions/setup-node@v2
        with:
          node-version: ${{matrix.node}}

We want to run our tests on Linux, Windows and macOS using node version 10, 12 and 14.
But since our sonar stage already ran on Linux using node 14, we’re excluding it from our build matrix to save time and compute.
In total, our build matrix will expand into 8 separate builds, all running in parallel.

We also want to limit our Docker tests to Linux, so we can limit the whole Docker setup to build running on ubuntu-latest:

- name: Setup Docker
  if: ${{matrix.os == 'ubuntu-latest'}}
  run: |
    docker pull s1hofmann/nut-ci:latest
    docker run -it -d --name nut-ci --shm-size 4gb --user $(id -u):$(id -g) -v ${PWD}:${PWD}:rw s1hofmann/nut-ci:latest bash

    ...

- name: Run Docker E2E tests
  if: ${{matrix.os == 'ubuntu-latest'}}
  run: docker exec nut-ci bash -c "bash $PWD/.build/build.sh ${PWD} ${{matrix.node}}"

Deploy stage

Snapshot releases should only happen on commits to develop:

name: Create snapshot release
on:
  push:
    branches:
      - develop

while latest releases are limited to semver tags:

name: Create tagged release
on:
  push:
    tags:
      - v*.*.*

When releasing new packages we do not require a sonar scan, so we just skip this stage completely and only run our test stage.
Depending on the outcome of this stage, a package release is triggered:

deploy:
  needs:
    - test
  runs-on: ubuntu-latest
  steps:
    - name: Set up Git repository
      uses: actions/checkout@v2
    - name: Set up node
      uses: actions/setup-node@v2
      with:
        node-version: 14
    - name: Publish tagged release
      uses: JS-DevTools/npm-publish@v1
      with:
        token: ${{ secrets.NPM_TOKEN }}

To be resolved?

There’s quite a lot of duplication in our workflows which I’d like to reduce. Something similar to how GitLab CI handles it would be nice, but it looks like it has yet to be supported by GitHub Actions.

Another thing which is not that great is missing support to skip builds via commit message.
It’s a common pattern to skip CI builds via commit message by prefixing the message with e.g. [skip_travis] or [skip_ci].
Unfortunately, GitHub Actions do not yet support such kind of shortcut and will still run your workflows.

Conclusion

Migrating the existing Travis CI pipeline to GitHub Actions turned out easier than initially expected.
Due to the wide range of community actions it was fairly easy to tick all the required boxes.

As a result we’re now able to run all our builds on a single CI system (Windows builds were carried out on AppVeyor since Windows build are in an early stage on Travis while the overall build runtime decreased!

Responsive on-screen keys

Simon Hofmann — Tue, 01 Dec 2020 22:51:23 +0000

For my most recent little game I noticed that touch input does not work that well with this type of game, but on-screen keyboards of mobile devices do not provide arrow keys, nor can they be forcefully opened without focusing input fields.
So, when played on a device with touch input (smartphone, tablet, etc.), users should be provided with dedicated on-screen keys — CSS to the rescue!

First try — Size dependent on-screen keys

In a first, naive approach I decided to generally display touch controls on smaller screens. I went with something like the following:

.on-screen-keys {
    display: none;
}
@media (max-width: 768px) {
    .on-screen-keys {
        display: inline-flex;
    }
}

However, screen size is not the best criteria to determine whether to display on-screen keys.

Next try — Pointer device

A better approach would be to determine whether we’re dealing with a touch device or not. A little research revealed the CSS pointer media feature, a way to determine how accurate the primary pointing device is.
Smartphones, tablets etc. which are primarily using touch input are identified as coarse input with limited accuracy, a distinction from fine input via mouse / keyboard.

.on-screen-keys {
    display: none;
}
@media (pointer: coarse) {
    .on-screen-keys {
        display: inline-flex;
    }
}

Demo

I put together a little sample on CodePen. Don’t expect to see anything when visiting it from a device with pointer: fine, but when viewed on e.g. a tablet, you should see the same buttons as used in my game.

Conclusion

Never underestimate CSS!

You don’t have to build a to-do app ...

Simon Hofmann — Fri, 27 Nov 2020 13:15:58 +0000

... to learn new things!

That’s what I thought to myself when I recently hopped onto the Svelte bandwagon to see what it had to offer!

I started to overhaul my website and my first impressions were amazing! Fronted development is pretty opinionated (as software development is in general), but for me, Svelte just hit the sweet spot!

It comes with batteries included to get quite many things done, but also gets out of your way when integrating 3rd-party tools.

Naturally, I wanted to build another thing to see whether my first impression was right. Based on my recent blog post, I re-implemented a co-op multiplayer Tetris clone.

It uses:

Thanks to GitHub Actions it is automatically deployed to my server.

My first impression of Svelte remained intact, it’s been a joyful experience to re-implement the game!

How to play

There are two different game modes:

Team-Blocks: Multiple playing field overlap. Every player has its own and only those rows completed on every field are cleared.
Tetrolyseus: Two players share a single game. One player can move blocks, the other one rotates.

Open games are either join via game lobby or directly via link. Private games won't show up in the lobby, so you can privately invite your teammates via link!

Once everybody set itself ready, the game is on!

Blocks are steered via keyboard input, but thanks to some media queries, there are on-screen controls on touch devices!

Where to play?

It’s available at: https://multris.s1h.org

Have fun!

Building a multiplayer game with colyseus.io

Simon Hofmann — Sat, 07 Nov 2020 19:14:35 +0000

Computer games are awesome! Not only are they fun to play, but they’re also quite fun to build. Virtually every programmer, at one point or another, has at least thought about building a game.

That said, building games is not easy, and it takes a lot of imagination to create something truly impressive. If you want to build a multiplayer game, you must not only create a great game but also set up all the networking, which is a daunting task in itself.

Colyseus is designed to reduce the burden of networking so you can fully concentrate on your game mechanics. To demonstrate what it has to offer, we’ll implement a multiplayer Tetris clone — we’ll call it Tetrolyseus.

Getting started - Colyseus Backend Setup

Colyseus provides an npm-init initialiser which automates the creation of new projects.

npm init colyseus-app ./my-colyseus-app

This interactive initialiser will take care of our basic setup. While it’s also possible to use Colyseus with plain old JavaScript or Haxe, we are going to stick with TypeScript.

? Which language you'd like to use? …
❯ TypeScript (recommended)
  JavaScript
  Haxe

Once completed we will have the following files generated for us in my-colyseus-app:

.
├── MyRoom.ts
├── README.md
├── index.ts
├── loadtest
├── node_modules
├── package-lock.json
├── package.json
└── tsconfig.json

We will dive right into Colyseus by taking a closer look at

index.ts
MyRoom.ts

index.ts

The newly created index.ts file is our main entry point which sets up our server:

const port = Number(process.env.PORT || 2567);
const app = express()


app.use(cors());
app.use(express.json())

const server = http.createServer(app);
const gameServer = new Server({
  server,
});

While not being necessarily required, the default colyseus-app templates also uses express, so we’re able to easily register additional route handlers on our backend. In case we do not want to provide additional handlers our setup boils down to:

const port = Number(process.env.PORT || 2567);

const gameServer = new Server();

The second part of our index.ts file is where we actually expose our game logic:

// register your room handlers
gameServer.define('my_room', MyRoom);

// skipped for brevity

gameServer.listen(port);
console.log(`Listening on ws://localhost:${ port }`)

Colyseus uses the notion of ”rooms” to implement game logic. Rooms are defined on our server with a unique name which our clients use to connect to it. A room handles client connections and also holds the game’s state. It is the central piece of our game, so we will see what they look like next.

MyRoom.ts

import { Room, Client } from "colyseus";

export class MyRoom extends Room {
  onCreate (options: any) {
    this.onMessage("type", (client, message) => {
      // handle "type" message
    });
  }

  onJoin (client: Client, options: any) {
  }

  onLeave (client: Client, consented: boolean) {
  }

  onDispose() {
  }
}

As we can see, a few lifecycle events are attached to a Colyseus room.

onCreate is the first method to be called when a room is instantiated. We will be initialising our game state and wiring up our message listeners in onCreate
onJoin is called as soon a new client connects to our game room
onLeave is the exact opposite to onJoin, so whenever a client leaves, disconnect and reconnection logic will be handled here
onDispose is the last method to be called right before a game room will be disposed. Things like storing game results to a database and similar tasks might be carried out in onDispose An additional event, although not included in the default room implementation, is onAuth. It allows us to implement custom authentication methods for joining clients as shown in the authentication API docs.

Now that we’ve gained an overview of a basic Colyseus backend setup, let’s start modelling our game state.

You can find the code we wrote so far in the accompanying repository on GitHub. The corresponding tag is 01-basic-setup:

git checkout tags/01-basic-setup -b 01-basic-setup

Managing Game State

In one way or another, every game is holding state. Player position, current score, you name it. State makes the backbone of a game.
When talking about online multiplayer games, state becomes an even more complex topic. Not only do we have to model it properly, but now we also have to think about how we’re going to synchronise our state between all players.
And that’s where Colyseus really starts to shine. Its main goal is to take away the burden of networking and state synchronisation so we’re able to focus on what matters - our game logic!

Stateful Game Rooms

Previously we learned that a Colyseus room is able to store our game state. Whenever a new room is created, we initialise our state:

import { Room, Client } from "colyseus";
import { MyGameState } from "./MyGameState";

export class MyRoom extends Room<MyGameState> {
  onCreate (options: any) {
    this.setState(new MyGameState());
    ...
  }

  ...
}

Every time a client connects to our room it will receive the full room state in an initial synchronisation, automatically.
Since room state is mutable, it has to be synced continuously. However, following the full state sync, Colyseus will only send incremental updates which are applied to the initial state. The interval for state syncs is configurable for each room via its patchRate and defaults to 50 milliseconds (20 fps). Shorter intervals allow for fast-paced games!

So without further ado, let’s model our state!

Position

The two-dimensional Tetrolyseus board consists of several rows and columns. The Position state object is used to store the position of our active Tetrolyso block by its top-left row and column:

import {Schema, type} from "@colyseus/schema";

export class Position extends Schema {
    @type("number")
    row: number;

    @type("number")
    col: number;

    constructor(row: number, col: number) {
        super();
        this.row = row;
        this.col = col;
    }
}

Our state class has to fulfil certain properties to be eligible for synchronisation:

It has to extend the Schema base class
Data selected for synchronisation requires a type annotation
A state instance has to be provided to the game room via setState

Position is a simple state class which synchronises two number properties: row and col. It nicely demonstrates how Colyseus Schema classes allow us to assemble our state from primitive types, automatically enabling synchronisation.

Board

Next up is our game board state. Similar to Position it stores two number properties, the rows and cols of our two-dimensional game board. Additionally, its values property holds an array of numbers, representing our board.
So far, we only worked with single data, so how are we going to model our state class holding a data collection? With Colyseus, collections should be stored in an ArraySchema, Colyseus’ synchronizable Arraydatatype for one-dimensional data.

import {ArraySchema, Schema, type} from "@colyseus/schema";

export class Board extends Schema {
    @type(["number"])
    values: number[];

    @type("number")
    rows: number;

    @type("number")
    cols: number;

    constructor(rows: number = 20, cols: number = 10) {
        super();
        this.rows = rows;
        this.cols = cols;
        this.values = new ArraySchema<number>(...(new Array<number>(rows * cols).fill(0)));
    }
}

Tetrolyso

A Tetrolyso block is basically just an extended version of a Board, having an additional number property storing it’s color. It is skipped here for brevity. Instead, please refer to the available implementation on GitHub.

GameState

What’s more interesting is our overall game state.

import {Schema, type} from "@colyseus/schema";
import {getRandomBlock, Tetrolyso} from "./Tetrolyso";
import {Position} from "./Position";
import {Board} from "./Board";

export class GameState extends Schema {
    @type(Board)
    board: Board;

    @type(Tetrolyso)
    currentBlock: Tetrolyso;

    @type(Position)
    currentPosition: Position;

    @type(Tetrolyso)
    nextBlock: Tetrolyso;

    @type("number")
    clearedLines: number;

    @type("number")
    level: number;

    @type("number")
    totalPoints: number;

    constructor(rows: number = 20, cols: number = 10, initialLevel = 0) {
        super();
        this.board = new Board(rows, cols);
        this.currentBlock = getRandomBlock();
        this.currentPosition = new Position(0, 5);
        this.nextBlock = getRandomBlock();
        this.level = initialLevel;
        this.clearedLines = 0;
        this.totalPoints = 0;
    }
}

It consists of a few number properties but additionally, it possesses several child schema properties to assemble the overall state.
Using such nested child state classes gives us great flexibility when modelling our state. @type annotations provide a simple and type-safe way to enable synchronisation and nested child schema allow us to break our state down which enables re-use.

Once again, if you want to along, the current tag is 02-gamestate in our repository.

git checkout tags/02-gamestate -b 02-gamestate

Working With Game State - Frontend

Now that our first draft of our state is completed, let’s see how we can work with it. We will start with building a frontend for our game, since it allows us to visualise our game state.
Colyseus comes with a JavaScript client which we’re going to use:

npm i colyseus.js

We won’t be using any frontend framework, only plain HTML, CSS and TypeScript, so the only two additional things used to build our frontend will be:

We will include nes.css via CDN, so we only need to add Parcel to our devDependencies:

npm i -D parcel

Just enough to build the following layout:

+----------------------------------------------------------+
|                                                          |
|  Title                                                   |
|                                                          |
+----------------------------------------------------------+
             +--------------------+ +------------+
             |                    | |            |
             |                    | | Score      |
             |                    | |            |
             |                    | +------------+
             |                    | +------------+
             |                    | |            |
             |                    | | Level      |
             |                    | |            |
             |      Playing       | +------------+
             |      Field         | +------------+
             |                    | |            |
             |                    | | Next Piece |
             |                    | |            |
             |                    | +------------+
             |                    |
             |                    |
             |                    |
             |                    |
             |                    |
             |                    |
             +--------------------+

The HTML representation of our layout looks like this:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Tetrolyseus</title>
    <link href="https://unpkg.com/nes.css@2.3.0/css/nes.min.css" rel="stylesheet"/>
    <link href="https://fonts.googleapis.com/css2?family=Press+Start+2P&display=swap" rel="stylesheet">
    <link rel="stylesheet" type="text/css" href="index.css">
</head>
<body>
<div class="nes-container is-dark with-title">
    <p class="title">Tetrolyseus</p>
    <p>A cooperative approach to the famous blocks game.</p>
</div>
<div id="playingfield">
    <div id="board" class="nes-container is-rounded is-dark"></div>
    <div id="infobox">
        <div class="nes-container is-dark with-title">
            <p class="title">Score</p>
            <p id="score"></p>
        </div>
        <div class="nes-container is-dark with-title">
            <p class="title">Level</p>
            <p id="level"></p>
        </div>
        <div class="nes-container is-dark with-title">
            <p class="title">Next</p>
            <div id="preview"></div>
        </div>
    </div>
</div>
</body>
<script src="index.ts" type="application/javascript"></script>
</html>

Connecting To The Backend

First of all, we’re going to establish a connection to our backend:

document.addEventListener('DOMContentLoaded', async () => {
    const client = new Client(process.env.TETROLYSEUS_SERVER || 'ws://localhost:2567');

    ...
});

Once connected, we can now join or create a game room:

const room: Room<GameState> = await client.joinOrCreate<GameState>("tetrolyseus");

The name we’re providing to joinOrCreate must be one of the game rooms defined on or backend. As its name might imply, joinOrCreate will either join an existing room instance, or create a new one. Besides that, it’s also possible to explicitly create or join a room.
In return, joinOrCreate provides us a Room instance holding our GameState, giving us access to our Board, the current Tetrolyso, its current Position and so on. Everything we need to render our game!

Game Rendering

Now that we have access to our current GameState, we’re able to render our UI. Using CSS Grid and our Board state, we can draw our playing field:

const drawBoard = (board: Board): void => {
    const boardElement = queryBoardElement();
    const elementRect = boardElement.getBoundingClientRect();
    const blockHeight = Math.floor((elementRect.height - 32) / board.rows);
    boardElement.style.gridTemplateColumns = `repeat(${board.cols}, ${blockHeight}px)`;
    boardElement.style.gridTemplateRows = `repeat(${board.rows}, ${blockHeight}px)`;
    boardElement.style.height = "fit-content";
    boardElement.style.width = "fit-content";

    const boardPosition = queryByRowAndColumn(board);

    for (let row = 0; row < board.rows; ++row) {
        for (let col = 0; col < board.cols; ++col) {
            const cellDiv = document.createElement("div");
            cellDiv.id = `cell-r${row}-c${col}`
            cellDiv.style.background = `#${boardPosition(row, col).toString(16)}`;
            boardElement.append(cellDiv);
        }
    }
}

Given our two-dimensional grid, we can also display the current Tetrolyso:

const drawTetrolyso = (currentBlock: Tetrolyso, currentPosition: Position) => {
    const blockPosition = queryByRowAndColumn(currentBlock);

    for (let row = currentPosition.row; row < currentPosition.row + currentBlock.rows; ++row) {
        for (let col = currentPosition.col; col < currentPosition.col + currentBlock.cols; ++col) {
            if (blockPosition(row - currentPosition.row, col - currentPosition.col) !== 0) {
                const boardSquare = <HTMLDivElement>document.querySelector(`#cell-r${row}-c${col}`);
                boardSquare.style.background = `#${currentBlock.color.toString(16)}`;
                boardSquare.style.border = `1px solid black`;
            }
        }
    }
}

Receiving State Updates

So far, we are able to render our UI given the current state. However, to get our game moving we have to re-render our UI every time our state changes.
Rooms provide certain events we can attach a callback to, so we can attach our rendering code to the onStateChange handler:

room.onStateChange((newState: GameState) => {
    clearBoard();
    clearPreview();
    drawBoard(newState.board);
    drawPreview(newState.nextBlock);
    drawTetrolyso(newState.currentBlock, newState.currentPosition);
    drawScore(newState.totalPoints);
    drawLevel(newState.level);
});

Handling Player Input

At this point you might be wondering when we’re going to implement some game logic to e.g. move our Tetrolyso around, check collisions and so on.
Long story short - we won’t! At least not in our frontend. Our UI should serve a single purpose: rendering our state. State manipulations should happen in our backend.
Whenever one of our players hits a key, we send a message to our backend describing what we want to do, e.g. move or rotate the current block. If our game rules allow us to carry out our desired action, the game state will be updated and our frontend will re-render the UI due to this state change.

document.addEventListener('keydown', (ev: KeyboardEvent) => {
    if (ev.code === "Space") {
        room.send("rotate", {});
    } else if (ev.code === "ArrowLeft") {
        room.send("move", LEFT);
    } else if (ev.code === "ArrowRight") {
        room.send("move", RIGHT);
    } else if (ev.code === "ArrowDown") {
        room.send("move", DOWN);
    }
});

room.send allows us to pass messages from our client to our server. keydown events on one of our arrow keys will instruct our backend to move the current Tetrolyso either left, right or down, hitting space will rotate it.

Frontend Wrap-Up

Our declarative approach to game logic keeps our frontend simple and allows us to focus on what we want to achieve: rendering our game state.
The last thing we’re going to add here is an npm script to build our frontend:

"scripts": {
  "start:frontend": "parcel frontend/index.html"
},

The current frontend state can be found in tag 03-frontend.

git checkout tags/03-frontend -b 03-frontend

Working With Game State - Backend

Ok, time to get started with our game backend. But before we continue writing code, let’s move our existing code to a dedicated subfolder called backend.

backend
├── TetrolyseusRoom.ts
└── index.ts

We will start our backend via the start:backend npm script:

"scripts": {
  "start:backend": "ts-node backend/index.ts",
  "start:frontend": "parcel frontend/index.html"
},

Initialising State

Now that everything is in place, let’s continue extending our TetrolyseusRoom. Being a stateful room, the first thing we’re going to do is to initialise our state:

import {Client, Room} from "colyseus";
import {GameState} from "../state/GameState";

export class TetrolyseusRoom extends Room<GameState> {
    onCreate(options: any) {
        this.setState(new GameState())
    }

    onJoin(client: Client, options: any) {
    }

    onLeave(client: Client, consented: boolean) {
    }

    onDispose() {
    }
}

We haven’t changed much so far, but if we start both our backend and frontend, we should be presented with our game board, showing the level, score, our current Tetrolyso and the next one. Everything rendered based on our initialised state.

Scoring

Next, let’s compute our score for clearing lines following the Nintendo scoring system.

const baseScores: Map<number, number> = new Map<number, number>([
    [0, 0],
    [1, 40],
    [2, 100],
    [3, 300],
    [4, 1200]
]);

export const computeScoreForClearedLines = (clearedLines: number, level: number): number => {
    return baseScores.get(clearedLines) * (level + 1);
}

The scoring implementation is tagged at 04-scoring.

git checkout tags/04-scoring -b 04-scoring

Detecting Collisions

Our blocks are represented by a series of 0’s and 1’s along with row and column information. When visualised, a Z block looks like the following in our game:

+--------+
|110||001|
|011||011|
|000||010|
+--------+

As we can see, due to their shape, some blocks may have empty rows or columns. When it comes to collision detection we have to make up for these empty values, otherwise we won’t be able to use up all the space of our board.
A simple way to accomplish this is to determine the offset by which a blocks exceeds the board and check if any non-zero “block-element” lies within this range.

   +-------------------------+
   |                         |
   |                         |
   |                         |
+-------+                    |
|00|1100|                    |
|00|1100|                    |
|00|1111|                    |
|00|1111|                    |
|00|1100|                    |
|00|1100|                    |
+-------+                    |
   |                         |

export const isLeftOutOfBounds = (board: Board, tetrolyso: Tetrolyso, position: Position): boolean => {
    if (position.col >= 0) {
        return false;
    }

    const blockElement = queryByRowAndColumn(tetrolyso);

    const offset = -position.col;
    for (let col = 0; col < offset; ++col) {
        for (let row = 0; row < tetrolyso.rows; ++row) {
            if (blockElement(row, col) !== 0) {
                return true;
            }
        }
    }
    return false;
}

The same scheme applies for collision checks on the bottom and right side of our board.

Checking whether our current block collides with any of the already existing blocks in our board is quite similar as well. We just check for overlapping non-zero elements between our board and the current block to determine collisions:

export const collidesWithBoard = (board: Board, tetrolyso: Tetrolyso, position: Position): boolean => {
    const blockElement = queryByRowAndColumn(tetrolyso);
    const boardElement = queryByRowAndColumn(board);

    for (let boardRow = position.row; boardRow < position.row + tetrolyso.rows; ++boardRow) {
        for (let boardCol = position.col; boardCol < position.col + tetrolyso.cols; ++boardCol) {
            const blockRow = boardRow - position.row;
            const blockCol = boardCol - position.col;
            if (blockElement(blockRow, blockCol) !== 0 && boardElement(boardRow, boardCol) !== 0) {
                return true;
            }
        }
    }
    return false;
}

The completed collision detection implementation is tagged at 05-collision.

git checkout tags/05-collision -b 05-collision

Making Our Game Work - Game Logic

Until now our game has been rather static. Instead of moving blocks we just witnessed a single, static block which didn’t move.
Before we can get things moving, we have to define some rules our game has to follow. In other words, we have to implement our game logic, which sums up to the following steps:

Calculate next position of falling block
Detect collisions and either move the current block or freeze it at its current position
Determine completed lines
Update scores
Update board (remove completed lines, add empty ones)
Check whether we reached the next level

Game logic implemented in our room re-uses functionality from 05-collision to update our state:

detectCompletedLines() {
    let completedLines = [];
    for (let boardRow = this.state.board.rows - 1; boardRow >= 0; --boardRow) {
        if (isRowEmpty(this.state.board, boardRow)) {
            break;
        }

        if (isRowCompleted(this.state.board, boardRow)) {
            completedLines.push(boardRow);
        }
    }
    return completedLines;
}

updateBoard(completedLines: number[]) {
    for (let rowIdx = 0; rowIdx < completedLines.length; ++rowIdx) {
        deleteRowsFromBoard(this.state.board, completedLines[rowIdx] + rowIdx);
        addEmptyRowToBoard(this.state.board);
    }
}

dropNewTetrolyso() {
    this.state.currentPosition = new Position(
        0,
        5
    );
    this.state.currentBlock = this.state.nextBlock.clone();
    this.state.nextBlock = getRandomBlock();
}

moveOrFreezeTetrolyso(nextPosition: Position) {
    if (
        !isBottomOutOfBounds(this.state.board, this.state.currentBlock, nextPosition) &&
        !collidesWithBoard(this.state.board, this.state.currentBlock, nextPosition)
    ) {
        this.state.currentPosition = nextPosition;
    } else {
        freezeCurrentTetrolyso(this.state.board, this.state.currentBlock, this.state.currentPosition);
        this.dropNewTetrolyso();
        this.checkGameOver();
    }
}

Full game logic is tagged at 06-game-logic.

git checkout tags/06-game-logic -b 06-game-logic

Making Our Game Run - Game Loop

Great, we have our game logic set up! Now, let’s assemble our game loop to get things running!

Our game loop performs all the steps we listed in the previous section:

loopFunction = () => {
    const nextPosition = this.dropTetrolyso();
    this.moveOrFreezeTetrolyso(nextPosition);

    const completedLines = this.detectCompletedLines();
    this.updateClearedLines(completedLines);
    this.updateTotalPoints(completedLines);
    this.updateBoard(completedLines);
    this.checkNextLevel();
}

We will use a Delayed instance for our game clock:

gameLoop!: Delayed;

Our onCreate handler will start the loop:

onCreate(options: any) {
    ...
    const loopInterval = 1000 / (this.state.level + 1);
    this.gameLoop = this.clock.setInterval(this.loopFunction, loopInterval);
    ...
}

So our blocks will initially drop at one row per second, becoming faster as we level up.

In case we reached the next level, we restart our loop:

checkNextLevel() {
    const nextLevel = this.determineNextLevel();
    if (nextLevel > this.state.level) {
        this.state.level = nextLevel;
        this.gameLoop.clear();
        const loopInterval = 1000 / (this.state.level + 1);
        this.gameLoop = this.clock.setInterval(this.loopFunction, loopInterval);
    }
}

The last thing missing in our onCreate are message handlers. Our frontend communicates with our backend via messages. So if we want to be able to rotate or move our blocks, our backend has to process these messages accordingly.

onCreate(options: any) {
    ...
    this.onMessage("rotate", (client, _) => {
        const rotatedBlock = this.state.currentBlock.rotate();
        const rotatedPosition = keepTetrolysoInsideBounds(this.state.board, rotatedBlock, this.state.currentPosition);
        if (!collidesWithBoard(this.state.board, rotatedBlock, rotatedPosition)) {
            this.state.currentBlock = rotatedBlock;
            this.state.currentPosition = rotatedPosition;
        }
    });
    this.onMessage("move", (client, message: Movement) => {
        const nextPosition = new Position(
            this.state.currentPosition.row + message.row,
            this.state.currentPosition.col + message.col
        );
        if (
            !isLeftOutOfBounds(this.state.board, this.state.currentBlock, nextPosition) &&
            !isRightOutOfBounds(this.state.board, this.state.currentBlock, nextPosition) &&
            !isBottomOutOfBounds(this.state.board, this.state.currentBlock, nextPosition) &&
            !collidesWithBoard(this.state.board, this.state.currentBlock, nextPosition)
        ) {
            this.state.currentPosition = nextPosition;
        }
    });
}

At this point, we should be able to play a game of Tetrolyseus. And if we open our frontend multiple time, we’re also already able to move and rotate our block from multiple sessions!

If you want to jump straight to this point, you can check out tag 07-game-loop.

git checkout tags/07-game-loop -b 07-game-loop

Multiplayer?

Now that we’re able to actually play Tetrolyseus, there’s one question left:

What’s the multiplayer approach?

Tetrolyesues implements a multiplayer mode which allows one player to only move a block and the other one is only able to rotate it. We will keep a list of current players and assign them the respective player type:

export enum PlayerType {
    MOVER,
    ROTATOR
}

export class Player {
    constructor(public readonly id: string, private _ready: boolean, private readonly _type: PlayerType) {
    }

    public get isReady(): boolean {
        return this._ready
    }
    public set isReady(isReady: boolean) {
        this._ready = isReady;
    }
    public isMover(): boolean {
        return this._type === PlayerType.MOVER;
    }
    public isRotator(): boolean {
        return this._type === PlayerType.ROTATOR;
    }
}

Our room holds a map of players

playerMap: Map<string, Player>;

and this map will be used in both onJoin and onLeave handlers:

onJoin(client: Client, options: any) {
    if (!this.playerMap.size) {
        const playerType = Math.random() >= 0.5 ? PlayerType.MOVER : PlayerType.ROTATOR;
        this.playerMap.set(client.id, new Player(client.id, false, playerType));
    } else {
        if (this.roomHasMover()) {
            this.playerMap.set(client.id, new Player(client.id, false, PlayerType.ROTATOR));
        } else {
            this.playerMap.set(client.id, new Player(client.id, false, PlayerType.MOVER));
        }
    }
}

onLeave(client: Client, consented: boolean) {
    this.playerMap.delete(client.id);
}

This map will be used to limit actions to the respective player in our onMessage handlers:

this.onMessage("move", (client, message: Movement) => {
    if (this.playerMap.has(client.id)) && this.playerMap.get(client.id).isMover()) {
        ...

this.onMessage("rotate", (client, _) => {
    if (this.playerMap.has(client.id) && this.playerMap.get(client.id).isRotator()) {
        ...

The first joining player will be assigned to be a MOVER or ROTATOR at random, the other player will take the other role.

Are We Ready Yet?

Until now our game loop started with the creation of our room. This imposes a bit of a problem for the first joining player, being only able to either move or rotate a block.

To mitigate this circumstance, let’s add a running flag to our GameState:

@type("boolean")
running: boolean;

Additionally, we’re gonna introduce a new message type, ReadyState:

export interface ReadyState {
    isReady: boolean;
}

export const READY = {
    isReady: true
}

export const NOT_READY = {
    isReady: false
}

The message handler for our ReadyState will update our Players’ state and once all roles have been assign and every player is ready, we will start the game loop:

onCreate(options: any) {
    ...
    this.onMessage("ready", (client, message: ReadyState) => {
        if (this.playerMap.has(client.id)) {
            this.playerMap.get(client.id).isReady = message.isReady;
        }

        if (this.roomHasMover() && this.roomHasRotator() && this.allPlayersReady()) {
            this.state.running = true;
            this.startGameLoop();
        }
    });
}

Our frontend will display a modal to set yourself ready:

<div class="nes-container is-dark with-title">
    <p class="title">Tetrolyseus</p>
    <p>A cooperative approach to the famous blocks game.</p>
</div>
<div id="ready-modal" class="nes-container is-rounded is-dark with-title">
    <p class="title">Ready to play?</p>
    <label>
        <input id="ready" type="radio" class="nes-radio is-dark" name="answer-dark" checked />
        <span>Yes</span>
    </label>

    <label>
        <input id="not-ready" type="radio" class="nes-radio is-dark" name="answer-dark" />
        <span>No</span>
    </label>
</div>
<div id="playingfield">
...

A click on one of the buttons will send the respective ReadyState message to our backend:

document.addEventListener('DOMContentLoaded', async () => {
    ...

    const readyModal = queryReadyModal();
    const readyButton = queryReadyButton();
    const notReadyButton = queryNotReadyButton();

    readyButton.addEventListener("click", () => room.send("ready", READY));
    notReadyButton.addEventListener("click", () => room.send("ready", NOT_READY));

    room.onStateChange((newState: GameState) => {
        if (newState.running) {
            if (!(typeof document.onkeydown === "function")) {
                document.addEventListener('keydown', handleInput);
            }
            readyModal.style.display = "none";
            renderGame(newState);
        } else {
            document.removeEventListener('keydown', handleInput);
        }
    });
});

Once the game is running, the modal will be hidden and the game is on!

If you want to check out the game right away, use tag 08-multiplayer.

git checkout tags/08-multiplayer -b 08-multiplayer

Ready To Ship?

That’s it, we’re ready to get our game out there!
One last thing to do are some additional scripts to create an application bundle for easier shipping. Let’s extend our package.json:

"scripts": {
  ...
  "build:backend": "tsc -p tsconfig.json",
  "build:frontend": "parcel build frontend/index.html",
  "clean": "rimraf ./dist ./app",
  "bundle": "npm run clean && npm run build:backend && npm run build:frontend && ncp dist/ app/public"
  ...
  },

We can instruct our backend express instance to also serve our frontend by adding the following config in backend/index.ts:

const app = express()

const staticPath = join(__dirname, '../public');
console.log(`Using static path '${staticPath}'`);
app.use(express.static(staticPath));

app.use(cors());

Running npm run bundle will create an application bundle in app:

app
├── backend
├── messages
├── public
└── state

The last tag to check out is 09-app-bundle.

git checkout tags/09-app-bundle -b 09-app-bundle

Summary

In this post we built a fully working multiplayer game from scratch without caring all too much about networking. Colyseus really keeps it out of our way and allows us to fully focus on our game!
Since great gameplay is what gets people hooked to our games, this is a really nice solution for building online multiplayer games!

Where To Go From Here?

Colyseus has a lot more to offer than we covered here.
Some of the things we didn’t touch so far are:

Social log-in
Password protected rooms
Configuring rooms
Handling drop-outs / reconnets

Another thing we could extend our game with would of course be a highscore list. Lot’s of space for improvements!

An iOS 14 widget for TravisCI

Simon Hofmann — Sun, 25 Oct 2020 15:22:03 +0000

I recently learned that it's possible to build iOS 14 widgets with Scriptable, a really useful app to automate things on iOS with JavaScript.

To give it a hands-on, I've built a Travis CI
widget which displays build states for a repository.

It supports all sizes of widgets and is available at https://github.com/s1h-org/scriptable-travis-widget

nut.js - Two years recap

Simon Hofmann — Mon, 31 Aug 2020 19:26:46 +0000

nut.js - Two years recap

A little over two years ago I began to evaluate possibilities to perform desktop automation with Node.js. I compared various existing libraries but at the end of the day, none of them really convinced me. They all had their pros and cons, but none of them met all of my requirements, which where:

The library is actively maintained
Fast and easy to install
Fully cross-platform compatible
Provides image matching capabilities

While the first three requirements could be met, the fourth one ruled out every single lib I was checking out. It seemed like no desktop automation library for node provided image matching capabilities - and that was when I decided to build one myself.
Two years later and I’m still actively maintaining nut.js, so I thought it might be a good time for a recap on what happened within these two years.

The Early Days - native-ui-toolkit

The first prototype I pieced together received the working title native-ui-toolkit. It combined robot-js for OS level interactions (grabbing screen content, keyboard / mouse input, clipboard access) with opencv4nodejs for image matching. Despite being an initial working prototype, this first draft already revealed some major issues.
robot-js only supported node up to version 8, which was quite a bummer with node 10 becoming the latest LTS version on Oct. 30th 2018. Additionally, development seemed to have stalled (and as I have checked today, there has been no new release since March 2018).
The second major problem came with opencv4nodejs. It required either a properly installed version of OpenCV on your system, or alternatively gives you the possibility to let opencv4nodejs compile OpenCV for you. Both options did not play well with my requirement of a fast and easy installation. Either have the user struggle with installing the correct version of OpenCV, which is not equally easy on all platforms, or have the lib recompile OpenCV on installation, which requires a full C++ toolchain and takes 30+ minutes.
Last, but no least, both opencv4nodejs and robot-js are native node add-ons. So essentially we’re dealing with shared libraries here, which means they have to be provided for every target platform, and depending on the technology used, the targeted node version. Since both libs did not provide a way to provide pre-built binaries out of the box, the only solution at that time was to recompile them on install. This in turn required a C/C++ toolchain and a working Python 2 installation. Again, not my kind of ”fast and easy installation”.
However, facing these problems lead to one of the most substantial design decision regarding the architecture of nut.js.
Instead of spreading 3rd party packages throughout the library, dependencies are limited to so called “provider packages“, which deal with library specifics. From there on, only user-defined types are used, fully hiding any external dependency.

These providers are used in an adapter layer, where they can be mixed and matched to implement desired functionality. User-facing API only relies on these adapters. This way a new / different provider wont ever be able to require changes in the user-facing API and changes are limited to the adapter layer (at most). It may sound like a glorious example of over-engineering at first, but in hindsight it proved to be one of the best design decisions I made with nut.js. Following this scheme I changed the native provider implementations three times so far, requiring minimal effort.

Growing Up - Moving Out

What started as a single repository under my GitHub account was about to grow into a dedicated organization with its own repo. I made plans on how to continue development on nut.js and decided to group upcoming repos under the nut-tree GitHub organization. Prior to moving the repo I ditched robot-js in favor of robotjs, a similar lib which provided pre-built binaries, so it didn't need to be built on install.
With its own organization and repo, nut.js also earned its own logo:

The one thing I was still struggling with was how I could provide a ready to use package of opencv4nodejs. As already mentioned, installing the correct version of OpenCV can be tedious and my understanding of great usability required some way to install the lib without including a compile step. So in addition to shipping a pre-compiled version of OpenCV I also had to provide pre-compiled bindings for various platforms and node versions, since opencv4nodejs uses nan for its bindings.

Building on-top of what it already provided I forked both opencv4nodejs and npm-opencv-build. I didn't require all of OpenCV, so I dug into it's build config until it fit my needs and started configuring CI pipelines. When running on CI, platform specific packages containing a pre-compiled version of OpenCV should be published following the @nut-tree/opencv-build-${process.platform} scheme. A first step in the right direction.

These OpenCV wrapper packages paved the way for shipping fully pre-built OpenCV bindings. My fork of opencv4nodejs, opencv4nodejs-prebuilt would install OpenCV libs for the current target platform and link against them during build. After reading and learning *a lot* about subtle differences regarding linking on macOS, Linux and Windows, I modified the opencv4nodejs build process in a way which made it possible to ship pre-compiled node bindings including the correct, pre-built OpenCV lib. Thanks to Travis CI and Appveyor I’m able to run a total of currently 39 jobs to pre-build these bindings, supporting node versions >= 10 as well as Electron >= 4 on three platforms.

Continuous Change

One big problem solved, time for another one! Node 12 was about to become the new LTS version, so naturally, my goal was to support node 12 as well. However, development of robotjs stalled. The original maintainer seemed to have move on and there hasn’t been a proper release since 2018.

Facing this issue, I decided to take care of it myself and forked the project. Once I got aquatinted with the project, I realised that node 12 support meant more than just an updated CI setup for pre-builds. Since robotjs also used nan for its bindings, it required code changes to stay compatible with node 12.
Given this fact, I decided to make the leap and migrated from nan to N-API. Along this way I also switched build systems, replacing node-gyp with cmake-js. The result of all these changes is libnut, which supports future node versions out of the box due to N-API's ABI stability.

After migrating to libnut, the current dependency tree of nut.js looks like this:

Continuous Improvement

Now that nut.js had a solid foundation, it was time for improvements.
In a first step, I added documentation.
Besides an improved readme file, I also added auto-generated API docs which are hosted via GitHub pages.

The next thing I tackled were pre-releases.
Every push to develop now triggers a pre-release build which publishes a development release to npm.
Whenever a new tag is pushed, a stable release will be published.
Stable releases are available under the default latest tag, development releases are published under the next tag.
Along the way I constantly improved the CI setup to make my life easier.

As a third improvement, I added a samples repository.
This monorepo holds several packages which demonstrate various use-cases of nut.js.
Samples range from keyboard and mouse interaction to Jest and Electron integration.

New Shores

So far, nut.js wrapped clipboardy, libnut and opencv4nodejs-prebuilt in a (in my opinion) nice API.
Initially, libnut was just a port of robotjs, so it provided exactly the same functionality.
And since I wanted libnut to be truly cross-platform compatible, new features had to either work on all platforms or wouldn’t be added, so when I started working on adding a desktop highlighting feature, I suddenly found myself digging through Xlib, Win32 and AppKit documentation, writing C/C++ as well as Objective-C/Objective-C++.
An exciting experience which really made me smile like a little kid on Christmas Eve once I saw windows appearing on each platform!

Exactly the same thing happened when I added support for window interactions.
Being able to determine open windows and their occupied screen region paves the way for additional features, which again made me smile in front of my machine!
Testing this feature also really made me appreciate the JavaScript ecosystem.
What seemed like a pretty tough task at first could be achieved with a single implementation for all platforms by starting an Electron application on the fly during a test.
A single test now verifies my native implementation works on every platform - isn’t this awesome?

Conclusion

So, how are things going after two years?

To be honest, I still enjoy working on nut.js very much!
I automated quite a lot early on, so now I’m able to concentrate on features and bugfixes.

I’m also still happy with my API design decisions.
API design is hard, and I guess you can’t make everybody happy, but I’m enjoying it myself, so that’s fine for me!

I’ve been able to fulfill my own requirement of a fast and easy install on every platform by spending some extra time on a pre-build setup, something I’m still proud of today!

As I already mentioned, automation plays a major role for nut.js.
But not only did I automate a lot of things, I also spent time working on a proper test infrastructure using multiple CI sytems and multi-staged pipelines to ensure I do not break things.
Being able to release fast and with confidence is really worth the invest!

Last, but not least, I’ve been super excited once I noticed RedHat picked up nut.js for their vscode-extension-tester.
Call me a fanboy, but seeing a company I’d known for almost 20 years for their Linux distribution and open source work starting to use my framework is quite a thing for me!

Two years and still going strong! 💪

GitHub Actions SSH Deploy Setup

Simon Hofmann — Thu, 16 Jul 2020 15:10:28 +0000

Lately I’ve been deploying a bunch of static webpages to my server.
Even though there are several offers out there which seamlessly integrate with GitHub and others, I still like this machine and I’m not going to get rid of it anytime soon.

The one thing I dislike, though, are manual deployments.
So, after I manually deployed the first page, I began to automate this task.
And since I’m hosting most of my code on GitHub, I started looking for solutions using GitHub Actions.

Preparations — Generate SSH Key

Before we take a look at the actual GitHub action we’re going to generate an SSH key:

ssh-keygen -m PEM -t rsa -b 4096 -C "you@your_mail.tld"

Our SSH key is required to be in PEM format, hence the -m PEM flag.

Preparations — Copy SSH Key

To log into our machine using our SSH private key we need to add our public key to the machine’s authorized_keys.
We’ll do so using ssh-copy-id:

ssh-copy-id -i /path/to/your/key youruser@yourhost

This will copy the public key of your key pair to the remote host and add it to its list of authorized_keys.

Action — Configure Secrets

We definitely don’t want to expose our private key required for login to our GitHub repository, so let’s configure some secrets for our deployment:

DEPLOY_KEY: Our SSH private key
DEPLOY_HOST: The host we’re going to connect to
DEPLOY_USER: Our user on the remote host
DEPLOY_PORT: The port we’re using. (Not necessarily required, port 22 will be used by default)
DEPLOY_TARGET: The target folder on the remote host we’re copying our assets to

Action — Add a Workflow File

With our secrets all set up, we’re ready to configure our workflow.
The following workflow is using easingthemes/ssh-deploy:

name: SSH Deploy

on:
  push:
    branches: [ master ]

jobs:
  deploy:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2
      - name: ssh deploy
        uses: easingthemes/ssh-deploy@v2.1.4
        env:
          SSH_PRIVATE_KEY: ${{ secrets.DEPLOY_KEY }}
          REMOTE_HOST: ${{ secrets.DEPLOY_HOST }}
          REMOTE_USER: ${{ secrets.DEPLOY_USER }}
          REMOTE_PORT: ${{ secrets.DEPLOY_PORT }}
          SOURCE: "dist/"
          TARGET: ${{ secrets.DEPLOY_TARGET }}

It will copy the content of our SOURCE folder to the target folder on our remote host. Our SOURCE is freely configurable, so we’re able to deploy just about anything via SSH.

Commit this workflow file in your repository under .github/workflows/deploy.yaml to get things started.

Shipping node addons with libraries

Simon Hofmann — Mon, 13 Jul 2020 15:47:48 +0000

The thing with linked 3rd-party libraries is that you'll have to provide locations to search for them.node addons are dynamically linked, shared libraries, so whenever you import an addon, the respective dynamic loader will have to load all required libraries.

How can we check which libraries we require?

macOS: otool -L your_addon.node
Linux: ldd you_addon.node

In order to load these required libraries, our loader is going to search in several places, e.g. in all paths listed in LD_LIBRARY_PATH on Linux.

But we're shipping our own lib, which is not in one of the standard search paths?

It is possible for a lib or executable (ELF format on Linux or Mach-O format e.g. macOS) to specify a runtime loader path.
These paths are hardcoded into the binary and may specify additional paths to search for libraries.

How can we retrieve a binaries `RPATH`?

macOS: otool -l your_addon.node | grep RPATH -A2
Linux: objdump -x your_addon.node | grep RPATH

Ok, `rpath` it is!

We can configure our rpath via linker flags:

"conditions": [
            ["OS==\"mac\"",
              {
                "link_settings": {
                  "libraries": [
                    "-Wl,-rpath,@loader_path",
                    "-Wl,-rpath,@loader_path/..",
                  ],
                }
              }
            ],
            ["OS==\"linux\"",
              {
                "link_settings": {
                  "libraries": [
                    "-Wl,-rpath,'$$ORIGIN'",
                    "-Wl,-rpath,'$$ORIGIN'/.."
                  ],
                }
              }
            ]
        ],

`$$ORIGIN`? `@loader_path`? What's this?

rpath entries are hardcoded, so having it fixed to e.g. /home/youruser/libs/foo/bar/ will break as soon as you try to use your addon on a different machine.
Both $ORIGIN and @loader_path are token which our dynamic loader is going to replace with the directory containing our binary. So no matter where our library will be installed, if we specify paths relative to our binaries location, the dynamic loader will be able to find it. ('$$ORIGIN' is just a little workaround required so node-gyp doesn't mess things up when trying to substitute values)
A good read regarding this topic is this article

Example

Our build generates the ./build/Release/addon.node file, which is linked against ./lib/my_library.dylib at build time.
Using @loader_path we can now specify an rpath relative to our binaries location, since @loader_path will be replaced with /whatever/path/to/our/package/build/Release.

{
    "link_settings": {
        "libraries": [
            "-Wl,-rpath,@loader_path/../../lib",
         ],
    }
}

At runtime, this will result in /whatever/path/to/our/package/build/Release/../../lib, exactly where our lib is located.

What about Windows?

Windows binaries do not have / use an rpath property.
However, the DLL search order will start searching in the directory from which the application is loaded.

A cross-platform approach

My approach to shipping required libraries is as follows:

Link your libraries during build
Copy your libraries to the resulting output directory, e.g. build/Release
Set the rpath to either $ORIGIN or @loader_path on Linux and macOS

This way, we're instructing the dynamic loader on Linux and macOS to search for our lib in the same folder as our resulting binary, which is the default behaviour on Windows.

Copying files can be done via an additional target in our gyp file:

{
    "target_name": "action_before_build",
    "type": "none",
    "copies": [{
        "files": [ "<!@(node ./lib/bundle.js)" ],
        "destination": "<(PRODUCT_DIR)"
    }]
}

Debugging GitHub Actions

Simon Hofmann — Mon, 22 Jun 2020 11:26:55 +0000

Have you ever spent 20+ commits to debug a CI setup? Yes?

EDIT: As pointed out by follow redditors, sshing into a GitHub actions runner might cause trouble with GitHubs TOS.

A better alternative to this approach is act, a handy CLI tool which allows us to run our Actions workflow locally. It uses Docker containers to replace build machines (only Linux is supported at the moment), so we're still able to debug our workflow without having to play the endless commit ping-pong.

~~Wanted to ssh into the build machine to check some things?~~

~~We were facing the same situation today and were happy to discover mxschmitt/action-tmate@v2.~~

~~This action allows you to ssh into the runner executing your workflow to manually verify your build.~~

It will install and start tmate, which allows you to join the running terminal session, so it's possible to either connect va ssh or web terminal. Connection details will be printed in the command output.

~~mxschmitt/action-tmate@v2 allowed us to identify the root cause of our problem and we were able to fix our build quickly, a huge win in terms of debuggability!~~

Webhook Triggers for GitHub Actions

Simon Hofmann — Thu, 18 Jun 2020 13:01:11 +0000

External Triggers for GitHub Actions

With GitHub Actions we have a really powerful tool at our hands to automate our workflows on GitHub.
Actions are deeply integrated with GitHub and allow us to run CI / CD pipelines, automate tasks to manage issues or to connect our repo with third-party services.
Workflows are triggered on events which are caused by certain activities on GitHub, e.g. a push to a repository or the creation of a new issue.
However, GitHub may not be our only source of events to trigger a workflow, so let’s see how we can trigger a workflow via web hook.

repository_dispatch

A repository_dispatch event is GitHub’s way to trigger a workflow from outside of GitHub. Every repository_dispatch event comes with an event_type property which allows us to limit our workflow execution to certain events. By default, every web hook call would trigger a workflow run.

Supported events are listed in our workflow file on the repository_event trigger:

name: webhook-trigger

on:
  repository_dispatch:
    types: 
      - webhook-one
      - webhook-two
      - your-custom-event-type

Webhook Trigger

repository_dispatch events are created via GitHub API.
We’re required to provide an event_type property, which corresponds to one of our configured repository_dispatch types above. It is also possible to pass additional payload via client_payload JSON object. This way we’re able to process additional data in our workflow.

Issuing a repository_dispatch event requires a personal access token with repo scope to call the GitHub API:

curl -H "Authorization: token <YOUR_GITHUB_TOKEN>" \
    --request POST \
    --data '{"event_type": "<YOUR_EVENT_TYPE>"}' \
 https://api.github.com/repos/<YOUR_GITHUB_USER>/<YOUR_GITHUB_REPO>/dispatches

Processing Event Types

Let’s assume we want to carry out different workflow steps, depending on the event_type. Conditionals and context expressions enable us to do so.
The github context gives us full access to the repository_dispatch payload, which stores our event_type value as action property.

So we’re able to bind workflow steps to the trigger event type:

- name: Event Name
  run: |
    echo "Event triggered by $GITHUB_EVENT_NAME"
    echo "Dispatch type ${{ github.event.action }}"
- name: Goodbye
  if: ${{ github.event.action == 'webhook-one' }}
  run: npx cowsay "Bye!"
- name: Adiós
  if: ${{ github.event.action == 'webhook-two' }}
  run: npx cowsay "Adiós!

The client_payload is also accessible via context expression, e.g.

${{ github.event.client_payload.unit }}

Conclusion

It took me a bit to put all pieces together, but once all dots were connected, GitHub Actions integrated smoothly in existing CI infrastructure via web hooks.
A small sample is available in this repo.
However, given the large amount of events we can process via GitHub Actions there is a huge potential yet to explore!

Test Driven Development for Docker Images

Simon Hofmann — Mon, 08 Jun 2020 14:13:43 +0000

Containers were one of the driving forces behind the modern DevOps culture. Where there used to be a strict barrier between developers and operations, is now a common, agreed upon standard. Developers are able to configure their runtime environment to their needs, focusing on what happens inside a container, IT-Ops takes care of things outside a container - how to run, scale and monitor them.
While the concept of process isolation has been around for much longer, the one thing which brought container technology to the masses was Docker in 2013. Gone should be the days when developers handed over some kind of deliverable and IT-Ops had to somehow take care of it. No more “but it works on my machine” - if it works on your machine, we will ship your machine!

With great power comes great responsibility

Given the ability to customise the runtime environment, responsibility for providing and maintaining a proper environment shifts from IT-Ops towards developers. We’re not only responsible for our application, but also the environment to run it in. Fortunate enough, we can apply well-known methods to make this effort less tedious: automated tests and test-driven development (TDD).

Meet Goss and DGoss

Goss is a neat little utility which helps you validate your server configuration.
As stated on its GitHub page it’s a YAML based alternative to Ruby-based serverspec. (And since it’s written in Go, I assume Goss is short for Go serverspec)

So, how will Goss help us test our Docker images? With Goss itself it's possible to verify a multitude of properties on Linux systems, ranging from filesystem layouts, permissions and content over network ports, installed packages for various distributions to running services. This would allow us to e.g. verify the content of our Apache config, that our apache2 service is actually enabled and running and last but not least, whether it’s listening on port 80.

Now when building a Docker image, we can apply these things too. DGoss is a wrapper script around Goss which allows us to target Docker containers, so let’s explore how we can write Docker tests.

Installation

As mentioned earlier, Goss and DGoss currently only support Linux systems. Since we want to target Linux containers, this is not a problem for us. We can get the latest release on GitHub:

curl -L https://github.com/aelsabbahy/goss/releases/latest/download/goss-linux-amd64 -o /usr/local/bin/goss
chmod +rx /usr/local/bin/goss

curl -L https://github.com/aelsabbahy/goss/releases/latest/download/dgoss -o /usr/local/bin/dgoss
chmod +rx /usr/local/bin/dgoss

In case you should decide to download Goss to a location outside your PATH, you’ll have to provide an additional environment variable GOSS_PATH, which points to the executable.

First Glance

To get a first impression, we will create a short test for the httpd Docker image. If not specified otherwise via GOSS_FILE environment variable, the default spec file is called goss.yaml:

port:
  tcp:80:
    listening: true
    ip:
    - 0.0.0.0
http:
  http://localhost:
    status: 200
    body: ["It works!"]

This exemplary test verifies that we have a server running on TCP port 80, listening on all IP addresses. Additionally, it makes sure it serves the default page which only displays ”It works!”.

To run our tests we execute dgoss run httpd instead of docker run httpd. Dgoss will start a container based on our specified image and copy all required files for us. Assuming the image under test works like expected we should see them succeed:

INFO: Starting docker container
INFO: Container ID: 76715e17
INFO: Sleeping for 0.2
INFO: Container health
INFO: Running Tests
Port: tcp:80: listening: matches expectation: [true]
Port: tcp:80: ip: matches expectation: [["0.0.0.0"]]
HTTP: http://localhost: status: matches expectation: [200]
HTTP: http://localhost: Body: matches expectation: [It works!]


Total Duration: 0.010s
Count: 4, Failed: 0, Skipped: 0
INFO: Deleting container

Test Driven Docker Development (TDDD)

Now the above example is not that interesting, let’s instead focus on building an image for e.g. a node backend. We will be using a Debian base image, debian:10.4-slim to be precise, and change the default shell to bash for our build.

FROM debian:10.4-slim

SHELL [ "/bin/bash", "-c" ]

Next to our Dockerfile we create a goss.yaml file as well as a little helper script to build and test our image, build_and_test.sh.

#!/usr/bin/env bash

set -e

IMAGE=s1hofmann/node-runtime
TAG=12.18.0

docker build -t $IMAGE:$TAG .
dgoss run -it $IMAGE:$TAG

The repository linked at the end of the post also contains a nodemon setup to automatically build and test the image on file changes.

Non-root images

The first thing we want to add to our image is a dedicated non-root user and group to run our app, so let’s add both a group and user test to our Goss file.

group:
  node:
    exists: true
    gid: 1000
    skip: false

user:
  node:
    exists: true
    uid: 1000
    gid: 1000
    groups:
    - node
    home: /home/node
    shell: /bin/bash
    skip: false

Since we did not yet apply any changes to our base image, our tests will fail:

+ IMAGE=s1hofmann/node-runtime
+ TAG=12.18.0
+ docker build -t s1hofmann/node-runtime:12.18.0 .
Sending build context to Docker daemon  4.096kB
Step 1/2 : FROM debian:10.4-slim
 ---> 108d75da320f
Step 2/2 : SHELL [ "/bin/bash", "-c" ]
 ---> Using cache
 ---> 39a73e053958
Successfully built 39a73e053958
Successfully tagged s1hofmann/node-runtime:12.18.0
+ dgoss run -it s1hofmann/node-runtime:12.18.0
INFO: Starting docker container
INFO: Container ID: ae8fbadc
INFO: Sleeping for 0.2
INFO: Container health
INFO: Running Tests
Group: node: exists:
Expected
    <bool>: false
to equal
    <bool>: true
Group: node: gid: skipped
User: node: exists:
Expected
    <bool>: false
to equal
    <bool>: true
User: node: uid: skipped
User: node: gid: skipped
User: node: home: skipped
User: node: groups: skipped
User: node: shell: skipped


Failures/Skipped:

Group: node: exists:
Expected
    <bool>: false
to equal
    <bool>: true
Group: node: gid: skipped

User: node: exists:
Expected
    <bool>: false
to equal
    <bool>: true
User: node: uid: skipped
User: node: gid: skipped
User: node: home: skipped
User: node: groups: skipped
User: node: shell: skipped

Total Duration: 0.003s
Count: 8, Failed: 2, Skipped: 6
INFO: Deleting container

Once we extended our image, our tests should pass:

FROM debian:10.4-slim

SHELL [ "/bin/bash", "-c" ]

RUN groupadd --gid 1000 node && useradd --uid 1000 --gid node --shell /bin/bash --create-home node

+ IMAGE=s1hofmann/node-runtime
+ TAG=12.18.0
+ docker build -t s1hofmann/node-runtime:12.18.0 .
Sending build context to Docker daemon  4.096kB
Step 1/3 : FROM debian:10.4-slim
 ---> 108d75da320f
Step 2/3 : SHELL [ "/bin/bash", "-c" ]
 ---> Using cache
 ---> 39a73e053958
Step 3/3 : RUN groupadd --gid 1000 node && useradd --uid 1000 --gid node --shell /bin/bash --create-home node
 ---> Using cache
 ---> 8700babb2f3b
Successfully built 8700babb2f3b
Successfully tagged s1hofmann/node-runtime:12.18.0
+ dgoss run -it s1hofmann/node-runtime:12.18.0
INFO: Starting docker container
INFO: Container ID: 4f21c178
INFO: Sleeping for 0.2
INFO: Container health
INFO: Running Tests
Group: node: exists: matches expectation: [true]
Group: node: gid: matches expectation: [1000]
User: node: exists: matches expectation: [true]
User: node: uid: matches expectation: [1000]
User: node: gid: matches expectation: [1000]
User: node: home: matches expectation: ["/home/node"]
User: node: groups: matches expectation: [["node"]]
User: node: shell: matches expectation: ["/bin/bash"]


Total Duration: 0.003s
Count: 8, Failed: 0, Skipped: 0
INFO: Deleting container

With our first tests passing we still want to make sure our image actually switches to the created user. A command test can be used to verify this:

command:
  container_user:
    exec: "id -un" 
    exit-status: 0
    stdout:
    - node
    skip: false
  container_group:
    exec: "id -gn" 
    exit-status: 0
    stdout:
    - node
    skip: false
  container_pwd:
    exec: "pwd" 
    exit-status: 0
    stdout:
    - /home/node
    skip: false

As expected, our new tests will initially fail, but once we extend our Dockerfile, they should pass again:

FROM debian:10.4-slim

RUN groupadd --gid 1000 node && useradd --uid 1000 --gid node --shell /bin/bash --create-home node

USER node:node
WORKDIR /home/node

Count: 14, Failed: 0, Skipped: 0

Installed Packages

Our upcoming node setup requires either of wget or curl. It is certainly not required to keep them in our image, but since curl has been really useful more than once, I like to have it pre-installed. Let’s add a package test for it:

package:
  curl:
    installed: true
    skip: false

To fix our now failing test, let’s also update our Dockerfile:

FROM debian:10.4-slim

SHELL [ "/bin/bash", "-c" ]

RUN groupadd --gid 1000 node && useradd --uid 1000 --gid node --shell /bin/bash --create-home node

RUN apt-get update \
    && apt-get install -y curl \
    && rm -rf /var/lib/apt/lists/*

USER node:node
WORKDIR /home/node

node

nvm turned into my go-to solution to install node. When installing node using nvm, we want to make sure that

A .nvm folder is located in our home directory
A node version corresponding to our image tag is installed at $HOME/.nvm/versions/node
Our node version is set as default in $HOME/.nvm/alias/default

A possible way to verify this is a file test which uses variables:

file:
  /{{ .Env.HOME }}/.nvm:
    exists: true
    owner: node
    group: node
    filetype: directory
  /{{ .Env.HOME }}/.nvm/versions/node/v{{ .Env.NODE_VERSION }}:
    exists: true
    owner: node
    group: node
    filetype: directory
  /{{ .Env.HOME }}/.nvm/alias/default:
    exists: true
    owner: node
    group: node
    filetype: file
    contains: [/^{{ .Env.NODE_VERSION }}$/]

The following additions to our Dockerfile are required to pass our latest test:

ARG NODE_VERSION=12.18.0
ENV NODE_VERSION $NODE_VERSION
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh | bash && chmod +x $HOME/.nvm/nvm.sh && . $HOME/.nvm/nvm.sh && nvm install $NODE_VERSION && nvm alias default $NODE_VERSION && nvm use default

Startup

Our image should provide an entrypoint script which either forwards all non-command parameters to node, or executes any other command so we’re able to start our container running e.g. a bash shell.
To make sure we're providing an entrypoint script we’re going to extend our file test:

file:
  ...
  /usr/local/bin/entrypoint.sh:
    exists: true
    owner: node
    group: node
    filetype: file

And with our entrypoint in place, we expect a running node process in our container:

process:
  node:
    running: true

In order to pass our last tests we have will have to copy our script and configure an entrypoint:

COPY --chown=node:node entrypoint.sh /usr/local/bin

ENTRYPOINT [ "entrypoint.sh" ]
CMD [ "node" ]

Conclusion

In this post we built a node Docker image the test-driven way. Instead of manually validating our setup at runtime we were able to write tests for our setup upfront, which made it easier for us to verify our image configuration.

The resulting image is configurable, so we’re free to choose our runtime, but still comes with a safety net which makes sure everything is in place. Best practices known from software development also apply to building Docker images!

The whole setup can be found on GitHub, if you have any questions, feel free to get in touch with me!

Inside node: How node is able to require binary modules

Simon Hofmann — Wed, 29 Jan 2020 23:46:57 +0000

This post has previously been publish on my blog at s1h.org

When talking about node modules, we mostly refer to “vanilla” JS modules. They are written in plain JavaScript, we can easily access their sources and they are easy to distribute. In summary: they are great! But in some occasions, we are hitting the boundaries of what’s doable with JavaScript in terms of performance, connectivity or platform use. One way to cope with these limitations are native node addons.

Addons are dynamically-linked shared objects written in C++. The require() function can load Addons as ordinary Node.js modules. Addons provide an interface between JavaScript and C/C++ libraries.

But in contrast to plain JS modules, native addons are compiled binaries. So how is possible to seamlessly require a binary module?

Over the years several ways of writing native addons established, with nodes N-API being the latest one. As cited above, native addons are (mostly) written in either C or C++, which opens a set of additional possibilities. We’re able to re-use existing high-performant C or C++ libraries for increased performance or wrap a specific low-level driver while keeping the expressiveness of our language of choice, JavaScript! This combination, the best of both worlds, sound very promising. By building a native node addon we just have to do a

const native_module = require("/my/module.node");

and we have native performance at our hand while writing JavaScript code.

const solution = require(“./investigation.node”)

The first component involved in loading our native addon is the require() function, which is provided through the CommonJS module loader. We won’t go into all the details of Module loading here, the thing we’re most interested in at the moment is the fact, that require() will call Module.load(), providing the path to a *.node native addon.
Depending on the file extension, Module.load() will hand off the actual loading process to one of the available extensions. The *.node extension in lib/internal/modules/cjs/loader.js looks like this:

// Native extension for .node
Module._extensions['.node'] = function(module, filename) {
  if (manifest) {
    const content = fs.readFileSync(filename);
    const moduleURL = pathToFileURL(filename);
    manifest.assertIntegrity(moduleURL, content);
  }
  // Be aware this doesn't use `content`
  return process.dlopen(module, path.toNamespacedPath(filename));
}

process.dlopen sounds a lot like dlopen(3) - Linux man page, so I guess we’re onto something! process.dlopen is provided through nodes internalBinding mechanism, the implementation behind it is located in src/node_binding.cc.

The heart of this method is a call to env->TryLoadAddon, which receives a callback to perform the actual loading process.

env->TryLoadAddon(*filename, flags, [&](DLib* dlib) {
…
});

Before we go any further from this point, let’s also take a look at a small sample addon to use for our experiments.

N-API module - Sample application

Instead of building a dedicated N-API sample for this post I’ll refer to a sample a friend of mine built for an introductory talk to N-API development at the MNUG - Munich NodeJS User Group.
This sample provides a native implementation of a square() function:

module.c:

#include <node_api.h>

napi_value square(napi_env env, napi_callback_info info) {
  napi_value argv[1];
  size_t argc = 1;
  napi_get_cb_info(env, info, &argc, argv, NULL, NULL);

  double value;
  napi_get_value_double(env, argv[0], &value);

  napi_value result;
  napi_create_double(env, value * value, &result);

  return result;
}

napi_value init(napi_env env, napi_value exports) {
  napi_value square_fn;
  napi_create_function(env, NULL, 0, square, NULL, &square_fn);

  napi_set_named_property(env, exports, “square”, square_fn);

  return exports;
}

NAPI_MODULE(square, init)

index.js:

//const {square} = require(‘bindings’)(‘square’);
const {square} = require(‘./build/Debug/square.node’);

console.log(square(4));

As we can see, we just require the compiled *.node file and are able to call our native square function.

Inside module.c, the following things happen:

napi_get_cb_info(env, info, &argc, argv, NULL, NULL); stores a list of arguments to our square function in an array:

napi_value argv[1];

Next, we store the first element of this list as a double value:

double value;
napi_get_value_double(env, argv[0], &value);

The result of square will be stored in an napi_value and returned.

napi_value result;
napi_create_double(env, value * value, &result);

return result;

Dynamic Loading

Since node addons just happen to be dynamic shared libraries, the four major requirements to handle dynamic libraries are:

Opening a library
Handling possible errors
Retrieving addresses of symbols
Closing an opened library

On POSIX systems, these tasks are handled via dlopen, dlerror, dlsym and dlclose. Within node, class DLib in src/node_binding.h encapsulates this functionality and if we take a look at its methods, we see that DLib::Open, DLib::Close and DLib::GetSymbolAddress are using mentioned functions.

bool DLib::Open() {
  handle_ = dlopen(filename_.c_str(), flags_);
  if (handle_ != nullptr) return true;
  errmsg_ = dlerror();
  return false;
}

void DLib::Close() {
  if (handle_ == nullptr) return;

  if (libc_may_be_musl()) {
    return;
  }

  int err = dlclose(handle_);
  if (err == 0) {
    if (has_entry_in_global_handle_map_)
      global_handle_map.erase(handle_);
  }
  handle_ = nullptr;
}

void* DLib::GetSymbolAddress(const char* name) {
  return dlsym(handle_, name);
}

For non-POSIX systems wrappers provided by libuv (uv_dlopen etc.) will be used, but functionality stays the same.

Connecting the strings

Being able to open a library, retrieve symbol addresses and close it again are a first steps to native module loading. However, there are still some things to resolve until we’re able to use our module, which is done in the callback function provided to env->TryLoadAddon:

[&](DLib* dlib) {
    // Skipped

    const bool is_opened = dlib->Open();

    node_module* mp = thread_local_modpending;
    thread_local_modpending = nullptr;

    if (!is_opened) {
      // Error handling, closing the lib
        // Skipped
    }

    if (mp != nullptr) {
      if (mp->nm_context_register_func == nullptr) {
        if (env->options()->force_context_aware) {
          dlib->Close();
          THROW_ERR_NON_CONTEXT_AWARE_DISABLED(env);
          return false;
        }
      }
      mp->nm_dso_handle = dlib->handle_;
      dlib->SaveInGlobalHandleMap(mp);
    } else {
      // Skipped
    }

    // -1 is used for N-API modules
    if ((mp->nm_version != -1) && (mp->nm_version != NODE_MODULE_VERSION)) {
      // We’re dealing with N-API
        // Skipped
    }
    CHECK_EQ(mp->nm_flags & NM_F_BUILTIN, 0);

    // Do not keep the lock while running userland addon loading code.
    Mutex::ScopedUnlock unlock(lock);
    if (mp->nm_context_register_func != nullptr) {
      mp->nm_context_register_func(exports, module, context, mp->nm_priv);
    } else if (mp->nm_register_func != nullptr) {
      mp->nm_register_func(exports, module, mp->nm_priv);
    } else {
      dlib->Close();
      env->ThrowError("Module has no declared entry point.");
      return false;
    }

    return true;
  }

In summary, this rather longish function takes care of following things:

It opens the native addon via DLib::Open()
Handles loading errors
Accesses the loaded module via thread_local_modpending
Calls modules register function

But how is it possible to access module data via thread_local_modpending? After a call to DLib::Open() thread_local_modpending holds the modules address without a call do dlsym, so there’s got to be an additional loading mechanism.

NAPI_MODULE()

The N-API sample shown earlier contains the following line:

NAPI_MODULE(square, init)

NAPI_MODULE is a macro defined in src/node_api.h. It receives a module name and the name of an init function. When expanded, this macro results in the following snippet:

extern “C” {
  static napi_module _module = {
    1,
    flags,
    __FILE__,
    regfunc,
    “square”,
    priv,
    {0},
  };

  static void _register_square(void) __attribute__((constructor));
  static void _register_square(void) {
    napi_module_register(&_module);
  }
}

This expanded macro will create a new N-API module:

typedef struct {
  int nm_version;
  unsigned int nm_flags;
  const char* nm_filename;
  napi_addon_register_func nm_register_func;
  const char* nm_modname;
  void* nm_priv;
  void* reserved[4];
} napi_module

The interesting part here is what follows:

static void _register_square(void) __attribute__((constructor));
static void _register_square(void) {
  napi_module_register(&_module);
}

_register_square is a function to, well, register our native addon called “square”. It passes our modules address to napi_module_register, which will wrap its data in a node::node_module and pass it on to node_module_register in src/node_binding.cc. This is how our modules address ends up at thread_local_modpending.
But still, there’s no call to _register_square, so how do things add up here?
The answer to this question is the constructor function attribute present on _register_square. Like a constructor in object oriented programming, methods with this attribute are called automatically on “instantiation”. As soon as we open our native addon via DLib::Open, _register_square will be called automatically and our modules address is stored in thread_local_modpending before execution continues. The above applies to non-Windows plattforms, but there's a similar concept which applies to Windows.

What’s left to do is calling our modules register function via
mp->nm_context_register_func(exports, module, context, mp->nm_priv);. The register function returns our modules exports and ultimately we’re able to use our native addon.

Loading native addons turns out to be quite interesting. It requires knowledge about compiler features, platform specific library handling and some time to dig into it, but in the end it has been a fun thing to do!

Let's see what will be the next thing we will take a closer look at!

DEV Community: Simon Hofmann

Migrating a non-trivial Travis CI pipeline to GitHub Actions

The existing pipeline

Let’s migrate!

Sonar stage

Test stage

Deploy stage

To be resolved?

Conclusion

Responsive on-screen keys

First try — Size dependent on-screen keys

Next try — Pointer device

Demo

Conclusion

You don’t have to build a to-do app ...

How to play

Where to play?

Building a multiplayer game with colyseus.io

Getting started - Colyseus Backend Setup

index.ts

MyRoom.ts

Managing Game State

Stateful Game Rooms

Position

Board

Tetrolyso

GameState

Working With Game State - Frontend

Connecting To The Backend

Game Rendering

Receiving State Updates

Handling Player Input

Frontend Wrap-Up

Working With Game State - Backend

Initialising State

Scoring

Detecting Collisions

Making Our Game Work - Game Logic

Making Our Game Run - Game Loop

Multiplayer?

Are We Ready Yet?

Ready To Ship?

Summary

Where To Go From Here?

An iOS 14 widget for TravisCI

nut.js - Two years recap

nut.js - Two years recap

The Early Days - native-ui-toolkit

Growing Up - Moving Out

Continuous Change

Continuous Improvement

New Shores

Conclusion

GitHub Actions SSH Deploy Setup

Preparations — Generate SSH Key

Preparations — Copy SSH Key

Action — Configure Secrets

Action — Add a Workflow File

Shipping node addons with libraries

How can we check which libraries we require?

How can we retrieve a binaries RPATH?

Ok, rpath it is!

$$ORIGIN? @loader_path? What's this?

Example

What about Windows?

A cross-platform approach

Debugging GitHub Actions

Webhook Triggers for GitHub Actions

External Triggers for GitHub Actions

repository_dispatch

Webhook Trigger

Processing Event Types

Conclusion

Test Driven Development for Docker Images

Meet Goss and DGoss

Installation

First Glance

Test Driven Docker Development (TDDD)

Non-root images

Installed Packages

How can we retrieve a binaries `RPATH`?

Ok, `rpath` it is!

`$$ORIGIN`? `@loader_path`? What's this?