How to make a Continuous Deployment server

I wanted to write this guide for a while because DevOps is one of the things that is not discussed much yet there are a few straightforward actions that you can integrate into your workflow that will make your developer life much easier.

I'm by no means a DevOps expert, I'm just sharing what stuck with me over the last year of experimenting with it.

Here are a few reasons why you should try it:

1. You can achieve a significant productivity boost.
2. Once everything is set, it doesn't require much attention.
3. It feels amazing every time you push code.

In this series: I will talk about continuous delivery (CD), reporting, error logging, and Github Actions. Good thing is that you can choose for yourself what you need and ignore the rest, as the pieces are mostly independent.

I will use Javascript for project examples and all the services that we going to make. Oh, and we also will need a VPS to deploy our server. You can use your own computer though.

We will start by making a simple CD server that deploys your code each time you commit to master. There are two parts:

1. Setting up CD server
2. Configuring push Webhooks

Setting up CD server

Note: this code is based on node-cd by Nikita Kolmogorov.

I won't go into the basics of setting a server up. If you have questions on that, you can refer to this guide, written by yours truly.

Our goal here will be to create a simple server that runs a Shell script each time it receives a message from outside. That script will download recent codebase from Github, install new dependencies (if any), and then restart the app via pm2.

First, we will need to make sure that the webhook request is authentic. Thing is, knowing the URL at which we expect to receive a webhook, anyone can send a request, thus being able to restart our app at his will. We want to give that power only to GitHub.

One way to solve that is to check the sender's IP and match it with a known list of GitHub addresses. That should work, but it's not a bulletproof solution, as IPs might change over time.

We will use a much more robust solution: cryptography, specifically HMAC. HMAC, or hash-based message authentication code, is a way to sign a message with a secret key. In essence, it concatenates a message and a secret to hash the result. Since a slight change of input will drastically change the hash, the only way to produce "correct" hash is to know the secret key.

For HMAC, we will need to generate a key which we will provide to GitHub. GitHub will sign all webhook requests with that key. In our server code, once a request is received we calculate the hash ourselves and compare it to what we got. If two hashes are identical, it means that the sender knows the key and therefore it's indeed GitHub that sent the request.

HMAC doesn't encrypt a message, though. So if someone will be able to intercept that message from GitHub, he will be able to see that you pushed a commit to the repository. It's not a big deal for us, but you should be careful if you're going to use HMAC for something confidential.

Alright, enough talking, let's write some code. We will start with two helper functions that will deal with HMAC.

const secret = process.env.GITHUB_SECRET;

function createComparisonSignature(body) {
    const hmac = crypto.createHmac('sha1', secret);
    const bodyString = JSON.stringify(body);
    const bodySignature = hmac.update(bodyString).digest('hex');
    return `sha1=${bodySignature}`;
}

function compareSignatures(signature, comparisonSignature) {
    const source = Buffer.from(signature);
    const comparison = Buffer.from(comparisonSignature);
    return crypto.timingSafeEqual(source, comparison);
}

Function createComparisonSignature calculates a hash and compareSignatures compares our hash and what we got from the request. We will need to import crypto which is a built-in Node.js module that deals with, you guessed it, cryptography.

Also, note the const secret part. You will need to create a .env file and put your GitHub key there.

const crypto = require('crypto');

In our router code, we will get the key, check it using the functions above, and act based on that check.

const signature = req.header('X-Hub-Signature');
const comparisonSignature = createComparisonSignature(req.body);

if (!compareSignatures(signature, comparisonSignature)) {
    console.log('Bad signature');
    res.status(403).end();
    return;
}

As you can see, if we got an invalid key, we simply send 403 and drop the request. If the hash is correct, we continue…

Now, the next step is optional, but it's really simple and might make things more readable. What we will do is to map the repository name with an "internal" project name. Best to see it in the code:

const projects = {
    'project-abc-server': 'abc',
    'project-xyz-backend': 'xyz',
};

const repository = req.body.repository.name;
const project = projects[repository];

Now we can refer to our projects as abc and xyz in the code, which will be handy later. Also, we can keep a list of "approved" projects and throw 400 status code if it's something we didn't expect:

if (!project) {
    console.log('Project not found');
    res.status(400).end();
    return;
}

Finally, the magic part: we execute a Shell script based on the project that was updated. We will start with a helper function that can run any script:

function execScript(project, filePath) {
    if (!fs.existsSync(filePath)) {
        return;
    }

    const execCallback = (error, stdout, stderr) => {
        if (error) {
            console.log(`Failed to deploy ${project}`);
            return;
        }

        if (stderr.length > 0) {
            console.log(`Failed to deploy ${project}`);
            return;
        }

        console.log(`Deployed ${project}`);
    }
    childProcess.execFile(filePath, execCallback);
}

Here, we again leverage Node.js API, namely fs and child_process to check the file existence and execute a binary file, respectively. We log the result of the execution to the console.

Note: npm warnings are treated as errors and are written to stderr. This means that if your project misses a description or repository URL, you will receive a 'Failed to deploy' error even if your script technically executes as it should.

And here's how we use execScript function:

const scriptPath = `./projects/${project}.sh`;
console.log(`Executing task at: ${scriptPath}`);
execScript(project, scriptPath);

res.status(200).end();

As for the script itself, it usually boils down to this:

cd ~/app/directory/
git pull -q
npm install
pm2 restart app

And that's it! Wrap it into express.js boilerplate and you will get the simplest possible CD server!

Configuring push Webhooks

What's left is to tell GitHub about all the beauty we created.

In your project's repo, go to Settings -> Webhooks and click Add webhook. There, you will need to paste the URL of the server we created in the previous step, as well as the secret key. I would also set Content-Type to application/json, but that's up to you.

Once you hit Add Webhook, GitHub will send a test request to your server, so you should see that in the app's logs. Also, GitHub will show you a response status code from the CD server, so if you got 200 it means everything should work fine.

Wrapping up

Here we first set up a simple yet powerful server for continuous deployment. It works awesome for simple workflows (npm install && pm2 restart app), but might as well contain the complex flow, as your Shell scripts can execute arbitrary logic.

We then use GitHub webhooks to trigger deployments on our server, therefore updating our app on each push.