DEV Community: Timur Badretdinov

Setting Up a Blog with Astro

Timur Badretdinov — Mon, 07 Feb 2022 01:07:33 +0000

In previous article, I went on my motivation to use Astro instead of other static site generators. Here, I will provide a guide on setting up a blog in Astro with minimal steps. I will cover the essential basics, and guide the next steps.

The fastest way

The fastest way to get started is to go to Astro New page and select a "Blog" template there. You can run the template inside a web sandbox or by cloning the git template.

To better understand how Astro works under the hood, we will now create a project from scratch.

Creating an empty Astro project

Note: before we start, I will introduce a few Astro concepts that will be used across this guide. First, there are Markdown pages, pages that have an .md extension and usually contain the blog post text. Second, there are Astro pages, pages that have an .astro extension and usually serve for unique pages (like index page). Finally, there are Astro layouts, pages that also have an .astro extension which defines how pages with similar structure (e.g. blog posts) should look like.

First, create an empty directory, initialize an NPM project, and install Astro as a dev dependency:

mkdir astro-blog
cd astro-blog
npm init -y
npm i -D astro

In your package.json, add build and dev scripts:

"scripts": {
    "dev": "astro dev",
    "build": "astro build"
},

Now, let's create an index page:

mkdir src
mkdir src/pages
touch src/pages/index.astro

Open index.astro with a text editor to make a bare bones page:

<!DOCTYPE html>
<html>
    <head>
        <meta charset="utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />
        <title>Blog</title>
    </head>
    <body>
        <main>
            <h1>Hello, Astro!</h1>
        </main>
    </body>
</html>

Finally, let's start a dev server to view the page:

npm run dev

Go to http://localhost:3000 to see the rendered page!

Creating post layout

Before we jump into making our first post, we need to create a layout for it. The way SSGs work is that for pages with similar structure (e.g. blog posts) you provide a template that will be applied to each page in that list. In Astro, layouts serve that goal. Let's create a layout:

mkdir src/layouts
touch src/layouts/post.astro

Open post.astro file to create the post layout:

---
const { content } = Astro.props;
const { title } = content;
---

<html lang="en">
    <head>
        <title>{title}</title>
    </head>
    <body>
        <slot />
    </body>
</html>

You can notice that this doesn't look like normal HTML. Let's untangle Astro layout syntax one by one.

Note: the syntax described below is similar for Astro components, layouts, and regular Astro pages (all files ending with .astro). The frontmatter part (---) is optional, and what's below that is a normal HTML with the addition of templating ({ JS_CODE_HERE }) and component slots (<slot />).

First, at the top of the layout, there is something called frontmatter script. It supports both JS and TS and also gives access to Astro.props object which provides component props. In the case of layouts, it will provide you with values that are defined in the Markdown page frontmatter. Here's an example:

title: My Astro Post
date: 2022-06-01T13:00:00

const { content } = Astro.props;
const { title, date } = content;
console.log(title); // "My Astro Post"
console.log(date); // "2022-06-01T13:00:00"

Variables defined inside the frontmatter script can be used in the layout template, as we will see further.

Second, there is a <title> tag that contains an expression instead of a constant value. Inside layouts, we can execute arbitrary JS by wrapping it into curly braces. In this example, we "print" the value of a variable defined in the frontmatter (title), so that Astro will effectively render page title.

Finally, there is a <slot /> tag. In the context of layouts, a slot provides access to the content of the underlying Markdown page. To put it simply, inserting a slot tag into any part of the layout will render the Markdown page as HTML in that place.

Making a first post

Now that we have a layout for our posts, we can start creating them. Let's make the first one:

mkdir src/pages/post
touch src/pages/post/first-post.md

Inside first-post.md will be the post content:

---
title: My first post
layout: ../../layouts/post.astro
---

# My first post

This is my first Astro post.

First, we define the post metadata in frontmatter. We also specify the page layout here.

Then, we describe the post content itself using Markdown.

If you open ‌http://localhost:3000/post/first-post, you can see your page live! Note that your page has a title (as shown in the browser tab), which means that your layout was successfully applied.

You can practice a bit and create your second and third pages by copying the first one and adding some changes. Once you save them, they will be available in your browser.

Note: Astro has file-based routing. What that means is that the way that your pages are organized inside the pages directory will translate into relative URL paths of the generated site. For example, a page that is located at src/pages/post/first-post.md will have a /post/first-post URL.

Adding a list of all posts

Currently, the only way to open a page on our site is to paste its exact URL. Thankfully, we can fix that. We will add a list of all posts to our index page.

Inside index.astro, create a frontmatter script with the following content:

const posts = Astro.fetchContent('./post/*.md');

Then, inside the template, add the following under the <h1> tag:

<ul>
    {posts.map((post) => (
        <li>
            <a href={post.url}>{post.title}</a>
        </li>
    ))}
</ul>

Here, we're using JSX templating to create a list of links each pointing to the corresponding blog post.

Now your index.astro should look something like this:

---
const posts = Astro.fetchContent('./post/*.md');
---

<!DOCTYPE html>
<html>
    <head>
        <meta charset="utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />
        <title>Blog</title>
    </head>
    <body>
        <main>
            <h1>Hello, Astro!</h1>
            <ul>
                {posts.map((post) => (
                    <li>
                        <a href={post.url}>{post.title}</a>
                    </li>
                ))}
            </ul>
        </main>
    </body>
</html>

If you open http://localhost:3000 now, you will see a link pointing to your first post. As you create more posts, they will be automatically added to the list.

Building the site

So far, we were using Astro in dev mode, where the content is served dynamically. This is great for development, but to use our site in production, we will need to build it. Inside your repository, run npm run build to generate your site and npx http-server dist to serve it. This helps preview your site just before deploying it, but generally, you will use dev mode while you are working on your site.

Deploying

Deployment is the process of publishing your site on the Internet for everyone to see it. You can self-host your site, although it's more convenient to use a hosting provider.

I will use Netlify as an example since the process of deploying is similar across many popular static site hosting services.

The simplest way to deploy your site on Netlify is to use Drop. Simply build your site and update the output folder by drag and drop. Netlify will give you a public link. Congrats, you've just deployed your site!

While Drop is useful, it's very limiting. For example, updating your site will require you to upload build artifacts each time you want to make a change. It's also doesn't support custom domains, HTTPS, prerendering, and many other features. To use all of that you will need to create a Netlify site.

First, you will need to create a repository using the git provider of your choice (e.g. GitHub) and upload your source code there. If you don't want to publish the source code of your site, that's fine, you can create a private repository.

Second, you will need to create a Netlify site and connect your git repository. During setup, set npm run build as build command and dist as publish directory.

Once Netlify will finish the build, you should be able to access your site. Anyone with the link can see it now! From there, you can add a custom domain, enable SSL, create redirect rules, and much more.

And that's it. We just went from nothing to a blog built by Astro and deployed by Netlify (or any other hosting). We erred on the side of simplicity here to focus on vital Astro concepts. In future posts, we will look at more advanced stuff.

Why Use Astro for Static Site Generation

Timur Badretdinov — Mon, 31 Jan 2022 13:22:10 +0000

In this post, I will cover the benefits of using static site generators for a blog, as well as why Astro is my site generator of choice.

Static site generators (SSGs)

When it comes to serving text-like content over the Internet, there are two choices: static and dynamic serving. Static serving means generating files once and then serving them over and over while dynamic serving is generating files on the fly for each request. Recently, the line between static and dynamic serving starts to blur, but the divide still stands.

In most cases, static site generators take Markdown pages and templates as input and generate HTML, CSS, and JS files as output.

Why Astro is the future of SSG

There are many, many static site generators out there. I used a few of them (namely, Jekyll, 11ty, and Hexo), and they all are decent. So why use Astro exactly?

First, I think it's rational to divide all SSG software into legacy and modern tools. If we look at most of the popular SSGs, we can see that most of them were created long before modern tooling became popular. For example, Jekyll was created in 2008, 2 years before AngularJS was released! Back then, things like modern frontend frameworks, ES5/ES6, or web components didn't even exist.

Even though legacy generators still work perfectly, I can't help but feel frustrated when using them. They do their job but have terrible developer experience. It's hard to customize site pages by adding styles, creating templates, and deciding which page is rendered by which template. There's a stark difference in using legacy site generators versus using something like React or Vue.

Speaking of modern tools for site generation, Astro is not the only "new school" generator. There are many tools that are tackling the problem, like Gatsby and Gridsome. However, I feel that most of them doing it wrong. Here are some of the features that make Astro a better choice. Many of these features are not unique to Astro, although Astro is the only generator to my knowledge that has all of them.

"Bring your own framework"

Most of the modern generators are tied to a specific front-end framework. For example, Gatsby and Docusaurus work only with React, while Gridsome and VuePress are exclusive to Vue. By untying the connection between framework and site generator, Astro allows reusing components across any framework.

Components inside Markdown

This is my favorite feature of Astro. Most of the site generators don't allow you to easily inject custom components into the page's content. By using components inside Markdown, you can customize your page with better granularity.

Vite

Let's just say that Vite is fast, really fast. Many projects start to adopt Vite and ditch Webpack, and for a good reason. Vite provides instant build, hot module reloading, doesn't require a lot of configuration, and has a rich plugin library. Astro supports Vite by default, so you can enjoy all these benefits by simply using Astro.

No JS by default

By default, Astro doesn't output any Javascript. This is the killer feature for performance. You don't need to spend time optimizing the bundle or minimizing the list of dependencies. Astro sites load fast by default.

Conclusion

In short, Astro has the performance of legacy tools with the developer experience of modern site generators. You can use React, Vue, and Svelte, but the output will include minimal or no Javascript at all.

DevOps shouldn't be hard: GitHub Actions

Timur Badretdinov — Thu, 09 Jan 2020 10:00:39 +0000

So far we were able to set up a deployment and reporting servers, making a full way from pushing a new commit to updating the app in production. But what can we automate before pushing to master? What if we run a set of checks to make sure our app is behaving correctly before deploying it? That's where GitHub Actions will come handy.

GitHub Actions is a feature that allows you to run custom checks and, well, actions each time your remote git repository changes. Two major ways to use it is to run automated checks on our codebase or use it for continuous deployment. As we do all our CD work on our own server, we're most interested in the former. That is, we will leverage GitHub Actions to run tests and other checks to make sure our codebase is OK.

There are several ways to manage your git workflow. I won't dive too much into it, but it boils down to whether you want to have feature branches, do you differentiate between develop and master, and whether you deploy your code automatically for each push. I researched this for a while, and here's what made the most sense to me. For context, I'm talking about an individual or a small team working on a small- to mid-size project.

Here's my workflow of choice:

there are two branches: develop and master
code is pushed to develop
each push triggers code checks, powered by GH Actions
assuming checks passed, new PR is created automatically
once PR is pushed to master, code is deployed
you get a notification on the check result

Setting Actions up

We don't need to configure anything to get started. Create a file under .github/workflows named nodejs.yml and commit it, and GitHub will automatically process it and show nodejs workflow under the Actions tab.

Our pipeline will consist of three jobs: build, notify, and create PR.

Build

Our build step will consist of 5 commands, running one after another.

npm audit: runs a security audit of dependencies
npm ci: makes a clean install of dependencies
npm run lint: lints your codebase (e.g. ESLint)
npm run build: builds your app (e.g. Webpack)
npm test: runs tests (e.g. Jest)

Of course, all the steps are optional. You can add your own checks as well.

Here's the full code of the build job:

build:
    runs-on: ubuntu-latest

    strategy:
      matrix:
        node-version: [8.x, 10.x, 12.x]

    steps:
    - uses: actions/checkout@v1
    - name: Use Node.js ${{ matrix.node-version }}
      uses: actions/setup-node@v1
      with:
        node-version: ${{ matrix.node-version }}
    - name: npm install, build, and test
      run: |
        npm audit
        npm ci
        npm run lint
        npm run build
        npm test

strategy.matrix allows us to test our app on multiple Node.js versions in parallel, which is handy.

Notify

Let's now send a webhook to our reporting server upon successful build. Note passing WEBHOOK_URL from repository secrets.

notify:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - name: Webhook
        uses: joelwmale/webhook-action@1.0.0
        env:
          WEBHOOK_URL: ${{ secrets.WEBHOOK_URL }}
          data: "{'app': 'my app', 'success': true}"

Once GitHub executes build, it will trigger a webhook to the specified URL so we can catch it and show some message.

Additionally, you can sign your webhook requests with JWT or HMAC (for example, using this action).

Create a PR

Finally, let's send PR to master after a successful build. Secrets with GITHUB_ prefix are provided by GitHub itself, so we don't need to do anything extra here.

master-pr:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
      - name: Create Pull Request
        uses: repo-sync/pull-request@v2.0.1
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}

The neat thing about this workflow is that you get an overview of all jobs for each PR.

Wrapping up

We made it! We managed to build an entire workflow of deploying an app.

Here's what we achieved.

Each time a new code is pushed:

The codebase is audited to make sure there are no security vulnerabilities
Build tool ensuring there are no build-time errors
Linter is run to make sure code formatted correctly
Tests are run to make sure app behaves correctly
PR is created
We receive a notification

Each time PR is merged to master:

CD server updates the app
CD server notifies reporting server
We receive a notification

In other words, all of the boring stuff is done automatically with minimal input from our, developer's, side.

I hope you enjoyed the series! From there, you can continue adding automation to your build and deploy pipelines based on your app requirements.

DevOps shouldn't be hard: reporting

Timur Badretdinov — Tue, 07 Jan 2020 09:45:11 +0000

Previously, we managed to make our app redeploy on each commit. As you can remember, we get the status of deployment by processing script execution results. So let's use it to send a notification to ourselves each time we deploy our code.

For that matter, we will create another Node.js server app on our VPS. You can extend the app that we created for continuous deployment, although I would not recommend that. Instead, we can do that Unix way, where each app does its job and does it well. Additionally, we can use the reporting server to notify us about deployments of other parts of our app, e.g. frontend.

As you can guess, we would need to implement both the client and server sides of the app. First, we will need a client (our CD server) that will send request upon successful (or failed) deployment. Second, we will make a server that will listen to those requests and send it further to the message provider of choice.

Speaking of what service to use to send those messages, that is 100% up to. Personally, I use Telegram bot to deliver messages back to me, so I will use that as an example, but this design allows using any method to deliver messages, like SMS, email, Slack, or other.

Client

As with the CD server, we will need a way to check source authenticity. This time, we will use JWT, or JSON Web Tokens to sign our messages. Also, this time we will implement both signing and verification.

Let's start by writing two helper functions that will deal with JWT generation.

function _getHash(params) {
    const paramString = JSON.stringify(params);
    const hash = crypto
        .createHash('sha256')
        .update(paramString)
        .digest('hex');
    return hash;
}

function _sign(hash, secret) {
    const payload = {
        iss: 'server',
        sha256: hash,
    };
    const token = jwt.sign(payload, secret);
    return token;
}

Here, _getHash creates a SHA256 hash of a message body, and _sign, well, signs it using a secret. Let's use it in our client.

const axios = require('axios');
const crypto = require('crypto');
const jwt = require('jsonwebtoken');

const client = axios.create({
    baseURL: 'https://our.reporting.server.url',
});

async function sendSuccess(app) {
    const params = {
        success: true,
        app,
    }
    const secret = process.env.SECRET;
    const hash = _getHash(params);
    const token = _sign(hash, secret);
    await client.post('/cd/server', params, {
        headers: {
            'X-Signature': token,
        },
    });
}

Here, we get our secret from .env file, use it to sign a request body, and then send it to our reporting server.

Few things to note:

the URL at which the reporting server is located, replace our.reporting.server.url with yours.
endpoint which we send request to; I use /cd/server as I have other sources like Netlify to receive updates from, but you can use anything, including /.
X-Signature header: again, it can be almost anything, but I would suggest sticking to something similar as this is kinda standard.

And that's our client. Let's look at the server now.

Server

Again, we start with a helper function.

function checkSignature(data, signature, secret, issuer) {
    if (signature == undefined) {
        return false;
    }
    let decoded;
    try {
        decoded = jwt.verify(signature, secret);
    } catch(e) {
        return false;
    }
    const dataString = JSON.stringify(data);
    const hash = crypto
        .createHash('sha256')
        .update(dataString)
        .digest('hex');
    const hashMatches = decoded.sha256 == hash;
    const issuerMatches = decoded.iss == issuer;
    if (!hashMatches || !issuerMatches) {
        return false;
    }
    return true;
}

Similar to the one in the article on the CD server, this checkSignature function validates that the signature is authentic.

Here's the rest of the server code.

const crypto = require('crypto');
const jwt = require('jsonwebtoken');

app.post('/cd/server', async (req, res) {
    const data = req.body;
    const signature = req.header('X-Signature');

    const secret = process.env.SERVER_SECRET;
    const issuer = 'server';
    if (!checkSignature(data, signature, secret, issuer)) {
        res.status(403).end();
    }
    const success = data.success;
    const app = data.app;
    const error = data.error;
    bot.cd('Server', app, success);
    res.send('Hello server!');
});

What we do here is check the signature and send a message. A message is sent via the provider of your choice. Here, it's Telegram bot (bot.cd('Server', app, success);).

Bonus: Netlify

As another example, let's try sending a message each time our frontend updated on Netlify.

Now, Netlify obviously doesn't need to hit our CD server, as it does CD itself. Instead, the Netlify webhook will go straight into our reporting server.

Thankfully, here we can reuse most of the code that we wrote before (Netlify uses JWT to sign webhook requests).

app.post('/cd/netlify', async (req, res) {
    const data = req.body;
    const signature = req.header('X-Webhook-Signature');
    const secret = process.env.NETLIFY_SECRET;
    const issuer = 'netlify';
    if (!checkSignature(data, signature, secret, issuer)) {
        res.status(403).end();
    }
    const success = data.state == 'ready';
    const app = data.name;
    bot.cd('Netlify', app, success);
    res.send('Hello Netlify!');
});

Here, we extract the signature from a header, match it with our locally stored key, and send a message if the signature is valid.

Note: NETLIFY_SECRET and SERVER_SECRET don't have to be different, but I highly recommend making them so. Otherwise, if one key is leaked (say, by a hacker attack on Netlify), another one will be compromised as well, making your stack less secure.

To add webhooks on Netlify, open a project, then click Settings -> Build & Deploy -> Deploy notifications, then press Add notification -> Outgoing webhook. You can add webhooks for successful or failed builds among other events.

Bonus 2: Error handling

Okay, I know you're tired by now, but there's another exciting thing I want to share with you: error logging. In other words, it will allow you to be notified each time you have an error in your app.

In essence, it's very similar to sending a request from the CD server, only this time we will send the error.

In your Node.js app, add custom error handler:

function errorWatcher(err, req, res, next) {
    if (process.env.ENV == 'dev') {
        console.log(err);
    }
    if (process.env.ENV == 'prod') {
        _sendRuntimeFailure(err.toString());
    }
    next(err);
}

async function _sendRuntimeFailure(error) {
    const app = 'my-app';
    const params = {
        app,
        error,
    };
    const hash = _getHash(params);
    const secret = process.env.SECRET;
    const token = _sign(hash, secret);
    await client.post('/runtime', params, {
        headers: {
            'X-Signature': token,
        },
    });
}

Functions _getHash and _sign are the same as we used above. We also use .env to set the ENV variable to dev or prod. That way, only production errors will be sent to you.

The only thing left is to tell express about our handler.

app.use(errorWatcher);

We will also need to wrap our async routes to make sure the error is passed to our handler.

app.get('/endpoint', wrapAsync(router.endpoint));

// Helper function to pass error down the middleware chain
function wrapAsync(fn) {
    return function(req, res, next) {
        fn(req, res, next).catch(next);
    };
}

That's it. On the reporting server side, it's 100% identical to what we used for CD server and Netlify: get signature, verify it, and send a message is signature is valid.

Wrapping up

Here, we created another microserver, this time for reporting. The server collects events from multiple sources and routes them into a single place, e.g. Telegram. We managed to send events based on our CD server, Netlify, and express.js app's error handler.

How to make a Continuous Deployment server

Timur Badretdinov — Sun, 05 Jan 2020 12:15:43 +0000

I wanted to write this guide for a while because DevOps is one of the things that is not discussed much yet there are a few straightforward actions that you can integrate into your workflow that will make your developer life much easier.

I'm by no means a DevOps expert, I'm just sharing what stuck with me over the last year of experimenting with it.

Here are a few reasons why you should try it:

You can achieve a significant productivity boost.
Once everything is set, it doesn't require much attention.
It feels amazing every time you push code.

In this series: I will talk about continuous delivery (CD), reporting, error logging, and Github Actions. Good thing is that you can choose for yourself what you need and ignore the rest, as the pieces are mostly independent.

I will use Javascript for project examples and all the services that we going to make. Oh, and we also will need a VPS to deploy our server. You can use your own computer though.

We will start by making a simple CD server that deploys your code each time you commit to master. There are two parts:

Setting up CD server
Configuring push Webhooks

Setting up CD server

Note: this code is based on node-cd by Nikita Kolmogorov.

I won't go into the basics of setting a server up. If you have questions on that, you can refer to this guide, written by yours truly.

Our goal here will be to create a simple server that runs a Shell script each time it receives a message from outside. That script will download recent codebase from Github, install new dependencies (if any), and then restart the app via pm2.

First, we will need to make sure that the webhook request is authentic. Thing is, knowing the URL at which we expect to receive a webhook, anyone can send a request, thus being able to restart our app at his will. We want to give that power only to GitHub.

One way to solve that is to check the sender's IP and match it with a known list of GitHub addresses. That should work, but it's not a bulletproof solution, as IPs might change over time.

We will use a much more robust solution: cryptography, specifically HMAC. HMAC, or hash-based message authentication code, is a way to sign a message with a secret key. In essence, it concatenates a message and a secret to hash the result. Since a slight change of input will drastically change the hash, the only way to produce "correct" hash is to know the secret key.

For HMAC, we will need to generate a key which we will provide to GitHub. GitHub will sign all webhook requests with that key. In our server code, once a request is received we calculate the hash ourselves and compare it to what we got. If two hashes are identical, it means that the sender knows the key and therefore it's indeed GitHub that sent the request.

HMAC doesn't encrypt a message, though. So if someone will be able to intercept that message from GitHub, he will be able to see that you pushed a commit to the repository. It's not a big deal for us, but you should be careful if you're going to use HMAC for something confidential.

Alright, enough talking, let's write some code. We will start with two helper functions that will deal with HMAC.

const secret = process.env.GITHUB_SECRET;

function createComparisonSignature(body) {
    const hmac = crypto.createHmac('sha1', secret);
    const bodyString = JSON.stringify(body);
    const bodySignature = hmac.update(bodyString).digest('hex');
    return `sha1=${bodySignature}`;
}

function compareSignatures(signature, comparisonSignature) {
    const source = Buffer.from(signature);
    const comparison = Buffer.from(comparisonSignature);
    return crypto.timingSafeEqual(source, comparison);
}

Function createComparisonSignature calculates a hash and compareSignatures compares our hash and what we got from the request. We will need to import crypto which is a built-in Node.js module that deals with, you guessed it, cryptography.

Also, note the const secret part. You will need to create a .env file and put your GitHub key there.

const crypto = require('crypto');

In our router code, we will get the key, check it using the functions above, and act based on that check.

const signature = req.header('X-Hub-Signature');
const comparisonSignature = createComparisonSignature(req.body);

if (!compareSignatures(signature, comparisonSignature)) {
    console.log('Bad signature');
    res.status(403).end();
    return;
}

As you can see, if we got an invalid key, we simply send 403 and drop the request. If the hash is correct, we continue…

Now, the next step is optional, but it's really simple and might make things more readable. What we will do is to map the repository name with an "internal" project name. Best to see it in the code:

const projects = {
    'project-abc-server': 'abc',
    'project-xyz-backend': 'xyz',
};

const repository = req.body.repository.name;
const project = projects[repository];

Now we can refer to our projects as abc and xyz in the code, which will be handy later. Also, we can keep a list of "approved" projects and throw 400 status code if it's something we didn't expect:

if (!project) {
    console.log('Project not found');
    res.status(400).end();
    return;
}

Finally, the magic part: we execute a Shell script based on the project that was updated. We will start with a helper function that can run any script:

function execScript(project, filePath) {
    if (!fs.existsSync(filePath)) {
        return;
    }

    const execCallback = (error, stdout, stderr) => {
        if (error) {
            console.log(`Failed to deploy ${project}`);
            return;
        }

        if (stderr.length > 0) {
            console.log(`Failed to deploy ${project}`);
            return;
        }

        console.log(`Deployed ${project}`);
    }
    childProcess.execFile(filePath, execCallback);
}

Here, we again leverage Node.js API, namely fs and child_process to check the file existence and execute a binary file, respectively. We log the result of the execution to the console.

Note: npm warnings are treated as errors and are written to stderr. This means that if your project misses a description or repository URL, you will receive a 'Failed to deploy' error even if your script technically executes as it should.

And here's how we use execScript function:

const scriptPath = `./projects/${project}.sh`;
console.log(`Executing task at: ${scriptPath}`);
execScript(project, scriptPath);

res.status(200).end();

As for the script itself, it usually boils down to this:

cd ~/app/directory/
git pull -q
npm install
pm2 restart app

And that's it! Wrap it into express.js boilerplate and you will get the simplest possible CD server!

Configuring push Webhooks

What's left is to tell GitHub about all the beauty we created.

In your project's repo, go to Settings -> Webhooks and click Add webhook. There, you will need to paste the URL of the server we created in the previous step, as well as the secret key. I would also set Content-Type to application/json, but that's up to you.

Once you hit Add Webhook, GitHub will send a test request to your server, so you should see that in the app's logs. Also, GitHub will show you a response status code from the CD server, so if you got 200 it means everything should work fine.

Wrapping up

Here we first set up a simple yet powerful server for continuous deployment. It works awesome for simple workflows (npm install && pm2 restart app), but might as well contain the complex flow, as your Shell scripts can execute arbitrary logic.

We then use GitHub webhooks to trigger deployments on our server, therefore updating our app on each push.

Deploying your Node.js app on Digital Ocean VPS

Timur Badretdinov — Fri, 27 Dec 2019 05:45:56 +0000

So you wrote your Node.js application and need a way to launch it online? Here, I present a guide on setting things up that doesn't require prior knowledge of anything OS-related.

First, let's go through the pros and cons of hosting your app on VPS.

Pros:

it's cheap
you can host multiple projects on the same server (perfect for side projects)
you have a lot of control over server administration

Cons:

it takes longer to set everything up
it adds some time to maintain your server

Ultimately, I recommend going this route if you like trying new things and want to learn some DevOps.

For the sake of this guide, you will need your app, a VPS, and a domain name.

Installing required software

Assuming you're just started with a fresh VPS, you'll likely have a clean Ubuntu. We will need to install Node.js (to run your app), nginx (to set up a reverse proxy; more on that later), and certbot (to issue an SSL certificate). You will also likely need a DB (e.g. MySQL or PostgreSQL), but that's up to you.

The easiest way to install these packages is to use apt. Look into documentation of the corresponding projects to get up-to-date instructions.

Once you're done, you will also need to install pm2, Node.js process manager:

npm install pm2

And we're done! Now let's set it all up.

Downloading and running your app

Well, assuming you already have your code hosted on GitHub, this should be easy. git clone should make its job.

One thing I should mention is that you will need to create an SSH key and link it to your GitHub. That way, your server will be able to pull your private repository without password input. It's also much faster than type password manually each time you're pulling the newest code on your server.

Let's test an app by running it.

npm install
node src/app.js

pm2

As I mentioned, pm2 is a process management tool for your Node.js apps. It can do a lot of stuff like logging your app's output, recording hardware resource usage, restarting your app in case it goes down, but right now we're interested in starting an app in "daemon" mode. In short, "daemon" means your app won't stop when you press Ctrl+C or close a console window. Instead, it will run in the background until you explicitly tell it to stop.

To start an app, run

pm2 start src/app.js --name nodejs-app

And that's it! Your app is running in the background, forever.

Now you can control your app by its name:

pm2 stop nodejs-app
pm2 start nodejs-app
pm2 restart nodejs-app

nginx

Now, nginx is a lot of things, really. Here, we will only scratch its surface by setting up a reverse proxy for our app.

So what's a reverse proxy? Here's one way to put it: reverse proxy is a middleman between client and server that transforms an external route into an internal route. Say, if nginx catches request http://app1.example.org it will route that request to an app deployed on port 3000, which is the port that we are using for our app. http://app2.example.org can route to an app on port 3001, and so forth.

Our config will include the domain we're expecting requests from, http://app.example.org, and the port at which our app is deployed, 3000.

First, create a configuration for your app:

nano /etc/nginx/sites-enabled/app.example.org.conf

Then, paste the following:

server {
    server_name app.example.org;
    listen 80;

    location / {
        proxy_pass http://localhost:3000/;
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}

Finally, test that our config is ok and restart nginx.

nginx -t
systemctl restart nginx

Now, if you actually want to test this thing, you will need a domain. The subdomain will also work fine, just replace all app.example.org above with your full domain name.

Woohoo! You (and everyone else!) should now be able to access your app by typing its URL in a browser bar.

certbot

We're almost done! The final step is to set up an SSL certificate for your app so you can access it via https.

We will do this via certbot — a CLI tool for the awesome certificate service called Let's Encrypt.

Start by simply typing

certbot

You will see a list of domains. Likely you will have only one, the one we configured above, so it will be hard to choose the wrong one. Follow the guide, as certbot will create a certificate and update your nginx configuration.

And that's it. Now, when you type your URL, it will work with https. Not only that, but it will also redirect non-secure http calls to https as well (if you chose that during configuration).

Wrapping up

I hope you didn't get lost in the process and successfully launched your app. If you found yourself spending an hour or two, don't worry: it will be much faster next time. Once you master it, you will be able to push projects to the Web in minutes!