DEV Community: Suman Sarkar

Gitlab CI/CD + NodeJs + pm2

Suman Sarkar — Fri, 14 Jan 2022 16:16:59 +0000

✋ Hi this is Suman Sarkar, a web-dev from Kolkata with 5 years of experience in programming and little to none experience with CI/CD. Today I'll talk about how to setup Gitlab CI/CD with self hosted runners.

👉 Things we will cover in this article

😕 What is CI/CD?
👍 Setup a minimal expressjs API with pm2
🧑‍💻 Setup our first ever Gitlab pipeline to install & restart our server whenever an update is pushed on the “dev” branch
🏃 Install self-hosted runners on a linux server
🔒 Register our local runner to Gitlab
🔑 Add environment variables to Gitlab

😕 What is CI/CD?

From my perspective CI/CD or Continuous Integration & Continuous Deployment are processes that you set up for your own convenience so that you don't have to do boring things manually over and over, it is basically automating your workflow when you push an update to your project. Most of us do git pull and then sort of restart the server in order to make the changes into effect, there might be additional steps like building or testing and few other procedures that are specific to your project. I’ll not cover these today, today I’ll only cover how to setup CI/CD for an expressjs application with pm2, Gitlab pipeline and self-hosted runners.

👍 Setup a minimal expressjs API with pm2

We start with creating a directory for our Node JS Express API



mkdir node-cicd-pm2
cd node-cicd-pm2

Then we initialise our project with npm init -y. This creates a package.json file in our project folder with basic information for our project.
Next we add our dependencies by running



npm i –save express dotenv

Lets create our very minimal server by creating our index.js and pasting the below mentioned code.



const express = require('express');
const dotenv = require('dotenv');

const app = express();
dotenv.config();

app.get('', (req, res) => {
    res.status(200).send('Hello World!');
})

app.listen(process.env.PORT, () => {
    console.log(`Server is running on port http://localhost:${process.env.PORT}`);
})

Here, we have required our dependencies express and dotenv then we have added a route that returns 'Hello World!'. We have also added a .env file with only 1 variable.



PORT="3001"

and ecosystem.config.js file with the following content



module.exports = {
    apps: [{
        name: "node-cicd-pm2",
        script: "./index.js"
    }]
}

This will be used later to start our server as a process.

Now, we start our server by running node index.js and visit http://localhost:3001/. It works on my machine! 😈

🏃 Setup our first ever Gitlab pipeline

We start with creating a file specifically named .gitlab-ci.yml. This is an YML file, if you don't like YML, bad news for you, but you can just copy paste and get things done.
Now, paste the following code. I'll explain this in detail.



stages:
  - build_stage
  - deploy_stage

Lets talk about stages, stage are the necessary steps that you can group and describe. We have 2 stages build_stage and deploy_stage. Though we are not building anything here but I like to call it the build stage where we'll install the dependencies. We will cover the deploy stage later.



.base-rules:
  rules:
    - if: '$CI_COMMIT_BRANCH == "dev"'
      when: always
    - if: '$CI_PIPELINE_SOURCE == "push"'
      when: never
    - if: $CI_COMMIT_TAG
      when: never

Rules are to describe exactly when your pipeline should run. Here we are specifying that we want to run our pipeline whenever something is pushed onto dev branch by specifying when to always.
$CI_PIPELINE_SOURCE is a special(pre-defined) env. variable provided by Gitlab. It describes the mode our change. These can be the following values push, web, schedule, api, external, chat, webide, merge_request_event, external_pull_request_event, parent_pipeline, trigger, or pipeline. For the same of this article I'll not cover all of them, I am not familiar with most of them anyway.
You can read more about the variables here on Gitlab.

Next up we have caches. The way every stage works is, it cleans or deletes everything a it has produce during its lifetime. In the build stage we will create a node_modules folder which will contain our project's dependencies. When the build_stage is finished we don't want it to be deleted. We want it to passed to the deploy_stage



cache: &global_cache
  key: $CI_COMMIT_REF_SLUG
  policy: pull-push
  paths:
    - node_modules/
    - package-lock.json

We have created a global cache policy here. The policy is pull-push meaning that the stages using this cache policy can pull from global cache and can push to it as well. In order to create new caches with every update, we must provide a slug or an unique identifier. Here we are using $CI_COMMIT_REF_SLUG variable for that. Notice how we are specifying that we only want to cache node_modules directory and package-lock.json since these are the outputs that are generate with npm install.

Let's now define our build_stage



build:
  stage: build_stage
  extends: .base-rules
  script:
    - npm i
  cache:
    <<: *global_cache
    policy: push
  tags:
    - local_runner

The build_stage extends the base_rule so that it will run only when something is pushed on the dev branch.
In this stage we don't want to pull anything from the global-cache, we just want to push the node_modules directory and package-lock.json file in the global-cache. We will cover tags later int this article.

Later we have the deploy_stage



deploy:
  stage: deploy_stage
  extends: .base-rules
  script:
    - "pm2 start ecosystem.config.js"
  cache:
    <<: *global_cache
    policy: pull
  tags:
    - local_runner

In this stage we are pulling the cache from global-cache and then starting our server with pm2 start command. By pulling the cache we get our node_modules directory with our project dependencies.

If you have followed correctly, you should end up with a file with these content



stages:
  - build_stage
  - deploy_stage

.base-rules:
  rules:
    - if: '$CI_COMMIT_BRANCH == "dev"'
      when: always
    - if: '$CI_PIPELINE_SOURCE == "push"'
      when: never
    - if: $CI_COMMIT_TAG
      when: never

cache: &global_cache
  key: $CI_COMMIT_REF_SLUG
  policy: pull-push
  paths:
    - node_modules/
    - package-lock.json

build:
  stage: build_stage
  extends: .base-rules
  script:
    - "node --version"
    - npm i
  cache:
    <<: *global_cache
    policy: push
  tags:
    - local_runner

deploy:
  stage: deploy_stage
  extends: .base-rules
  script:
    - "pm2 start ecosystem.config.js"
  cache:
    <<: *global_cache
    policy: pull
  tags:
    - local_runner

💻 Install self-hosted runners on a linux server

A little bit of background on runners, runners are like workers who does something that a computer should do. Like executing any commands or installing your project dependencies. Behind the scene they are docker containers provided by Gitlab. By default Gitlab uses a Ruby container but you can specify your container type. In this article though we will not use Gitlab's runners, we will install our own runner which is an open-source application made by Gitlab and maintained by the dev community. Self hosted runners are completely free so you don't have to worry about money 🤑.

Installing the runner on your server is easy, you just have to run few commands. Visit this page for instruction related to your OS environment. I'm running Ubuntu 20.10 so I'll follow with GNU/Linux Binary guide.. If you are using any debian machine then follow me.. Fire up your terminal and run the following commands..



sudo curl -L --output /usr/local/bin/gitlab-runner "https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64"
sudo chmod +x /usr/local/bin/gitlab-runner
sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash
sudo gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner
sudo gitlab-runner start
sudo gitlab-runner status

Step by step we get the binary, give it executable permissions, create a user called gitlab-runner to run the runners process and then start our gitlab-runner service. The gitlab-runner user is created for security purpose so that it doesn't run as root user. It is generally advised by people who are smarter than me and have was more knowledge about operating systems 😅.
Now, after the last command you should see something like this
Again, it worked on my machine so I'm good! 😅. We are not done with this step though.. We have to login as the gitlab-runner user and install node, npm and pm2. I could not find any reference to what is the default password of gitlab-runner user so I will just reset it using the passwd command.



passwd gitlab-runner

Setup your new password and login as the gitlab-runner user by running su gitlab-runner
For install node I'm using nvm. Just follow the same process mentioned below and you should have everything you need.



curl https://raw.githubusercontent.com/creationix/nvm/master/install.sh | bash
source ~/.bashrc

this should install nvm in you machine.
Next, we install node and pm2 globally,



nvm install 16.13.2
npm i -g pm2

🔒 Register our local runner to Gitlab

We are almost done with our setup..
Now, we need to register our runner to Gitlab, to do this go to Setting > CI/CD in your repository and expand the "Runners" section.
At the left side you should see "Specific runners" section.
The token should look something like this "fy7f3BqhVzLq3Mr-xxxx"
In your local machine or wherever you have installed you runner just run



sudo gitlab-runner register

This should prompt you to specify an instance URL. Type https://gitlab.com and press enter.
Then paste the registration token that you found on Gitlab and press enter, next provide a description for your runner
the most important step, providing a tag for your runner or tags. In the .gitlab-ci.yml file I had mention the tags as local_runner so I will put that here. You can add multiple tags separated by comma but that's not mandatory. Tags will identify the runners to do their job. At last choose shell as the executor. The End? Not yet! :'(

🔑 Add environment variables to Gitlab

Now we need to add env variable to Gitlab CI/CD section so that the we can provide a PORT to our application. This is important because .env file is not commited to your version control. We add our env variable PORT under Setting > CI/CD > Variables section and we add the variable as protected. Next, super important - we need to make our dev branch as protected branch. Otherwise it won't fine the variables. You can do this from Settings > Repository > Protected branches section in your repo.

✅ That is it, we are done with our pipeline setup. If everything is done correctly, when you commit a change on your dev branch it should trigger a pipeline with 2 job and you runner should start the pm2 process at 3001 port.

Thanks for reading this article 🧑‍💻 If you face any problems, let me know in the comments down below! 🙂
Happy hacking!

How to use JsDoc annotations with VsCode for intellisense - PART 1 👨‍💻😎

Suman Sarkar — Sun, 19 Sep 2021 16:08:54 +0000

The Problem 🤦‍♂️

For many of us JavaScript devs, we love the fact that TypeScript exists. It has type hinting, type checking, helps with intellisense and many more. As a JavaScript developer you can start using TypeScript right now. But there is a catch that you cannot use TS in your favourite project without re-writing it and making it compatible for TS. But what if you want these goddies without going through the struggle of re-writing your entire application?

The Solution 👨

We can utilise JsDoc with VSCode to get all these feature without going through the hassle. For those of you who are not familiar with JsDoc and VsCode, JsDoc is an API documentation generator for JavaScript and VSCode or Visual Studio Code is microsoft's lighter version of it's legendary IDE Visual Studio. VSCode has excellent support for many programming languages and if you do not like using products that are managed by Microsoft then you will be happy to realise that VSCode is open source but if you still do not want any customisation made by Microsoft then you can use Code OSS

Let's get started already 💁‍♂️

@param

Now, Lets write it using JSDoc specs

Excelente! 👍 You have typehinting in VSCode with the help of JsDoc.

@typedef

Now, lets write another example with custom data types. Here we'll work with moment JS

Here, we are using @typedef to define custom type definations, in this case type Moment which is provided in the "moment" library and using it in the @param annotation to get type hinting. Notice when I type startDate, it suggest methods coming with a moment object.

How do I create a custom type? 🤷‍♂️

Its easy, you just have to know a little bit of TypeScript. Let me show you the directory structure.. It looks like this

src
┣ controllers
┃ ┗ post.controller.js
┗ models
┃ ┣ post.model.d.ts
┃ ┗ post.model.js

The controller method is utilising PostModel and PostDocument type definations and suggesting the properties that are available.

The interfaces defined in post.model.d.ts defines your types and helps with suggestions. This is very useful because in mongoose static methods and schema properties does not appear normally in suggesions. So from now on you can import any type in your project and utilise it's definations.

Note: In the example of moment js we saw that type definations were provided in the library itself but in case if it is not then chances are you will find the type definations in the npm repository. For example you can install type definations for the bcrypt library on @types/bcrypt

Enforcing correct types 🙅‍♂️

In the moment JS example we passed 2 argument to getDiff function. How do we make sure that when executing this function we only pass 2 moment js object and not anything else. Well, there are 2 ways

@ts-check

@ts-check enables errors in your JavaScript files. In order to use it in a JavaScript file, you need to add it at the top of the file.

Notice how on line #17 VSCode is complaining that the type of the first argument passed is not correct.

VSCode Implicit Project Config ⚙

You can enable this globally as well in your JS project by toggling

"js/ts.implicitProjectConfig.checkJs": true

That is it, now you can utilise JSDoc and VSCode together for type hinting. Thanks for reading tutorial. We are hoping to update more tutorials like these very soon.

Tools used in this tutorial

Visual Studio Code (IDE)
JsDoc (API documentation generator)
Peek (Screen recorder)
VSCode theme - GitHub Dark Default

Part 2

Part 2 of this tutorial is updated here - How to use JsDoc annotations with VsCode for intellisense - PART 2 👨‍💻😎
In part 2 we discuss about @callback, @class, @constructor, @private and @public.

From author

Before you leave, I just want to thank you for reading this article 🖤. Me and my friend started working on our own blog thalava.com
Please visit our blog for more tutorials. We are excited to share more tutorials.
Thanks again, love from thalava.com 🖤