DEV Community: Tamilarasu S

Deploy Node.js using Gitlab CI pipeline

Tamilarasu S — Wed, 07 Jul 2021 16:59:02 +0000

Prerequisite

Repository in Gitlab with Node.js project
Production server for which you have SSH access
Configure a folder in the server for a production Node.js server using steps from this digitalocean guide

Goal

Setup the Gitlab CI/CD pipeline to push the Node.js code to the server and deploy it whenever code is merged to master branch.

Step 1 - Clone the repository in the server using deploy token

Using a deploy token ensures that we can pull the code without the need to enter the credentials manually or save the credentials in the deploy script.

Create a deploy token for the repository or for the associated group using the steps mentioned in the documentation
Clone the repository in the production server's folder using the steps from the documentation. Make sure to use the correct repository url.

Step 2 - Give Gitlab access to your server

We are going to use Gitlab CI/CD Variables to save a private SSH key which Gitlab will use to authenticate with the server.

We are going to use SSH keys to authenticate rather than username and password as it is more secure.

This can be configured at a repository level or at a group level.
To view all the CI/CD variables of your repository,

Go to your project’s Settings > CI/CD
Expand the Variables section.

You can also view the same at a group level by first navigating to the group (Menu > Groups > Your Groups) and following the same steps.

If you already have a SSH_PRIVATE_KEY private key variable listed, you can skip this step.

To create a new variable, Select the Add Variable button and fill in the details:

Key: SSH_PRIVATE_KEY
Value: <ssh_private_key_details>. (To generate a new SSH public and private key pair, follow steps from this guide. Make sure to not accidentally overwrite any existing key pairs.)
Type: Variable
Choose other settings based on your needs

Click Add Variable to add the variable

Step 3 - Add Gitlab SSH public key to your server

Now, we need to add SSH public key to the list of authorized_keys in the production server.

SSH into your server (ssh root@example.com)
Add the SSH public key to authorized_keys
```
nano ~/.ssh/authorized_keys
```

3. Paste the SSH public key(starts with`ssh-rsa`) in a new line
4. Save the file

## Step 4 - Configuring Gitlab CI/CD
Gitlab looks for `.gitlab-ci.yml` in the root folder of your repository for CI/CD pipeline configurations

Add a new file `.gitlab-ci.yml` in the root folder

before_script:

apt-get update -qq
apt-get install -qq git # Setup SSH deploy keys
"which ssh-agent || ( apt-get install -qq openssh-client )"
eval $(ssh-agent -s)
ssh-add <(echo "$SSH_PRIVATE_KEY")
mkdir -p ~/.ssh
'[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'

deploy:
stage: deploy
environment:
name: production
url: https://example.com
script:
- bash deploy/deploy.sh
only:
- master

- Update the url in the above file to your domain

### Explanation
- We are using the default `ubuntu` docker package
- We install `git` package and then configure it to add our `SSH_PRIVATE_KEY`
- We also configure `StrictHostKeyChecking` to `no`, to ensure git doesn't show manual prompt during initial connection.
- We have setup pipeline named `deploy` with a single pipeline stage `deploy` which listens to commits on `master` and runs the script in `deploy/deploy.sh`

## Step 5 - Setup the deploy script
Add a new file `deploy.sh` in `deploy` folder

!/bin/bash

DEPLOY_SERVER=$DEPLOY_SERVER

echo "Deploying to ${DEPLOY_SERVER}"
ssh root@${DEPLOY_SERVER} 'bash' < ./deploy/server.sh

- Set the CI/CD variable `DEPLOY_SERVER` with value `domain.com` for the repository using step 1.

### Explanation
- We set server variable
- We ssh into our server and then execute code at `deploy/server.sh`

## Step 6 - Setup the server script
Add a new file `server.sh` in `deploy` folder

Pull code

cd /var/www/html/folder-name/
git checkout master
git pull origin master

Build and deploy

yarn install
yarn run build
pm2 restart server

- Update the `folder-name` in the above script to match the folder name used in the prerequisite step

### Explanation
- We navigate to the folder
- We get the latest changes from master branch
- We install dependencies and then optionally run build(incase of typescript project)
- Finally we restart the pm2 server

## Step 7 - Setup a Gitlab runner (One time setup)
We need a runner to run our CI/CD pipeline jobs. This step is optional if a runner is already configured for your group in Gitlab. 

To setup a new Gitlab group runner
1. Install Gitlab Runner in any server with atleast 2GB of RAM using the steps from the [documentation](https://docs.gitlab.com/runner/install/). The server should be separate from where Gitlab is installed for security reasons.
2. Go to your group's __Settings > CI/CD__ 
3. Expand the __Runners__ section.
4. Under the __Set up a group runner manually__ section, copy the url and token
5. Then register the runner in your server using steps from [documentation](https://docs.gitlab.com/runner/register/index.html)
  - Provide default image as `ubuntu` and empty tags
6. Once the runner is registered, go back to the __Runners__ section in Gitlab to see the runner appear under __Available runners__ section

Push the `.gitlab-ci.yml` and `deploy/deploy.sh`, `deploy/server.sh` files to master to start the automated deployment.

When the deployment is complete, you should see a green checkmark in the repository home page similar to 
![Pipeline Status Image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3yjjkz0hz3ihpz84rtkm.png)

You can click the status icon to go to the pipeline and then to the individual job to see the command line output of your script

## Troubleshooting
- If the job is not scheduled, make sure you have setup a runner and that the runner is active. Also check the runner parameters like `active`, `protected` and `tags` to see if any of the conditions are incorrect.

## References
- https://medium.com/devops-with-valentine/deploy-over-ssh-from-gitlab-ci-pipeline-6a0d7b87e4a
- https://medium.com/@hfally/a-gitlab-ci-config-to-deploy-to-your-server-via-ssh-43bf3cf93775
- https://codeburst.io/gitlab-build-and-push-to-a-server-via-ssh-6d27ca1bf7b4
- https://erangad.medium.com/create-ci-cd-pipeline-for-nodejs-express-with-gitlab-6c420a956b10

Build and deploy React using Gitlab CI pipeline

Tamilarasu S — Wed, 07 Jul 2021 16:34:04 +0000

Prerequisite

Repository in Gitlab with react project
Production server for which you have SSH access
NGINX setup for your domain to a folder in the server. See this tutorial on how to do it

Goal

Setup the Gitlab CI/CD pipeline to create a react production build and deploy to our server whenever code is merged to master branch. The attached steps can also be used for other SPA like Angular, Vue.js.

Step 1 - Give Gitlab access to your server

We are going to use Gitlab CI/CD Variables to save a private SSH key which Gitlab will use to authenticate with the server.

We are going to use SSH keys to authenticate rather than username and password as it is more secure.

This can be configured at a repository level or at a group level.
To view all the CI/CD variables of your repository,

Go to your project’s Settings > CI/CD
Expand the Variables section.

You can also view the same at a group level by first navigating to the group (Menu > Groups > Your Groups) and following the same steps.

If you already have a SSH_PRIVATE_KEY private key variable listed, you can skip this step.

To create a new variable, Select the Add Variable button and fill in the details:

Key: SSH_PRIVATE_KEY
Value: <ssh_private_key_details>. (To generate a new SSH public and private key pair, follow steps from this guide. Make sure to not accidentally overwrite any existing key pairs.)
Type: Variable
Choose other settings based on your needs

Click Add Variable to add the variable

Step 2 - Add Gitlab SSH public key to your server

Now, we need to add SSH public key to the list of authorized_keys in the production server.

SSH into your server (ssh root@example.com)
Add the SSH public key to authorized_keys
```
nano ~/.ssh/authorized_keys
```

3. Paste the SSH public key(starts with`ssh-rsa`) in a new line
4. Save the file

## Step 3 - Configuring Gitlab CI/CD
Gitlab looks for `.gitlab-ci.yml` in the root folder of your repository for CI/CD pipeline configurations

Add a new file `.gitlab-ci.yml` in the root folder

image: node

cache:
paths:
- node_modules/

before_script:

apt-get update -qq
apt-get install -qq git
"which ssh-agent || ( apt-get install -qq openssh-client )"
eval $(ssh-agent -s)
ssh-add <(echo "$SSH_PRIVATE_KEY")
mkdir -p ~/.ssh
'[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'

deploy:
stage: deploy
environment:
name: production
url: example.com
script:
- bash deploy/deploy.sh
only:
- master

- Update the url in the above file to your domain

### Explanation
- We are using the [Node docker image](https://hub.docker.com/_/node) as the starting point
- We are caching the `node_modules` folder to improve the speed of the build
- We install `git` package and then configure it to add our `SSH_PRIVATE_KEY`
- We also configure `StrictHostKeyChecking` to `no`, to ensure git doesn't show manual prompt during initial connection.
- We have setup pipeline named `deploy` with a single pipeline stage `deploy` which listens to commits on `master` and runs the script in `deploy/deploy.sh`

## Step 4 - Setup the deploy script
Add a new file `deploy.sh` in `deploy` folder

!/bin/bash

DEPLOY_SERVER=$DEPLOY_SERVER
SERVER_FOLDER="html-folder-in-server"

Building React output

yarn install
yarn run build

echo "Deploying to ${DEPLOY_SERVER}"
scp -r build/ root@${DEPLOY_SERVER}:/var/www/html/${SERVER_FOLDER}/

echo "Finished copying the build files"

- Update the server folder to the folder name that you have created in the production server
- Set the CI/CD variable `DEPLOY_SERVER` with value `domain.com` for the repository using step 1.

### Explanation
- We set server and folder variables
- We install dependencies and then start a new react production build using `yarn`
- Once the build is done, we copy the `build` folder to `/var/www/html/html-folder-in-server/build` location in server

## Step 5 - Setup a Gitlab runner (One time setup)
We need a runner to run our CI/CD pipeline jobs. This step is optional if a runner is already configured for your group in Gitlab. 

To setup a new Gitlab group runner
1. Install Gitlab Runner in any server with atleast 2GB of RAM using the steps from the [documentation](https://docs.gitlab.com/runner/install/). The server should be separate from where Gitlab is installed for security reasons.
2. Go to your group's __Settings > CI/CD__ 
3. Expand the __Runners__ section.
4. Under the __Set up a group runner manually__ section, copy the url and token
5. Then register the runner in your server using steps from [documentation](https://docs.gitlab.com/runner/register/index.html)
  - Provide default image as `ubuntu` and empty tags
6. Once the runner is registered, go back to the __Runners__ section in Gitlab to see the runner appear under __Available runners__ section

Push the `.gitlab-ci.yml` and `deploy/deploy.sh` files to master to start the automated deployment.

When the deployment is complete, you should see a green checkmark in the repository home page similar to 
![Pipeline Status Image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3yjjkz0hz3ihpz84rtkm.png)

You can click the status icon to go to the pipeline and then to the individual job to see the command line output of your script

## Troubleshooting
- If the job is not scheduled, make sure you have setup a runner and that the runner is active. Also check the runner parameters like `active`, `protected` and `tags` to see if any of the conditions are incorrect.
- If the job fails to produce a build output due to memory allocation issue, try increasing the memory of the server which hosts the runner. 

## References
- https://medium.com/devops-with-valentine/deploy-over-ssh-from-gitlab-ci-pipeline-6a0d7b87e4a
- https://medium.com/@hfally/a-gitlab-ci-config-to-deploy-to-your-server-via-ssh-43bf3cf93775
- https://codeburst.io/gitlab-build-and-push-to-a-server-via-ssh-6d27ca1bf7b4
- https://erangad.medium.com/create-ci-cd-pipeline-for-nodejs-express-with-gitlab-6c420a956b10