DEV Community: Ahaiwe Emmanuel

Integrate Docker Image Signing in CI Pipeline

Ahaiwe Emmanuel — Tue, 23 May 2023 21:02:22 +0000

Why Bother with Docker Image Signing?

a. Supply chain security: Ensures that the image is vetted before deployment to a production environment.

b. To pass vulnerability tests and checks: Vulnerability assessment tools used by QA engineers and other testers analyze the integrity of images to ensure it is unmodified and original. It verifies that the image is from the expected source.

c. Customer Trust: Customers and consumers of the image can have confidence in its originality and integrity.

d. Image Integrity: If anyone without access to the source tries to modify it, the signature verification would fail. Hence only you can sign your image with your chosen password or key.

As an engineer, you want to maintain the integrity of the applications you are deploying to the production environment, even when there are attacks by shady internet characters. For Docker, there is a manual way to do this by doing docker sign trust image:tag before pushing the image to your repository and subsequently, deployment. Automating this process on your continuous integration is what this article focuses on. Our tools of choice are Github and CircleCI and cosign.

Prerequisite

Integrate CircleCI with GitHub as seen here
Container repository
A Dockerfile to build your image.

Let's Dive Right In

Create CircleCI Folder Create a CircleCI job with Docker and Nodejs preinstalled into the workflow. To do this, create a .circleci folder at the root of your project. In that folder, create a config.yml file with the given configurations:

version: 2.1
jobs:
  deploy-sign-images-to-production:
    docker:
      - image: wecs/circle-ci-gcloud-node:0.1.0
    working_directory: ~/signed-images
      - run:
          name: Build and Deploy Signed Images
          command: |
            chmod u+x ./deployment.sh
            if [ "${CIRCLE_BRANCH}" == "main" ]; then
              ./deploy.sh
            fi

Create Bash Script Create a bash file called deploy.sh still in the root of your project. This will help install cosign to sign the docker image.

# !/bin/sh
set -e
set -o pipefail

# login to docker hub
echo “$DOCKERHUB_PASSWORD” | docker login -u “$DOCKERHUB_USERNAME” — password-stdin
docker build -t dockerhuburl/imagename:$CIRCLE_SHA1 .

# download cosign
curl -LO https://github.com/sigstore/cosign/releases/latest/download/cosign-linux-amd64

# give cosign executable access
chmod +x cosign-linux-amd64

# move cosign to your CI machine bin folder
mv cosign-linux-amd64 /usr/local/bin/cosign

# check cosign version and that it installed successfully
cosign version

# generate key pair using set $COSIGN_PASSWORD
cosign generate-key-pair

# push to docker hub
docker -- push dockerhuburl/imagename:$CIRCLE_SHA1

# sign image
cosign sign --key cosign.key dockerhuburl/imagename:tag:$CIRCLE_SHA1 -y

# verify image
cosign verify --key cosign.pub dockerhuburl/imagename:tag:$CIRCLE_SHA1:$CIRCLE_SHA1

Explaining Environmental Variables Created
First, we set need to set CircleCI environment variables. Read this article https://circleci.com/docs/env-vars/ for more insight. Set $DOCKERHUB_PASSWORD and $DOCKERHUB_USERNAME to your docker hub login details. You do not need to set $CIRCLE_SHA1 as it is a unique variable generated by CircleCI you can take advantage of in your pipeline. Then finally create an environment variable called $COSIGN_PASSWORD and pass any value you want to it. For this tutorial, I use 1234 as the password. This would be the password to help generate the cosign.key and cosign.pub files.
Final Review
Very importantly, we need to note that we need the image to first exist in the docker repository so we can be able to sign it. Hence, why we push the image first before signing and verifying it, don't forget to have a Dockerfile in your environment. Create a pull request and merge it to your main branch on GitHub. Switch to your project dashboard on CircleCI to monitor your build. You should have your signed image pushed to docker hub. Hurray!!!

The sequel article would touch on how to create a policy that allows only signed images to be deployed in your Kubernetes cluster.

TESTING DOCKER CONTAINER STRUCTURE

Ahaiwe Emmanuel — Fri, 20 May 2022 17:08:45 +0000

A Brief History of Docker Containers

In the early days of software development, developers ran their applications on their local machines during testing and deployed them to a production virtual machine.

The problem with this approach is that some dependencies which might work in the development environment might not work in production. Hence the famous line by engineers, "it is working on my local machine but not working in production!" Or worse, it could be working on the local machine of some engineers but not working for others. Docker addressed this problem.

Docker containers provide applications with all it needs to run in a containerized environment. Therefore, eliminating the need for applications to depend on environment-specific system requirements. As long as it runs in a development environment, it would certainly work in a production environment.

Containers in Docker are spun up from images. Images provide a convenient way to package applications with the preconfigured server environment. So, Docker images are the package and containers are the package in current use.

Testing Docker Container Structure

What if there is a way to make sure that our Docker images have all the required files and directory to run correctly, execute commands correctly and generally reduce the rate of deploying a problematic image to production?

There is a way! Google's container structure test is the answer. container-structure-test.

Let's say you have folders called src and test at the root of your application and a Dockerfile for your Node js application with these commands:



FROM node:14-alpine

# Create app directory

WORKDIR /usr/node-app

ENV PORT=3000

COPY . .

RUN npm install


EXPOSE 3000

ENTRYPOINT ["npm", "run", "start" ]

Build the Docker image with this command:



docker build -t my-container-name .

Follow these commands to install Container Structure Tests on MacOs or Linux:

OS X

Install via brew:



$ brew install container-structure-test

curl -LO https://storage.googleapis.com/container-structure-test/latest/container-structure-test-darwin-amd64 && chmod +x container-structure-test-darwin-amd64 && sudo mv container-structure-test-darwin-amd64 /usr/local/bin/container-structure-test

Linux



curl -LO https://storage.googleapis.com/container-structure-test/latest/container-structure-test-linux-amd64 && chmod +x container-structure-test-linux-amd64 && sudo mv container-structure-test-linux-amd64 /usr/local/bin/container-structure-test

If you want to avoid using sudo:



curl -LO https://storage.googleapis.com/container-structure-test/latest/container-structure-test-linux-amd64 && chmod +x container-structure-test-linux-amd64 && mkdir -p $HOME/bin && export PATH=$PATH:$HOME/bin && mv container-structure-test-linux-amd64 $HOME/bin/container-structure-test

Create a file called config.yaml at the root of your project. Write the following in that file:



schemaVersion: 2.0.0
metadataTest:
  env:
    - key: PORT
      value: 3000
  exposedPorts: ["3000"]
  # volumes: ["/test"]
  entrypoint: ["npm", "run", "start" ]
  # cmd: ["/bin/bash"]
  workdir: "/usr/node-app"
  # user: "luke"

fileExistenceTests:
  - name: 'src directory exists'
    path: '/usr/node-app/src'
    shouldExist: true
  - name: 'data directory exists'
    path: '/usr/node-app/test'
    shouldExist: true

Save the file and run this command:



container-structure-test test --image my-container-name \
--config config.yaml

This should give a successful test output.

Congratulations! You just tested your first docker container structure! You can research further ways to add it to a deployment pipeline or automate the process of testing. Thanks for your time.

An easy Approach to Algorithms (Part 1)

Ahaiwe Emmanuel — Tue, 07 Dec 2021 16:30:14 +0000

Two Pointer Technique

Top companies typically would hire you for your demonstrated problem-solving abilities. A less experienced engineer is chosen over one who is more experienced. What skill makes one stand out? How well you can solve a problem and not how many problems you have solved. Algorithms are what big tech companies like Google use to test problem-solving proficiency. You can show your world-class abilities by learning about the Two Pointer Technique, the first in a series of algorithm fundamentals. We discuss saving time and space using an optimized algorithm with the best performant big-O notation.
Two pointer technique involves using two array indexes in a sorted array. The aim is to save time and space. Typically placed at the two ends of an array, it finds pairings in optimized time. A typical question would look like this:

Example: In an unsorted array, find if a pair exists with a given sum targetSum.
A typical brute force approach would be to create a function and have a nested for loop where we compare pairs:

pairExists(array, targetSum) { 
for(let i = 0; i < array.length -1; i++){
        let firstNumber = array[i];

        for(let j = i + 1; j < array.length; j++){
            let secondNumber = array[j];

            if(firstNumber + secondNumber === targetSum){
                return [firstNumber, secondNumber];
            }
        }
    }
}

The above nested for loop approach would lead to an O(n^2) time complexity because we iterate twice in our algorithm. And while this might work, it is not optimal when we increase the size of the array to a million.

Two Pointer Technique examples

Two Number Sum:

Write a function that takes an unsorted array of distinct integers and an integer representing a target sum. If any two numbers sum up to the target sum, they are returned in an array. If no two integers sum up to the target sum, an empty array is returned.

Key points:

unsorted array
distinct integer
target sum

// o(nlog(n)) | o(1) space
function twoNumberSum(array, targetSum) {
    array.sort((a, b) => a - b);
    let left = 0;
    let right = array.length - 1;

    while(array[left] < array[right]){
        const currentValue = array[left] + array[right];
        if (currentValue === targetSum ){
            return [array[left], array[right]]
        }
        else if (currentValue < targetSum){
            left++;
        }
        else if (currentValue > targetSum){
            right--;
        }
    }
    return [];
}

First, we sort the array in O(N*log(N)), which is far better than O(n^2) in the brute force approach. Refer to this article for more information.
Then we set our pointer variables and call them left and right. We iterate from the beginning of the array at index 0 and the end of the array at array.length -1 and move the left pointer forward if we get a value lesser than the target sum and the right pointer if we get a value greater than the target sum.
Two pointer algorithm typically uses just a loop to iterate and compare values! Compared to the brute force approach of nested loops, this is quite optimal.
The while loop iterates in an O(n) time and O(1) space complexity (it doesn't create another array to check values).

Complexity
Finally, we can say our two number sum algorithm runs in O(N*log(N)) time and O(1) space algorithm because the array sort function is the highest time complexity our algorithm performs.

Three-Number Sum:

Write a function that takes an unsorted array of distinct integers and an integer representing a target sum. The function should find three numbers in the array whose sum equals the target sum. It should return a two-dimensional array sorted in ascending order per array. It should return an empty array if no three numbers that equal the target sum is found.

Key points:

unsorted array
distinct integer
target sum
return two-dimensional arrays sorted in ascending order
return empty numbers do not sum up to the target sum

// o(n^2) time | o(n) space
function threeNumberSum(array, targetSum) {
    array.sort((a,b) => a - b);
    let tripleValueArray = [];
    for (let i = 0; i < array.length - 2; i++) {
        let leftNumber = i + 1;
        let rightNumber = array.length - 1;

        while (leftNumber < rightNumber) {
            let currentNumber = array[i] + array[leftNumber] +       array[rightNumber];

            if (currentNumber === targetSum) {
                tripleValueArray.push([ array[i], array[leftNumber], array[rightNumber] ]);
                leftNumber++;
                rightNumber--;
            } else if (currentNumber < targetSum) {
                leftNumber++;
            } else if (currentNumber > targetSum) {
                rightNumber--;
            }
        }
    }
    return tripleValueArray;
}

First, we sort the array in O(N*log(N)), which is far better than O(n^3) in a brute force approach of three for loops nested in themselves.
Next, we use for (let i=0; i < array.length - 2; i++) in our loop because we always want two extra values to check with and not iterate over. Remember pointer position for a three number sum would look like this:
[-8, -6, 1, 2, 3, 5, 6, 12]
Where -8 would be the starting current number, -6 the starting left number and 12 the starting right number. We move the left pointer if the addition of all three values is less than the target sum and the right pointer to the right if it is greater than the target sum.

Remember, the array is sorted so moving from left to right or right to left increases or decreases sum value respectively. The sum of -8+(-6)+12 = -2. But if we move the left pointer from -6 to 1 and sum -8+1+12 = 5. A bigger number! Similarly, moving the right pointer from -12 would result in -8+(-6)+6 = -8. A much smaller number.

The only condition when we move both pointers towards the middle is if the sum of all three values equals the target sum if (currentNumber === targetSum). We use the conditions:
leftNumber++; and rightNumber--; to break out of the while loop. We then return whatever is pushed into tripleValueArray. If nothing is pushed, we return it because it is declared as an empty array.

Complexity
The time complexity for our three number sum is O(N^2) because we have two loops, an outer for loop and inner while loop in the algorithm.
The space complexity is O(N) because it is created in constant time. Although, we cannot tell the size of our tripleValueArray.

Four-Number Sum

Write a function that takes an unsorted array of distinct integers and an integer representing a target sum. The function should find four numbers in the array whose sum equals the target sum. It should return a two-dimensional array in no particular order. It should return an empty array if no four numbers that equal the target sum is found.

// o(n^2) time | o(n^2) space
function fourNumberSum(array, targetSum) {
    const temporaryPairSum = {};
    const quadruplet = [];

    for (let i=1; i < array.length - 1; i++){
        for(let j = i+1; j < array.length; j++){
            let currentSum = array[i] + array[j];
            let difference = targetSum - currentSum;

            if ( difference in temporaryPairSum){
                for (const arrayPair of temporaryPairSum[difference]){
                    quadruplet.push(arrayPair.concat([array[i], array[j]]))
                }
            }
        }
        for (let k=0; k < i; k++){
            let currentSum = array[k] + array[i];
            if(!(currentSum in temporaryPairSum)){
                temporaryPairSum[currentSum] = [[array[k], array[i]]];
            } else {
                temporaryPairSum[currentSum].push([array[k], array[i]]);
            }
        }
    }
    return quadruplet;

}

We use a hash table to store pair values. For this algorithm, we start our outer for loop from index 1 and iterate to array.length - 1 index. The inner for loop of the equation also start from index 1 + 1 position. But why do we do this?

We want to prevent duplication of values so we skip saving anything in our hash table temporaryPairSum during the first iteration. We only save values when we iterate the second time from index 0 while comparing the values with whatever is currently in the array index "i" as shown in this part of the equation
for (let k=0; k < i; k++).

Remember we skipped the first value in our outer for loop by starting at array index 1 here for (let i=1; i < array.length - 1; i++).

Next, we solve for the additional two arrays in the multidimensional array and subtract them from the target sum. We then check if the difference already exists in the hash table

const difference = targetSum - currentSum;
 if ( difference in temporaryPairSum)

If it does, then congratulation! We push the two array values, add them to our quadruplet multidimensional array.

The second part of the inner for loop is where the "difference" referred to is added. Pay close attention here!

We iterate starting from index 0 to where the iteration of the outer for loop is currently for (let k =0; k < i; k++). Then we check if we have initialized the sum of two array pairs (referred to as difference in the outer for loop. If it is not initialized, we do so here:
allPairSum[currentSum] = [[array[k], array[i]]];

Please note that our hash table uses the sum of two array pairs as key and a multidimensional array as value. This helps to track duplicates that can be found in the iteration. For example, our hash table with duplicates would look like this assuming 17 is the target sum difference:

{
17: "[ [array[k], array[i]], [array[k], array[i]]  ]"
}

Where duplicates would be a different arrangement of the same values.

 7 + 10 = 17 and 10 + 7 = 17:
{
17: "[ [10, 7], [7, 10]  ]"
}

We push the duplicate to the hash table using this line
allPairSum[currentSum].push([array[k], array[i]]);

The quadruplet multidimensional array is returned at the end of the algorithm. It can also be an empty array if no quadruplet is found.

Complexity
The average time complexity analysis for this is O(2N^2) which then evaluates to O(N^2). This is because, in big-O scaling, the constant of N which in this is 2 is irrelevant. The major complexity comes from the unknown size of N. The worst-case scenario for the algorithm is O(N^3).

You might also be wondering why we have just O(N^2) complexity after having about 4 for loops? This is because 2 of the inner for loops, starts just before or after the starting index of the outer for loop. If you look closely, the first inner for loop starts an index next to the outer for loop for(let j = i+1; j < array.length; j++) and the last for loop of the equation for (let k=0; k < i; k++) starts just before the outer for loop. These types of for loops evaluates to O(2N). We get O(2N^2) = O(N^2) by adding the time complexity of the outer for loop. For the worst-case scenario O(N^3), it is the time complexity used to iterate through pair duplicates in the hash table for (const arrayPair of temporaryPairSum[difference]).

The space complexity is O(n^2) as you never really know the space the hash table or the quadruplet multidimensional array might take.

To read up on Big-O notation check out this article. For further reading, please visit this link.

Big-O Notation for Coding Interviews Explained Fast

Ahaiwe Emmanuel — Tue, 23 Nov 2021 16:00:49 +0000

This definition from Wikipedia elegantly explains big-O in computer science as a classification of algorithms according to how their run time or space requirements grow as the input size grows.

It is an asymptotic analysis which implies that it is input bound. It calculates the worst-case scenario of an algorithm based on input, bounded by time and space.

The calculation for time and space is similar. In this explanation, we calculate the big O notation for time.

We would explain how to identify O(1), O(log(N)), O(N), O(N^2). It is worthy to note that the algorithms are arranged in terms of their time performance in the previous sentence.

For example, we calculate the time complexity of 3 algorithms with a given array input below:

a = [...] where array size n

We have three algorithms that take 
array 'a' as input and returns values.

F1(a) => 1 + a[0]; 
algorithm F1 returns 1 plus the first
array value of 'a'

F2(a) => sum(a); 
algorithm F2 returns the sum of 
array value 'a'

F3(a) => pair(a); 
algorithm F3 returns the pairing of the 
array values in 'a'.
for example if a = [1,2,3], F3 returns 
1,2  
1,3  
2,1  
2,3...

If the size of the array 'n' increases
to 10,000,000, we find out that:

F1 algorithm still runs in constant
time as it refers to a pointer to a
memory location like array[0] is
O(1)
F2 algorithm runs in linear time as
the time taken increases proportionally
to the size of 'n' array. The expression
sum(array) where array size is an input
that can vary is O(N)
F3 algorithm would most likely have a
nested for loop that would mean it runs
in O(N^2)

Logarithms in Big O

The expression log (n) = y in computer science assumes log is in base 2 and not base 10 like mathematics. This is a very important distinction to make. Therefore the full expression is log2 (n) = y.

To get 'n' we simply do 2^y = n. That means to get 'n' in log (n)=4, we simply do 2^4 = 16.

How to Identify O(log(N)) algorithms
Remember that in computer science the expression log (n) = y evaluates to log2 (n) = y, log is in base 2. Thus an expression log (n)=4 which can be solved as 2^4 = 16 simply multiplies 2 four times 2*2*2*2.

If we added 1 to 2^4+1 we would get 16*2 which is
2^5=32

It means that when the input doubles, you only have to do one more operation. The magic happens here!

To understand if a notation is O(log(N)), if we have an algorithm that keeps cutting an array in half like a binary search as shown below:

[0,1,2,3,4,5,6,7,8,9]
[0,1,2,3,4, | 5,6,7,8,9] => [0,1,2,3,4]
[0,1,2 | ,3,4] => [0,1,2]
[0,1 | ,2] => [0,1]
[0 | 1] => [0]

Or if it cuts a tree data structure in half. Read about tree data structure here, it is most likely a O(log(N)).

For further studies, visit https://www.algoexpert.io

Data Structure and Endianness in Nodejs

Ahaiwe Emmanuel — Tue, 23 Nov 2021 09:44:19 +0000

Data structure is the collection of data values, the relationship among them and the functions or operations that can be applied to the data according to Wikipedia.

Memory in Data structure
The concept of memory is that there are finite memory slots. So it can be exhausted.
The program is always stored in a free memory slot. Memory is stored as bit and byte. 8 bits make a byte.

For example:

1 represented as a byte is 0000 0001         
2 represented as a byte is 0000 0010
3 represented as a byte is 0000 0011
4 represented as a byte is 0000 0100

These are represented in base 2. And that gives us a finite amount of numbers we can store because base 2^8 = 256. So to save bigger numbers, we increase the bytes.

In java for example, "int" is the equivalent of 4 bytes, 32 bits integer. Type "long" represents 8 bytes, a 64 bits integer.

Endianness in Data Structure
The concept of Endianness in computer memory storage refers to how bytes are read. It could either be the smallest byte value first called Little-endian (read from left to right) or biggest byte value first called big-endian (read from left to right).
For example,

65,000 decimal number in base 2 is: 0b11111101 11101000.
0b here helps us know it is in base 2 and not 1,111,110,111,101,000

The answer above is in big-endian byte because the byte with the biggest 0b11111101 is written first reading from left to right. We can confirm this by converting the big-endian padded by another byte of zeros "0b1111110100000000 to base 10" on google. The result is 64,768 which is very close to our initial number of 65,000.

For networks, the standard is Big-endian and for most PCs, the standard is Little-endian. Check this code snippet online to find out what endian your machine runs. My machine runs on Little-endian.

Endianness has largely ceased to be a matter of concern because of modern computer languages removing this needless complexity and processors which are Bi-endian and can handle both.

For further studies, please visit https://www.algoexpert.io

Serving Multiple Applications with Nginx

Ahaiwe Emmanuel — Fri, 01 May 2020 22:28:36 +0000

Imagine this scenario, you just got your dream DevOps job. You are very excited. In no time, you get your first task to host a javascript, python and PHP application on a single Linux centos server. At first, you are confused about how you'll serve multiple applications from a single server. You do a little digging and then you strike gold. Nginx!

"NGINX is open-source software for web serving, reverse proxying, caching, load balancing, media streaming, and more. It started out as a web server designed for maximum performance and stability. In addition to its HTTP server capabilities, NGINX can also function as a proxy server for email (IMAP, POP3, and SMTP) and a reverse proxy and load balancer for HTTP, TCP, and UDP servers."

Prerequisite:

Linux centos server

STEP 1

ssh root@server_ip_address

You'll be asked to input your password.

Installing Nginx on Centos
Nginx packages can be gotten from EPEL repositories:

sudo yum install epel-release

Install Nginx:

sudo yum install nginx

Output should look like:

Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
Importing GPG key 0x352C64E5:
Userid     : "Fedora EPEL (7) <epel@fedoraproject.org>"
Fingerprint: 91e9 7d7c 4a5e 96f1 7f3e 888f 6a2f aea2 352c 64e5
Package    : epel-release-7-9.noarch (@extras)
From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
Is this ok [y/N]:

Press y and enter

Upon completion, start Nginx and check it's status using:

sudo systemctl enable nginx
sudo systemctl start nginx
sudo systemctl status nginx

If all goes well, you should see this:

● nginx.service - The nginx HTTP and reverse proxy server
  Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; vendor preset: disabled)
  Active: active (running) since Mon 2018-03-12 16:12:48 UTC; 2s ago
  Process: 1677 ExecStart=/usr/sbin/nginx (code=exited, status=0/SUCCESS)
  Process: 1675 ExecStartPre=/usr/sbin/nginx -t (code=exited, status=0/SUCCESS)
  Process: 1673 ExecStartPre=/usr/bin/rm -f /run/nginx.pid (code=exited, status=0/SUCCESS)
Main PID: 1680 (nginx)
  CGroup: /system.slice/nginx.service
          ├─1680 nginx: master process /usr/sbin/nginx
          └─1681 nginx: worker process

Open HTTP(80) and HTTPS(433) if you are behind a firewall with these commands:

sudo firewall-cmd --permanent --zone=public --add-service=http
sudo firewall-cmd --permanent --zone=public --add-service=https
sudo firewall-cmd --reload

Open http://YOUR_IP in your browser of choice, you should see an Nginx homepage displayed. Congratulation!!!

Step 2

Create a New User Account with sudo Privileges
For security reasons, it is advisable not to use the root user to run commands but to create a new user with sudo privileges:

useradd username

Replace username with your username with Sudo privileges.

Set the User Password

passwd username

You should get the output:

output
Changing password for user username.
New password:
Retype new password:
passwd: all authentication tokens updated successfully.

Add User to the sudo Group

usermod -aG wheel username

Switch to Newly Created User

su - username

Then list \root directory content:

sudo ls -l /root

You'll see a banner message the first time you sudo from this account:

output
We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:

    #1) Respect the privacy of others.
    #2) Think before you type.
    #3) With great power comes great responsibility.

[sudo] password for username:

Serve Multiple Applications

Log in to the Server Using Your username with Root Privileges

Our directory structure would look like this:

/var/www/
├── javascriptapp.com
│   └── public_html
├── pythonapp.com
│__ └── public_html

Let's create a directory where your javascript and python application would reside:

sudo mkdir -p /var/www/javascriptapp.com/public_html
sudo mkdir -p /var/www/pythonapp.com/public_html

Important!
To avoid any permission issues, let us change ownership to our username with sudo privileges:

sudo chown -R username: /var/www/javascriptapp.com
sudo chown -R username: /var/www/pythonapp.com

sudo vi /var/www/javascriptapp.com/public_html/index.html

Copy and paste the following lines:

\<!DOCTYPE html>
\<html lang="en" dir="ltr">
  \<head>
    \<meta charset="utf-8">
    \<title>Welcome to javascriptapp.com</title>
  \</head>
  \<body>
    \<h1>Success! javascriptapp.com home page!</h1>
  \</body>
\</html>

Create a Server Block
Server blocks are stored in the /etc/nginx/conf.d directory and must end with .conf

sudo vi /etc/nginx/conf.d/javascriptapp.com.conf

Copy and paste the following lines into the file, remember to replace the server name with your value:

server {
    listen 80;
    listen [::]:80;
    root /var/www/javascriptapp.com/public_html;
    index index.html;
    server_name example.com www.javascriptapp.com;
    access_log /var/log/nginx/javascriptapp.com.access.log;
    error_log /var/log/nginx/javascriptapp.com.error.log;
    location / {
        try_files $uri $uri/ =404;
    }
}

Save the file and check that configuration is right:

sudo nginx -t

Restart Nginx:

sudo systemctl restart nginx

Then visit your server name:
http://javascriptapp.com

NB: This must be a configured DNS to your server else just use the server IP address.

Final Step

Configure Nginx for Node JS Application

Upload node project to /var/www/javascriptapp.com/public_html, build the application and start node application on a designated port.

Edit the application .conf file to look like this:
sudo vi /etc/nginx/conf.d/javascriptapp.com.conf

server {
    listen 80;
    listen [::]:80;
    root /var/www/javascriptapp.com/public_html;
    server_name example.com www.javascriptapp.com;
    access_log /var/log/nginx/javascriptapp.com.access.log;
    error_log /var/log/nginx/javascriptapp.com.error.log;
    location / {
        proxy_pass http://internal_server_ip:node_app_PORT;
        try_files $uri $uri/ =404;
    }
}

NB: You can get the internal_server_ip by running hostname -I on the terminal. See https://www.linuxtrainingacademy.com/determine-public-ip-address-command-line-curl/ for more.

Restart Nginx:

sudo systemctl restart nginx

And visit your server name or IP to verify that your application is served.

Congratulation! Your javascript app is successfully served!
You can follow a similar process to serve your python application in the created directory.

This post is inspired by https://www.linuxtrainingacademy.com/determine-public-ip-address-command-line-curl/

Redis for Caching in Node js

Ahaiwe Emmanuel — Mon, 06 Jan 2020 01:01:06 +0000

Why use Redis?
Making database request and persisting data in an application can become costly as an application grows in the number of features and users it has. Think of the load time it takes for your browser to open a new page. We can observe that it can take a while to load, especially if internet connectivity is poor. Imagine your application making these request to services like a database under the hood. One can end up with an application with bad user experience because it might take a long time to load and navigate pages. Redis is the answer to this concern.

Redis
Is an in-memory data structure store, used as a database, cache or message broker. It is open-source so you can make contributions! Yaay :)

Downloading and Installing Redis MacOS
Using Homebrew run:

brew install redis

Launch Redis on Computer Start
ln -sfv /usr/local/opt/redis/*.plist ~/Library/LaunchAgents

Test if Redis is Running

redis-cli ping

If it replies "pong" then bravo! we are good to go.

Downloading and Installing Redis on Windows
Visit https://github.com/ServiceStack/redis-windows/tree/master/downloads
and download the latest zip file. Run the executable script called Redis server.

Add “C:\Program Files\Redis\” to the end of the variable value and click “OK.” if it doesn't already exist.

Test if Redis is Running

redis-cli ping

If it replies "pong" then bravo! we are good to go.

Caching Database Requests with Redis
Assumption - you have node installed on your device.

Create a folder and call it redis-tut or whatever you like :)

Run:

npm install express node-fetch redis

to install the needed packages for this tutorial.

Create a file called redis.js in that folder.

Import the packages and instantiate them like this:

const express = require('express'); const fetch = require('node-fetch'); const redis = require('redis'); const PORT = process.env.PORT || 5000; const REDIS_PORT = process.env.PORT || 6379; const client = redis.createClient(REDIS_PORT); const app = express();

Set response to request

function setResponse(username, repos) { return `<h2>${username} has ${repos} Github repos</h2>`; }

Make a get request to get all public repositories of a given github username:

async function getAllPublicRepos(req, res, next) { try { console.log('Fetching Public Data of Supplied Username...'); const { username } = req.params; const response = await fetch(`https://api.github.com/users/${username}`); const data = await response.json(); const repos = data.public_repos; // Set data to Redis called username client.setex(username, 3600, repos); res.send(setResponse(username, repos)); } catch (err) { console.error(err); res.status(500); } }

Create a middleware for data caching

function cache(req, res, next) { const { username } = req.params; client.get(username, (err, data) => { if (err) throw err; if (data !== null) { res.send(setResponse(username, data)); } else { next(); } }); }

Initialize API routes for use in the application

app.get('/repos/:username', cache, getRepos); app.listen(5000, () => { console.log(`App listening on port ${PORT}`); });

Visit

http://localhost:5000/repos/{any-github-repo-username}

to test the application. You should see a summary of the number of public repositories a given user has.

Inspect the page and switch to the network tab. Refresh the page. You would notice that the page loads immediately because the data values have been cached. Hurray!!!

If you followed through to this point, congratulations! You have successfully set up Redis for caching in Nodejs.

Please share your comment and ways this tutorial could be improved. Thank you for your time :)

Credit: A lot of this tutorial was influenced by Brad Traversy