DEV Community: Marius Vincent NIEMET

How to create a local Kubernetes cluster with Kind

Marius Vincent NIEMET — Mon, 17 Jun 2024 12:06:15 +0000

While developing apps that will live in a Kubernetes environment it’s always better to have a local cluster to test our app or to debut issues. In this article, we will learn how to create a local Kubernetes cluster using kind.

Kind (Kubernetes in Docker) is a tool for running local Kubernetes clusters using Docker container “nodes”. kind was primarily designed for testing Kubernetes but may be used for local development or CI.

Installation

Depending on your OS you can check the documentation here. If you are on Windows like me you can:

Install with A Package Manager chocolatey. Once chocolatey is installed run the command below:

choco install kind

Install from release binaries with PowerShell

curl.exe -Lo kind-windows-amd64.exe https://kind.sigs.k8s.io/dl/v0.23.0/kind-windows-amd64
Move-Item .\kind-windows-amd64.exe c:\some-dir-in-your-PATH\kind.exe

Since Kind leverages docker, you will need to have docker installed on your machine or you can let Kind install it for you during the process. You can run the command below to check if it has been successfully installed.

kind --version

Creating a cluster

Once you have Kind installed we create a cluster by running the command below:

kind create cluster

It will create and run a docker container on your machine, that container will act as the control plane of your cluster and all the control plane components will be installed in it. Run the command below to check the containers that are currently running:

docker ps

We have one container running with the name kind-control-plane, this is our first cluster. It’s a cluster with a single node that will act as a node and control plane as well.

Cluster with a name

The cluster created above has the name kind which is the name by default but we can create a new cluster and specify a different name:

kind create cluster --name second-cluster

Now we have two clusters, the first is kind the second second-cluster.

Cluster with multiple nodes

In production most of the time we have multiple nodes for running our apps, it’s always better to have the same infra locally for testing. Kind allows us to create a cluster with multiple nodes (control plane and worker nodes), to do that we have to create a file to specify the configuration.

# three node (two workers) cluster config
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: worker
- role: worker

Create a file named kind-example-config.yaml copy and paste the content above then run the command below:

kind create cluster --config kind-example-config.yaml

The cluster has been successfully created and you can see from the image below we have three containers running, the control plane, and two workers nodes.

You can expose extra ports for the worker nodes so they can be accessible from the machine, to do that you have to update your configuration:

apiVersion: kind.x-k8s.io/v1alpha4
kind: Cluster
nodes:
- role: control-plane
  extraPortMappings:
  - containerPort: 30000
    hostPort: 30000
    listenAddress: "0.0.0.0" # Optional, defaults to "0.0.0.0"
    protocol: tcp # Optional, defaults to tcp
  - containerPort: 31321
    hostPort: 31321
  - containerPort: 31300
    hostPort: 31300
- role: worker
- role: worker

Deleting a cluster

You can delete a cluster by running the command below:

kind delete cluster

If the cluster’s name is mentioned, it will delete the default cluster the one that has the name kind

kind delete cluster --name second-cluster

Interact with the cluster

Once kind is installed we can already use the kubectl commands:

kubectl get node

If you have multiple clusters on your machine you have to specify the name of the cluster as context for each command:

Kubectl get nodes --context kind-kind-2 
# or 
kubectl get nodes --context kind-second-cluster

Load docker images

One of the advantages of having a local cluster is that we don’t need to host our images in a registry to use them in Kubernetes Deployment. But since our nodes are running inside containers we have to make our images available inside those containers so they can be used. To do that Kind provide a command to load images into the cluster nodes:

kind load docker-image my-image-name:tag

This will make the image my-image-name available for use in the cluster.

Install Helm

From now we have a working Kubernetes cluster, we might need Helm to easily install packages. According to your OS check the documentation. If you are on Windows like me you can install it by using chocolatey.

choco install kubernetes-helm

helm version

Install nginx using Helm

Add and update the repo:

helm repo add stable https://charts.helm.sh/stable
helm repo update

Install Nginx

helm install my-nginx stable/nginx-ingress

Get the pods list

Get the services list

We have nginx running inside our cluster.

Conclusion

Throughout the article, we have discovered what is Kind and how to create a Kubernetes cluster for local development. In the next article, we will learn how to deploy our first app in Kubernetes.

I hope you enjoy this article as much as I enjoyed writing it.

Feel free to reach out to me on LinkedIn or Twitter.

Containers Orchestration and Kubernetes

Marius Vincent NIEMET — Mon, 10 Jun 2024 14:18:45 +0000

In my previous articles, we learned about containers and how to create a stack for your multi-container app using docker-compose. However, managing a multitude of containers can quickly become complex and unwieldy. This is where container orchestration comes into play. In this article, we will do a quick overview of the challenges that can be handled using an orchestration tool and then introduce Kubernetes, its role, and its components.

What is container orchestration?

Once the containers are running, container orchestration tools automate life cycle management and operational tasks based on the container definition file, including:

Deployment: your app is running in production and you want to deploy a new version without downtime.
Scaling up: you have a peak load at a certain time of the day you want your app to handle that load by automatically adding instances of your app and servers.
Scaling down: once the load has decreased the resources should be removed as well to avoid paying for unnecessary machines.
Performance and Health: your app is made by multiple services communicating with each other, if one of them fails you should be able to notice and restart the service.
Networking: exposing your services to the end user and handling the communication between the internal services with load balancing.

Even though this article will be focused on Kubernetes I want to mention that there are multiple container orchestration platforms such as Mesos, Docker Swarm, OpenShift, Rancher, Hashicorp Nomad, etc.

What is Kubernetes?

Kubernetes is a platform built by Google for container orchestrations. It aims to provide automated solutions for all the challenges mentioned before such as deployment, horizontal scaling, performance and health monitoring of the application, etc.

Kubernetes has multiple components that work together in order to orchestrate the app's load.

Control plane

The control plane is responsible for running the Kubernetes cluster and it has its components.

Each component plays a role within the control plane:

etcd: it’s a fault-tolerant and distributed key-value store that stores the cluster state and configuration.
API Server: exposes the API used to interact with the cluster and the internal and external requests.
Scheduler: is responsible for scheduling new pod creation and deciding in which worker node they will be run.
Controller: It monitors the current state of the cluster and takes action to move that state to the desired state. It runs separate processes such as node controller, replica, Endpoint, service account, and token controller.
Cloud manager controller: it can embed cloud-specific control logic, such as accessing the cloud provider’s load balancer service. It enables you to connect a Kubernetes cluster with the cloud provider's API. Additionally, it helps decouple the Kubernetes cluster from components that interact with a cloud platform, so that elements inside the cluster do not need to be aware of the implementation specifics of each cloud provider.

Node

Nodes are virtual or physical machines responsible for running the application instances. Each node has a bunch of components running in it.

Kube Proxy: it communicates with the Kubernetes control plane and allows network communication to pods between pods.
Kubelet: it’s the tool responsible for running the state on each node, checking container health, and taking action.
Container runtime: each node needs a container runtime to run containers, it can be docker or any other runtime.
Container Networking: Container networking enables containers to communicate with hosts or other containers.
Pod: it’s the smallest unit Kubernetes allows to interact with. Inside a POD we can run containers.

Cluster

A Kubernetes cluster is the collection of the control plane with its worker nodes. A cluster can scale up to 5000 worker nodes.

Kubernetes Objects

Kubernetes objects are persistent entities in the Kubernetes system. They represent the state of the cluster, once an object is created and applied Kubernetes will work to ensure that the current state is equal to the desired.

Volume: At its core, a volume is a directory, possibly with some data in it, accessible to the containers in a pod.
Service: it’s a method used to expose an app within the cluster.
Namespace: it’s a way to create an isolated collection of resources within the cluster. Names of resources need to be unique within a namespace, but not across namespaces. and pods which we have already introduced in the previous section.

Helm package manager

With Kubernetes, Helm serves as a package manager, working similarly to npm in Node.js and yum in Linux. Helm deploys charts as complete and packaged Kubernetes applications, which include pre-configured versioned application resources. It is possible to deploy different chart versions by using different configuration sets.

Conclusion

In conclusion, we have explored the basics of container orchestration, the fundamentals of Kubernetes, its architecture, and its key components. We have also covered its basic objects and introduced Helm the package manager.

I hope you enjoy this article as much as I enjoyed writing it.

Feel free to reach out to me on LinkedIn or Twitter.

Containers Orchestration and Kubernetes

Marius Vincent NIEMET — Mon, 10 Jun 2024 14:18:45 +0000

What is container orchestration?

Once the containers are running, container orchestration tools automate life cycle management and operational tasks based on the container definition file, including:

Deployment: your app is running in production and you want to deploy a new version without downtime.
Scaling up: you have a peak load at a certain time of the day you want your app to handle that load by automatically adding instances of your app and servers.
Scaling down: once the load has decreased the resources should be removed as well to avoid paying for unnecessary machines.
Performance and Health: your app is made by multiple services communicating with each other, if one of them fails you should be able to notice and restart the service.
Networking: exposing your services to the end user and handling the communication between the internal services with load balancing.

What is Kubernetes?

Kubernetes has multiple components that work together in order to orchestrate the app's load.

Control plane

The control plane is responsible for running the Kubernetes cluster and it has its components.

Each component plays a role within the control plane:

etcd: it’s a fault-tolerant and distributed key-value store that stores the cluster state and configuration.
API Server: exposes the API used to interact with the cluster and the internal and external requests.
Scheduler: is responsible for scheduling new pod creation and deciding in which worker node they will be run.
Controller: It monitors the current state of the cluster and takes action to move that state to the desired state. It runs separate processes such as node controller, replica, Endpoint, service account, and token controller.
Cloud manager controller: it can embed cloud-specific control logic, such as accessing the cloud provider’s load balancer service. It enables you to connect a Kubernetes cluster with the cloud provider's API. Additionally, it helps decouple the Kubernetes cluster from components that interact with a cloud platform, so that elements inside the cluster do not need to be aware of the implementation specifics of each cloud provider.

Node

Nodes are virtual or physical machines responsible for running the application instances. Each node has a bunch of components running in it.

Kube Proxy: it communicates with the Kubernetes control plane and allows network communication to pods between pods.
Kubelet: it’s the tool responsible for running the state on each node, checking container health, and taking action.
Container runtime: each node needs a container runtime to run containers, it can be docker or any other runtime.
Container Networking: Container networking enables containers to communicate with hosts or other containers.
Pod: it’s the smallest unit Kubernetes allows to interact with. Inside a POD we can run containers.

Cluster

A Kubernetes cluster is the collection of the control plane with its worker nodes. A cluster can scale up to 5000 worker nodes.

Kubernetes Objects

Volume: At its core, a volume is a directory, possibly with some data in it, accessible to the containers in a pod.
Service: it’s a method used to expose an app within the cluster.
Namespace: it’s a way to create an isolated collection of resources within the cluster. Names of resources need to be unique within a namespace, but not across namespaces. and pods which we have already introduced in the previous section.

Helm package manager

Conclusion

I hope you enjoy this article as much as I enjoyed writing it.

Feel free to reach out to me on LinkedIn or Twitter.

Containerize your multi-services app with docker compose

Marius Vincent NIEMET — Thu, 28 Mar 2024 01:44:02 +0000

This article was posted first on my personal blog mariusniemet.me

In this previous article, we learned what a container is and took a brief introduction by learning how to create an image and then run a container, it was a good starting point.

But with the new industry standard most of the time you will work on apps made by multiple services that have dependencies between them, frontend, backend, database, caching, queue, and so on.

Project architecture

The image above describes the architecture of the project we want to containerize, as you can see we have four different services

Front: a single-page app built with React
Back: a graphQL server built with Nestjs
Database: a MySQL db
Cache: a Redis cache

If you want to follow along you have to download the source code here.

What is docker compose?

Docker Compose is a tool for defining and running multi-container applications. Compose simplifies the control of your entire application stack, making it easy to manage services, networks, and volumes in a single, comprehensible YAML configuration file.

How to create a stack?

First, you have to create a Dockerfile for each custom service (service for which you are responsible for building the image), for our current project, we have to create a Dockerfile for the front and back services.

Front

cd client

create a file named Dockerfile and add the content below:

FROM node:18-alpine

WORKDIR /app

COPY package.json .

RUN npm install

COPY . .

EXPOSE 5173

CMD [ "npm", "run", "dev" ]

Back

cd graphql-server

create a file named Dockerfile and add the content below:

FROM node:16-alpine AS base

# Set working directory
WORKDIR /app

# Copy package.json and package-lock.json
COPY package*.json ./

FROM base AS dev
# Install dependencies
RUN npm install 

# Copy source code
COPY . .

# Expose port 3000
EXPOSE 3000

# Start app
CMD ["npm", "run", "start:dev"]

FROM base as prod 
ENV NODE_ENV production 

# Install app dependencies
RUN npm install

# Bundle app source
COPY . .

# Creates a "dist" folder with the production build
RUN npm run build

# Start the server using the production build
CMD [ "node", "dist/main.js" ]

The compose file

Once we have the Dockerfile for our services the next step is to create a compose.yaml file, docker compose relies on that file for the stack configuration. Compose also supports docker-compose.yaml and docker-compose.yml for backward compatibility with earlier versions. If both files exist, Compose prefers the canonical compose.yaml.

name: composeintroduction

If you don’t set a name compose will use the name of the directory in which the compose file is located.

Services

A service is an abstract definition of a computing resource within an application. A service is defined by a Docker image and a set of runtime arguments (mostly docker run arguments). While creating a service you can use an existing image by setting the image name and tag compose will fetch it from the registry for you or you can ask compose to build a new image for you by specifying the path to the Dockerfile.

Let’s create the compose service for the backend:

name: composeintroduction

services: 
  backend:
    build:
      context: ./graphql-server
      dockerfile: Dockerfile

services is a top-level argument and all our services will be declared here.

First, we have the service name in this case it’s backend and since for that service, we won’t rely on an existing image we have to specify how compose should build the image that’s the role of the build section, context is used to specify the subfolder in which the Dockerfile is located and the dockerfile argument is to specify the name. If your file is named Dockerfile you can remove that argument but if the name is different for any reason you have to specify it.

If your Dockerfile uses a multi-stage build you can specify the stage you are targeting and the command you want docker-compose to run inside your container once it’s ready.

name: composeintroduction

services: 
  backend:
    build:
      context: ./graphql-server
      dockerfile: Dockerfile
      target: dev
    command: npm run start:dev

From now we will have a service named backend that will be built using the Dockerfile inside the graphql-server subfolder, the target stage will be dev and the command used will be npm run start:dev

Port definition:

As we have mentioned above all the arguments you usually pass to the docker run command are available here so to define ports for our container we use the syntax below:

name: composeintroduction

services: 
  backend:
    build:
      context: ./graphql-server
      dockerfile: Dockerfile
      target: dev
    command: npm run start:dev
    ports:
      -"3000:3000"

The port at the left is the external accessible on the host machine and the right is the port inside the container (the same exposed in the Dockerfile).

Environnment variable

There are multiple ways to pass environment variables to a service:

By using a file: An .env file in Docker Compose is a text file used to define environment variables that should be made available to Docker containers when running docker compose up. This file typically contains key-value pairs of environment variables, and it allows you to centralize and manage configuration in one place.

The .env file content:

DB_PASSOWORD=password
DB_USERNAME=user

name: composeintroduction

services: 
  backend:
    build:
      context: ./graphql-server
      dockerfile: Dockerfile
      target: dev
    command: npm run start:dev
    ports:
      -"3000:3000"
    env_file:
      - .env

By passing key-value directly: There are two syntaxes for that

name: composeintroduction

services: 
  backend:
    build:
      context: ./graphql-server
      dockerfile: Dockerfile
      target: dev
    command: npm run start:dev
    ports:
      -"3000:3000"
    environment:
      DB_PASSWORD: password
      - DB_USERNAME=user

It’s up to you to choose the way you want to set variables, for the rest of this article we will use the syntax below:

name: composeintroduction

services: 
  backend:
    build:
      context: ./graphql-server
      dockerfile: Dockerfile
      target: dev
    command: npm run start:dev
    ports:
      -"3000:3000"
    environment:
      DB_PASSWORD: password
      DB_USERNAME: user

Now let’s create the service for the database.

services: 
   database: 
      image: mysql:latest
      environment: 
        MYSQL_ROOT_PASSWORD: password
        MYSQL_DATABASE: db
        MYSQL_USER: user
        MYSQL_PASSWORD: password

For the database service, since we will use the MySQL official image we don’t need the build section here, compose will fetch the image with the tag latest for us from the registry.

Last update the compose file by adding the remaining services:

name: composeintroduction

services: 
  backend:
    build:
      context: ./graphql-server
      dockerfile: Dockerfile
      target: dev
    command: npm run start:dev
    ports:
      - "3000:3000"
    environment:
      DB_PASSWORD: password
      DB_USERNAME: user

  database: 
    image: mysql:latest
    environment: 
      MYSQL_ROOT_PASSWORD: password
      MYSQL_DATABASE: db
      MYSQL_USER: user
      MYSQL_PASSWORD: password

  cache: 
    image: redis:latest
    ports:
    - "6380:6379"


  frontend: 
    build: 
      context: ./client
      dockerfile: Dockerfile
    command: npm run dev
    ports: 
    - "5173:5173"

Now we have added the frontend service and the cache service. We can now run our stack. First, we have to build the images by calling the command below inside our project’s folder :

docker compose build

Then compose will build the backend and the frontend image by using the specifications we have added in the compose file, your terminal should display something like the image below:

Once the images are built you can run the stack by calling the command below:

docke compose up -d

The flag -d is for running the command in detach mode. By running this command compose will run all the containers declared in the compose file, for the backend and frontend compose will use the images built first for the database and the cache compose will fetch the images from the registry.

If everything went well you should have the result below just make sure the ports we have exposed aren’t used by other containers on your machine.

Each container’s name includes the stack name, the service name, and a number. Here we have one since there is only one instance by service.

Communication

Once we have our services running we want to establish communication between them, the backend should talk to the database and the cache, and the frontend should talk to the backend.

There are two things to consider here:

The first is that compose creates a default network for your entire stack if you pay attention to the image above you have the first line Network composeintroduction_default that shows the name of the default network.
The second is that compose comes with a DNS so if your services are in the same network you can use the service name + the port to communicate compose will figure out the IP address of the service you want to communicate with.

Based on these two pieces of information we understand that we can already establish the communication between our services just by using the service name, let’s update the backend first:

import { CacheModule } from '@nestjs/cache-manager';
import { Module } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';
import * as redisStore from 'cache-manager-redis-store';
import { AppController } from './app.controller';
import { AppService } from './app.service';

@Module({
  imports: [
    TypeOrmModule.forRoot({
      type: 'mysql',
      host: 'database',
      port: 3306,
      username: process.env.DB_USERNAME,
      password: process.env.DB_PASSWORD,
      database: process.env.DB_NAME,
      entities: ['dist/**/*entity.js'],
      synchronize: true,
    }),
    CacheModule.register({
      isGlobal: true,
      store: redisStore,
      host: 'cache',
      port: 6379,
    }),
  ],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}

Then the front:

import { ApolloClient, ApolloProvider, InMemoryCache } from "@apollo/client";
import React from "react";
import ReactDOM from "react-dom/client";
import App from "./App.tsx";
import "./index.css";

const client = new ApolloClient({
  uri: "http://backend:3000/graphql",
  cache: new InMemoryCache(),
});

ReactDOM.createRoot(document.getElementById("root")!).render(
  <React.StrictMode>
    <ApolloProvider client={client}>
      <App />
    </ApolloProvider>
  </React.StrictMode>
);

Now if we rebuild our images using docker compose build then run the containers using docker compose up -d we can access the frontend at http://localhost:5174 and fetch the data from the backend that will communicate with the cache and the database.

You may think that everything is great since our services can communicate with each other but we have a tiny issue here, all our services are inside the same default network exposed externally which means we can access the db, the cache, and the backend from the host machine which is something we don’t necessarily like, the idea is to have something like the image below:

Here we have two networks, public which is accessible from the host machine and private which is accessible only internally. Since the frontend will be accessed from the host and will communicate with the backend it should be inside the two networks.

Now let’s see how to do that in compose:

First, we start by adding a new top-level section named networks all the networks have to be declared there then we add two entries public and private which are the names of our networks. The private should have a parameter name internal set to true which will tell compose to not expose those services outside.

networks:
  public: 
  private: 
    internal: true

Once our networks are declared the last thing to do is to update the services definition by adding a network section:

name: composeintroduction

services: 
  backend:
    build:
      context: ./graphql-server
      dockerfile: Dockerfile
      target: dev
    command: npm run start:dev
    ports:
      - "3000:3000"
    environment:
      DB_PASSWORD: password
      DB_USERNAME: user
    networks:
      - private

  database: 
    image: mysql:latest
    environment: 
      MYSQL_ROOT_PASSWORD: password
      MYSQL_DATABASE: db
      MYSQL_USER: user
      MYSQL_PASSWORD: password
    networks:
      - private

  cache: 
    image: redis:latest
    ports:
    - "6380:6379"
    networks:
      - private


  frontend: 
    build: 
      context: ./client
      dockerfile: Dockerfile
    ports: 
    - "5173:5173"
    networks:
      - private 
      - public

networks:
  public: 
  private: 
    internal: true

We have added a network section for each service and only the frontend service has two networks now the services inside the private network can’t be accessed from the host. Note that the names private and public are not mandatory you can give any name you want the thing that makes the network private is the internal: true

Persistent volume

If we don’t want to lose our database and cache data each time the services are started we have to create volumes and assign them to the services.

The top-level volumes declaration lets you configure named volumes that can be reused across multiple services let’s do it :

volumes:
  db-data: 
  cache-data:

Then update the services definition:

cache: 
    image: redis:latest
    ports:
    - "6380:6379"
    networks:
      - private
    volumes:
      - cache-data:/data
database: 
    image: mysql:latest
    environment: 
      MYSQL_ROOT_PASSWORD: password
      MYSQL_DATABASE: db
      MYSQL_USER: user
      MYSQL_PASSWORD: password
    networks:
      - private
    volumes: 
      - db-data:/var/lib/mysql

Once the volumes are declared and assigned to the services we can restart the stack how many times we want the data won’t be lost.

Startup order

When we run the docker compose up command, from the logs printed we can see in which order the services are started

We can notice that the order is the following cache, database, frontend then backend but there are two issues here:

First, the order won’t always be the same
Second, the front will be available before the backend although it is the first that calls the second. To fix these issues compose provides an attribute nameddepends_on available for each service that will be used to control the startup and shutdown order.

The depends_on attribute takes a list of services for which the current service depends and for each service, we have to set a condition that can be service_started or service_healthy .

Let’s update the backend service then explain:

backend:
    build:
      context: ./graphql-server
      dockerfile: Dockerfile
      target: dev
    command: npm run start:dev
    ports:
      - "3000:3000"
    environment:
      DB_PASSWORD: password
      DB_USERNAME: user
    networks:
      - private
    depends_on:
      database:
        condition: service_started
      cache:
        condition: service_started

We have updated the backend service definition by adding the depends_on attribute. We have added two dependencies the database and the cache. Their condition is service_started that means compose will run the backend container only if the database and the cache have been already started.

Now let’s update the frontend service:

frontend: 
    build: 
      context: ./client
      dockerfile: Dockerfile
    ports: 
    - "5173:5173"
    networks:
      - private 
      - public
    depends_on:
      backend: 
        condition: service_healthy

Here the condition is service_healthy which is different than service_started the idea is to give the developers more control over when a dependency can be considered ready to receive requests cause having the backend started doesn’t always mean it’s ready to receive requests.

Now we have to define how compose will define if the backend service is healthy and to do that we are going to update the backend service definition by adding a new attribute named healthcheck :

backend:
    build:
      context: ./graphql-server
      dockerfile: Dockerfile
      target: dev
    command: npm run start:dev
    ports:
      - "3000:3000"
    environment:
      DB_PASSWORD: password
      DB_USERNAME: user
    networks:
      - private
    depends_on:
      database:
        condition: service_started
      cache:
        condition: service_started
    healthcheck:
      test: ["CMD", "curl", "-f", "http://backend:3000/healthcheck"]
      interval: 1m30s
      timeout: 10s
      retries: 3
      start_period: 40s
      start_interval: 5s

When a container has a healthcheck specified, it has a health status in addition to its normal status. This status is initially starting. Whenever a health check passes, it becomes healthy (whatever state it was previously in). After a certain number of consecutive failures, it becomes unhealthy. The value of the test attribute can be whatever you want here we are making an HTTP request since it’s a web service but for example, in the case of a database, you can replace it with a mysql ping command.

The final compose.yaml file should look like this:


name: composeintroduction

services: 
  backend:
    build:
      context: ./graphql-server
      dockerfile: Dockerfile
      target: dev
    command: npm run start:dev
    ports:
      - "3000:3000"
    environment:
      DB_PASSWORD: password
      DB_USERNAME: user
    networks:
      - private
    depends_on:
      database:
        condition: service_started
      cache:
        condition: service_started
    healthcheck:
      test: ["CMD", "curl", "-f", "http://backend:3000/healthcheck"]
      interval: 1m30s
      timeout: 10s
      retries: 3
      start_period: 40s
      start_interval: 5s

  database: 
    image: mysql:latest
    environment: 
      MYSQL_ROOT_PASSWORD: password
      MYSQL_DATABASE: db
      MYSQL_USER: user
      MYSQL_PASSWORD: password
    networks:
      - private

  cache: 
    image: redis:latest
    ports:
    - "6380:6379"
    networks:
      - private


  frontend: 
    build: 
      context: ./client
      dockerfile: Dockerfile
    ports: 
    - "5173:5173"
    networks:
      - private 
      - public
    depends_on:
      backend: 
        condition: service_started

networks:
  public: 
  private: 
    internal: true

Secrets

A secret is a blob of data, such as a password, SSH private key, SSL certificate, or another piece of data that should not be transmitted over a network or stored unencrypted in a Dockerfile or your application’s source code.

To use a secret in our compose stack we have to do the following actions:

First, declare the secret by using the secrets top-level element. The source of the secret is either file or external.

secrets:
  my_certificate: 
    file: ./basic-certificate.cert

Second, grant the access to the secret for the frontend service

    build: 
      context: ./client
      dockerfile: Dockerfile
    ports: 
    - "5173:5173"
    networks:
      - private 
      - public
    depends_on:
      backend: 
        condition: service_started
    secrets: 
      - my_certificate

The final compose file should look like this:

name: composeintroduction

services: 
  backend:
    build:
      context: ./graphql-server
      dockerfile: Dockerfile
      target: dev
    command: npm run start:dev
    ports:
      - "3000:3000"
    environment:
      DB_PASSWORD: password
      DB_USERNAME: user
    networks:
      - private
    depends_on:
      database:
        condition: service_started
      cache:
        condition: service_started
    healthcheck:
      test: ["CMD", "curl", "-f", "http://backend:3000/healthcheck"]
      interval: 1m30s
      timeout: 10s
      retries: 3
      start_period: 40s
      start_interval: 5s

  database: 
    image: mysql:latest
    environment: 
      MYSQL_ROOT_PASSWORD: password
      MYSQL_DATABASE: db
      MYSQL_USER: user
      MYSQL_PASSWORD: password
    networks:
      - private

  cache: 
    image: redis:latest
    ports:
    - "6380:6379"
    networks:
      - private
    volumes:
      - cache-data:/data


  frontend: 
    build: 
      context: ./client
      dockerfile: Dockerfile
    ports: 
    - "5173:5173"
    networks:
      - private 
      - public
    depends_on:
      backend: 
        condition: service_started
    secrets: 
      - my_certificate

secrets:
  my_certificate: 
    file: ./basic-certificate.cert

volumes:
  db-data: 
  cache-data:

networks:
  public: 
  private: 
    internal: true

Conclusion

We have come to the end of this article that aimed to show how to containerize a multi-container app using docker-compose. You can find the final version of the project here. Don’t hesitate to check the compose documentation there are a lot of fun things to discover like working with multiple compose files or docker-compose watch.

I hope you enjoy this article as much as I enjoyed writing it.

Feel free to reach out to me on LinkedIn or Twitter.

A Gentle Introduction to Containerization and Docker

Marius Vincent NIEMET — Wed, 13 Mar 2024 13:05:31 +0000

If you are a software engineer I bet you have already read or heard someone talking about containerization, docker, how it’s beneficial, and why you should start using it for your app.

In this blog, we are going to do an introduction to containerization by defining what a container is and then we will explore Docker with its concepts.

Virtual machine

A VM is the action of relying on software to simulate hardware capacity and create a virtual computer system. This enables running more than one virtual system, multiple OS, and applications on the same server with its OS.

Containers

A container is a lightweight unit that includes all the processes to run an application. It’s similar to a VM with one exception the container doesn’t need its guest OS it relates to the Host OS which makes it lightweight and easy to run. A container is isolated from all the processes running on the host machine, meaning it can’t communicate with anything if you don’t authorize it explicitly.

How to create a container?

Manifest: define the libs your app needs and all the steps to create your image.
Image: It represents your app with all the dependencies and the libs it needs to run.
Container: it’s your app running.

Even though we will focus on Docker for this article, I wanted to mention that there are more container creation and management tools such as Podman, Rkt, and so on.

Introduction to Docker

It’s an open platform that enables the creation and deployment of apps using containerization, in the short term it’s a container runtime. According to your OS, you can check the docker documentation to know how to install it, since I’m using Windows I have to download docker desktop.

In the first part, we have mentioned that to create a container we need a manifest that will describe the steps and libs, in the docker world that manifest is called a DockerFile.

Dockerfile

It’s a text document that contains all the commands a user could call to assemble an image. Let’s check an example of a Dockerfile for a nodejs app in this case it will be a NestJS app and then explain each part.

FROM node:17.4.0-alpine3.14

ENV NODE_ENV=production
ENV PORT=3000

WORKDIR /user/src/app

COPY package*.json ./

RUN npm install --production

COPY . . 

RUN npm run build 

EXPOSE $PORT

CMD ["node", "dist/main.js"]

FROM: is used to specify the image base
ENV: is used to specify environment variables that can be used by instructions or inside the container.
WORKDIR: specifies the working directory which means all the following instructions will take place inside that directory.
COPY: is used to copy files from a source to the destination here we copy the package.json and lock files to the working directory.
RUN: is used to run commands inside the container, here we install all the dependencies for the project.
COPY: another copy, here we copy the source code of the app to the container working directory. The .dockerignore file comes to play since we don’t wank to copy some files (node_modules, Dockerfile) to the container.
EXPOSE: is used to specify to docker that the container is listening on a specific PORT which is 3000.
CMD: specifies the default command to be executed when a container based on that image is run.

As part of the process, we have to create a .dockerignore file, it works like the files you are already familiar with like the .gitignore for git or the .npmignore. Create a new file at the root of the project and put the content below:

node_modules

.gitignore
.vscode

Dockerfile

package-lock.json
yarn.lock

The content is similar to what you might add to your .gitignore file. In the next part of the article, we will learn when this file comes into play.

Docker image

Once your Dockerfile is ready, you can create an image using the command below.

docker image build -t docker-example .

t: represents the tag for your image. You can check the list of your images by using the command below:

docker image ls

Container

Since we have a Dockerfile that has been used to build an image we can now use the image to run a container by using the command below:

docker run --rm -p 3002:3000 --name dockerexamplecontainer docker-example

docker run: This command runs a Docker container based on a Docker image.
--rm: This flag tells Docker to automatically remove the container once it stops running. This is useful for temporary containers that are only needed for a single run, as it helps avoid cluttering your system with unused containers.
-p 3002:3000: This flag maps port 3000 inside the container to port 3002 on the host machine. It allows you to access services running inside the container via port 3002 on your localhost.
--name dockerexamplecontainer: This flag assigns a name (dockerexamplecontainer) to the container. Naming containers can make them easier to manage and reference in other Docker commands. The last parameter is the image name.

You can check the list of all containers even those stopped with the command:

docker ps -a

check only containers currently running:

and access your app through localhost:3002

It’s possible to do some action to a container such as:

Stop

docker container stop dockerexamplecontainer

Pause

docker container pause dockerexamplecontainer

Remove

docker container rm dockerexamplecontainer

A container should be stopped before trying to remove it but you can also force the action by adding a flag:

docker container rm -f dockerexamplecontainer

Registry

So far we have learned how to create a Dockerfile, build an image, and then run a container based on that image. Images can also be shared you can build your image and share it with your teammates or anyone else so they don’t have to do the whole process on their own, you can also use images built by other people. Imagine you want to run a MySQL database inside a docker container you are not going to create a Dockerfile and then build the image by yourself instead you will have an image provided by the official MySQL Team.

That’s where the notion of registry comes into play, a registry is a place on the internet where you can store your images to make them accessible to everyone or a place where you can get images from other people.

There are a lot of docker-compatible registries almost every cloud provider has its registry but for this article, we will use the docker registry called docker hub. Go to the website create a new account and sign in then you can push or pull images.

How to pull an image from docker-hub?

docker login

You will be asked to enter your credentials

docker pull MySQL

The command above will pull the latest version of the image but you can also specify if you want a specific version

docker pull mysql:8.0

Then you can run MySQL inside a container by using:

docker run --name mysql-container -e MYSQL_ROOT_PASSWORD=my-secret-pw -d MySQL

Now you have a MySQL container and didn’t build the image alone.

How to push my image to the registry? For each image, you have to create a repository, and inside a repository, you can store multiple versions of your images you have to change the tag (version).

A repository can be private or public.

docker tag dockerexample mariusniemet/dockerexample:1

We have three params, your docker hub username, the repository name, and the version.

docker push mariusniemet/dockerexample:1

Now your image is available on docker-hub.

Network

From now we have a NestJS app running inside a container, and a MySQL database running inside another container and we want to connect the app to the database. We will face an issue, remember we mentioned earlier in this article that containers are isolated processes they can’t communicate with each other and to do that we have to create a network.

Container networking refers to the ability of containers to connect to and communicate with each other.

Each container has a DNS resolver, each time they want to communicate a container will first check if it knows which IP address is linked to that given container name if it already knows it will communicate with the container if it doesn’t know it will ask the docker DNS then communicate with the container.

How to create a network?

Docker allows you to create different types of networks for this article we will create the bridge which is the type by default.

docker network create networkexample

Check your docker network list with

docker network ls

Remove a network

docker network rm networexample

Run a container by specifying the network

docker run --name mysql-container -e MYSQL_ROOT_PASSWORD=my-secret-pw -d --network=networkexample MySQL

or connect a running container to the network

docker network connect networkexample 1cd04c0cbfc9

The last parameter is the containerId you want to connect to the network.

With these commands, you can connect the NestJS app with the Mysql database since they now belong to the same network.

Run the command below to display containers that are connected to the network.

docker network inspect networkexample

Volume

From now we have the NestJS app connected to the database, we can perform our basic CRUD operations and data will be stored in the database. Still, there is a last issue cause if we stop the database container and then run it we will notice that all the data have been lost that’s because containers and their data are volatile if you want to persist them you have to create something called volume.

Docker allows you to create bin mounts or managed volume, in this article we will use managed volume.

Volumes are the preferred mechanism for persisting data generated by and used by Docker containers. While bind mounts are dependent on the directory structure and OS of the host machine, volumes are completely managed by Docker.

How to create a volume?

docker volume create mysql-data

The last parameter is the volume name

How to check the volume list?

docker volume ls

How to remove a volume?

docker volume rm mysql-data

How to link a container to the volume?

docker run --name mysql-container-v mysql-data:/var/lib/mysql mysql

The params are:

the container name
the volume name
the path in the container
the image name

Now if you stop and run the container your data will be persisted.

Conclusion

This article wanted to be a brief and gentle introduction to the container and docker world, there is a lot more to learn so don’t hesitate to check the docker documentation.

I hope you enjoy this article as much as I enjoyed writing it.

Feel free to reach out to me on LinkedIn or Twitter.

How to implement a Graph (Adjacency List and Matrix) in JavaScript

Marius Vincent NIEMET — Mon, 17 Jul 2023 12:38:03 +0000

Graph-based questions are one of the most popular asked questions in software interviews. Last year I published an introduction to graphs, explaining the different graph types and how they can be represented. You can read that article as a reminder, it will help you to understand the implementation.

In this one, we are going to learn how to implement all those graph representations in JavaScript, since the implementation can change according to the language.

Let's start writing code

Adjacency List

It’s a way to represent a graph by using a linked list, each node will be linked to every node adjacent to him.

In JavaScript we don’t need to create a pure linked list, we will use the built-in data structures Set and Map and wrap all methods inside a JS class named Graph.

Unweight and Undirected Graph

class Graph { 
  adjacencyList; 

  constructor(){ 
    this.adjacencyList = new Map(); 
  }

}

So First we have created a class named Graph and that class has only one property named adjacencyList. Our constructor initiates the property with an empty Map. So that means our entire adjacency list graph will be represented using the Map.

Now we can add some methods inside the class.

class Graph { 
  adjacencyList; 

  constructor(){ 
    this.adjacencyList = new Map(); 
  }

  addNode(node){ 
    this.adjacencyList.set(node, new Set()); 
  }

  addEdge(node1, node2){ 
    this.adjacencyList.get(node1).add(node2); 
    this.adjacencyList.get(node2).add(node1); 
  }

  getNeighboors(node){ 
    return this.adjacencyList.get(node); 
  }

  hasEdge(node1, node2){ 
    return this.adjacencyList.get(node1).has(node2); 
  }

}

addNode: this method adds a new node inside the Graph. The method receives the node we want to add and we use the built-in function set that receives two parameters, the first is the key that will be used to retrieve the value and the second is the value we want to add. Here the value is a new Set since in the introduction we have mentioned that in an adjacency list, a node should be linked to all the nodes adjacent to it so that the Set will contain all nodes adjacent to the node used as key.
addEdge: an edge is a path between two nodes, since we are creating an undirected graph, it means that if there is a path from node1 to node 2, there is a path from node2 to node1 as well. It’s what the function to here it retrieves the list of adjacent nodes for node1 and node2 and then adds the inverse node inside each list.
getNeighboors: this method returns all the adjacent nodes to a given node, only getting the value inside the graph by using the given node as the key.
hasEdge: this function check if there is a path from node1 to node2, to do that it retrieves all nodes adjacent to node1 and then checks if node2 exists in that list.

Unweight and directed Graph

The difference between a directed and undirected graph is that inside a directed graph, if there is a path from node1 to node2 doesn’t mean there is a path from node2 to node1 as well, if you want you need to explicitly create that second path.

If we want to create a directed graph from the previous code implemented, we have just to edit the addEdge method by removing the second path creation.

 addEdge(node1, node2){ 
    this.adjacencyList.get(node1).add(node2); 
  }

Weighted Graph

A weighted graph is a graph in which there is a numerical number attached to the path between two adjacent nodes. That numerical number represents the distance between the nodes, it’s commonly used when we want to calculate the shortest path between two given nodes.

If we want to create a weighted graph, by using the class implemented previously we have to make some adjustments.

class Graph { 
  adjacencyList; 

  constructor(){ 
    this.adjacencyList = new Map(); 
  }

  addNode(node){ 
    this.adjacencyList.set(node, new Map()); 
  }

  addEdge(node1, node2, weight){ 
    this.adjacencyList.get(node1).set(node2, weight); 
    this.adjacencyList.get(node2).set(node1, weight); 
  }

  getNeighboors(node){ 
    return this.adjacencyList.get(node); 
  }

  hasEdge(node1, node2){ 
    return this.adjacencyList.get(node1).has(node2); 
  }

}

First, we have changed the data structure used when creating a new node, now we use a Map instead of a Set because the Map will help us to store a key and value while we only store value with Set.

Then when we create a new path between two nodes, we need to receive those two nodes and the path’s weight as well.

Finally, the node will be the key and the weight will be the value stored.

Adjacency Matrix

It’s a way to represent a Graph by using a 2D array. The indices represent the nodes, if the value at the cell (i,j) is equal to 1 which means there is a path between the node i and j otherwise it will be equal to 0.

Unweight and Undirected Graph

We will keep the overall Graph class as a wrapper.

class Graph { 
  numberNodes; 
  adjacencyMatrix; 

  constructor(numberNodes){ 
    this.numberNodes = numberNodes; 

    this.adjacencyMatrix = []; 

    for(let i = 0; i < this.numberNodes; i++){ 
      this.adjacencyMatrix[i] = new Array(this.numberNodes).fill(0); 
    }

  }

  addEdge(node1, node2){ 

    this.adjacencyMatrix[node1][node2] = 1; 
    this.adjacencyMatrix[node2][node1] = 1; 

  }

  getNeighboors(node){ 
     return this.adjacencyMatrix[node]; 
  }

  hasEdge(node1, node2){ 

    if(node1 >= 0 && node1 < this.numberNodes && node2 >= 0 && node2 < this.numberNodes){ 
      return this.adjacenMatrix[node1][node2] === 1 && this.adjacenMatrix[node2][node1] === 1; 
    }

    return false; 
  }

  removeEdge(node1, node2){ 
    if(node1 >= 0 && node1 < this.numberNodes && node2 >= 0 && node2 < this.numberNodes){ 
      this.adjacencyMatrix[node1][node2] = 0; 
      this.adjacencyMatrix[node2][node1] = 0; 
    }

  }

}

Since we will create a 2D array we have to know in advance the number of nodes we want to store, that's the role of the numberNodes property and then we have the second which represents the array itself.

Every time a new instance of the Graph is created we have to pass the number of nodes, and then we create a new empty array assigned to the adjacencyMatrix property.

Last we traverse that array by assigning to each cell a new array of size numberNodes filled with zero.

PS: here we don’t need a method to add a node inside the Graph.

addEdge: the method receives the two nodes and then we have to set the value at the cell adjacencyMatrix[node1][node2] to 1 and since it’s an undirected graph we have to do the same for the reverse direction adjacencyMatrix[node2][node1].
hasEdge: to check if there is a path or edge between two given nodes, first the method checks if the values received exist in the Graph and then we check if the value at adjacencyMatrix[node1][node2] is equal to 1 and since it’s an undirected graph we have to check for the reverse direction as well if so we return true.
getNeighboors: to retrieve all adjacent nodes to a given node we return the cell to the position node .
removeEdge: to remove we first check if the given values are valid, if so we set the value at the cell (node1, node2) to 0 and we have to do that for both directions since it’s an undirected Graph.

Unweight and directed Graph

For a directed Graph, we have to update the class above by removing every section of code where we were doing the same operation for the reverse direction.
Check the code below:

class Graph { 
  numberNodes; 
  adjacencyMatrix; 

  constructor(numberNodes){ 
    this.numberNodes = numberNodes; 

    this.adjacencyMatrix = []; 

    for(let i = 0; i < this.numberNodes; i++){ 
      this.adjacencyMatrix[i] = new Array(this.numberNodes).fill(0); 
    }

  }

  addEdge(node1, node2){ 

    this.adjacencyMatrix[node1][node2] = 1; 

  }

  getNeighboors(node){ 
     return this.adjacencyMatrix[node]; 
  }

  hasEdge(node1, node2){ 

    if(node1 >= 0 && node1 < this.numberNodes && node2 >= 0 && node2 < this.numberNodes){ 
      return this.adjacenMatrix[node1][node2] === 1; 
    }

    return false; 
  }

  removeEdge(node1, node2){ 
    if(node1 >= 0 && node1 < this.numberNodes && node2 >= 0 && node2 < this.numberNodes){ 
      this.adjacencyMatrix[node1][node2] = 0; 
    }

  }

}

Weighted Graph

For a weighted Graph, since we have to store the numerical number that represents the distance between two nodes, we will update the class written before, and instead of storing 1 when there is an edge or path between two nodes, we will store the distance.

Check the code below:

class Graph { 
  numberNodes; 
  adjacencyMatrix; 

  constructor(numberNodes){ 
    this.numberNodes = numberNodes; 

    this.adjacencyMatrix = []; 

    for(let i = 0; i < this.numberNodes; i++){ 
      this.adjacencyMatrix[i] = new Array(this.numberNodes).fill(0); 
    }

  }

  addEdge(node1, node2, weight){ 

    this.adjacencyMatrix[node1][node2] = weight; 
    this.adjacencyMatrix[node2][node1] = weight; 

  }

  getNeighboors(node){ 
     return this.adjacencyMatrix[node]; 
  }

  hasEdge(node1, node2){ 

    if(node1 >= 0 && node1 < this.numberNodes && node2 >= 0 && node2 < this.numberNodes){ 
      return this.adjacenMatrix[node1][node2] !== 0 && this.adjacenMatrix[node2][node1] !== 0; 
    }

    return false; 
  }

  removeEdge(node1, node2){ 
    if(node1 >= 0 && node1 < this.numberNodes && node2 >= 0 && node2 < this.numberNodes){ 
      this.adjacencyMatrix[node1][node2] = 0; 
      this.adjacencyMatrix[node2][node1] = 0; 
    }

  }

}

You may have noticed that in the code above since we now store the distance rather than 1 while checking if a path exists, we will check if the value at (node1,node2) is different to 0 .

Conclusion

In summary, this article has covered the implementation of graph data structures in JavaScript, specifically focusing on adjacency lists and adjacency matrix representations. We explored unweighted and undirected graphs, as well as variations such as directed and weighted graphs. By understanding these implementations, we gain the ability to solve graph-related problems and utilize graph algorithms effectively in real-world scenarios.

How to use Job Queue to handle email sending in your Nestjs server

Marius Vincent NIEMET — Mon, 26 Jun 2023 15:20:09 +0000

In this modern era, when a user uses your software they expect to be notified every time an action is made, sometimes by him or other users, there are many ways to inform a user, such as by sending a push notification, email, or SMS. Nevertheless, email sending is one of the most frequently used by software in addition to others, very often an email is sent when a new user signs up, resets his password, and so on.

In this article, we will see how you can send email from your Nestjs app, and to do that we will also use a mechanism called *Job Queue. *

What is a job Queue and why is it useful?

A Queue is a data structure that is used to execute some tasks in an organized and efficient manner, it follows a principle named FIFO (First in First Out) Which means the first task added to the queue will always be the first to be executed. And on the other hand job queue is a mechanism inspired by the queue data structure that is used to deal with common application scaling and performance challenges.

It’s commonly used for background processing, in applications where time or resource-consuming tasks need to be performed asynchronously, a job queue can be used to offload the work to a separate process or thread. This allows the main application to remain responsive while the tasks are executed in the background. It can be the case for email sending, imagine having thousands of users that register at the same time, and you need to send a welcome email to every one of them, this kind of scenario can slow down your application if email sending is handled by the main process instead you can use a Job queue that will store every email sending task and process them in a different thread than the main.

Overall, we want to build something like the illustration above. The client represents our NestJs application that will add jobs to the queue and then we will have some process that will get jobs from the storage and then execute them.

Requirements

To build our project we need the following requirements:

SMTP server to send our emails
Nodejs package that handles job queue
Nodejs package that handles email sending

SMTP Server

For this article, we will use the SMTP server provided by SendGrid which is commonly used, but you can also use a different SMTP server such as AWS SES or even the Gmail SMTP server. The only thing you will change here is your secrets.

So for the next step, I will assume you already have an email-sending provider ready and your secrets.

Job Queue Handling

As you may have already guessed, we are going to use Nestjs during this article, and the thing that I like about NestJs is it provides packages to handle some of the common concepts such as ORM, Job queue, Cron job, and so on that wrapped popular package, it helps to integrate those features on a Nest-friendly way to your application.

For Job Queue, NestJs provides a package named @nestjs/bull as an abstraction/wrapper on top of Bull, a popular, well-supported, high-performance Node. js-based Queue system implementation.

Bull uses Redis to persist job data, so you’ll need Redis installed on your system.

While working with Jobs queue, two concepts are essential to understand, it’s producers, storage, and workers or processors.

Producers: their role is to add jobs to the queue. While adding a job to the queue, you have to add the data needed to effectively execute the job as well.
Workers or Processors: their role is to process jobs added to the queue. they run on separate processes from the main application process.
Queue storage: a storage in which the jobs are stored, bull uses Redis as its queue storage.

Now let’s write some code:

Create a new NestJS project:

nest new job-queue-example

Package installation:

npm install --save @nestjs/bull bull

Once the installation process is complete, we can import the BullModule into the root AppModule.

import { BullModule } from '@nestjs/bull';
import { Module } from '@nestjs/common';
import { AppController } from './app.controller';
import { AppService } from './app.service';

@Module({
  imports: [
    BullModule.forRoot({
      redis: {
        host: 'localhost',
        port: 6379,
      },
    }),
  ],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}

The forRoot() method is used to register a bull package configuration object that will be used by all queues registered in the application. As we have mentioned above bull needs a Redis instance to store the queue which is why we need to provide Redis connection information.

Once the package is configured, the next step is to register the queue. It’s possible to have multiple queues for one project, every queue will handle different types of jobs like email sending which is our use case here, but your app can need to handle other tasks like notifications and Push messages, file processing, and so on.

For queue registering we use BullModule.regirsterQueue() it should be added to the module in which the job will be added to the queue. Since we want to keep this tutorial as short as possible, we won’t create another module, we will only use the app module, so let’s edit the App.module file once again:

import { BullModule } from '@nestjs/bull';
import { Module } from '@nestjs/common';
import { AppController } from './app.controller';
import { AppService } from './app.service';

@Module({
  imports: [
    BullModule.forRoot({
      redis: {
        host: 'localhost',
        port: 6379,
      },
    }),
    BullModule.registerQueue({
      name: 'emailSending',
    }),
  ],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}

Each queue is unique by its name property. A queue name is used for both adding and processing jobs, we will see that in the next part.

After this step, we have our queue package configured, and we have created a dedicated queue for email sending, now it’s time to create producer and processor.

Producer

As mentioned above, producers add jobs to the queue, and in nestjs, they are typically application services (a class that has the @Injectable decorator).

To add jobs to a queue, first, inject the queue into the service as follows:

import { InjectQueue } from '@nestjs/bull';
import { Injectable } from '@nestjs/common';
import { Queue } from 'bull';

@Injectable()
export class AppService {
  constructor(
    @InjectQueue('emailSending') private readonly emailQueue: Queue,
  ) {}
}

The queue is recognized by its name, it should be the name used to register the queue.

Create an interface to type the job data:

export interface Mail {
  from: string;
  to: string;
  subject: string;
  text: string;
  [key: string]: any;
}

Once the queue is injected into the application service, we can create a method that will add a job.

import { InjectQueue } from '@nestjs/bull';
import { Injectable } from '@nestjs/common';
import { Queue } from 'bull';
import { Mail } from './mail.interface';

@Injectable()
export class AppService {
  constructor(
    @InjectQueue('emailSending') private readonly emailQueue: Queue,
  ) {}

  async sendEmail(data: Mail) {
    const job = await this.emailQueue.add({ data });

    return { jobId: job.id };
  }
}

We have created a sendEmail method that uses the emailQueue to add a job. The parameters it receives are the data needed by the processor to send the email, it can be the user email address, the email content, and different information.

Named Jobs: jobs may have a name, which allows us to create specifics jobs, imagine our application has to send different types of email for example we can send a welcome email (when a new user is registered) or a reset password email (when a user want to reset his password) we don’t want these jobs to be handled by the same consumer.

import { InjectQueue } from '@nestjs/bull';
import { Injectable } from '@nestjs/common';
import { Queue } from 'bull';
import { Mail } from './mail.interface';

@Injectable()
export class AppService {
  constructor(
    @InjectQueue('emailSending') private readonly emailQueue: Queue,
  ) {}

  async sendWelcomeEmail(data: Mail) {
    const job = await this.emailQueue.add('welcome', { data });

    return { jobId: job.id };
  }

  async sendResetPasswordEmail(data: Mail) {
    const job = await this.emailQueue.add('reset-password', { data });

    return { jobId: job.id };
  }
}

Processor or consumer or worker

As mentioned above, processors process jobs from the queue, in Nestjs they are classes that have the @Processor decorator provided by @nestjs/bull. The decorator receives the name of the queue it has to consume.

Let’s create a file named EmailProcessor and then inside create a class with the same name and prefix by the decorator.

import { Processor } from '@nestjs/bull';

@Processor('emailSending')
export class EmailProcessor {}

Now we have our Processor class, let’s create the method (process) that will handle every job.


import { Process, Processor } from '@nestjs/bull';
import { Job } from 'bull';

@Processor('emailSending')
export class EmailProcessor {
  @Process('welcome')
  async sendWelcomeEmail(job: Job<Mail>) {
    const { data } = job;

    // send the welcome email here
  }

  @Process('reset-password')
  async sendResetPasswordEmail(job: Job<Mail>) {
    const { data } = job;

    // send the reset password email here
  }
}

The @Process is a decorator provided by @Nestjs/bull it helps us to create specific processors, in the previous section we have created different types of jobs according to the type of email we want to send, we have to do the same for processes as well.

The reset-password processor will be called every time a job of the type reset-password will be added to the queue same for the welcome process.

That’s all for the Job queue setting.

Email sending

You may notice that in the previous part, we didn’t write the logic to send emails, we have only added a comment for where the logic will be written, it’s because the emails sending will be handled by another package that we need to set up as well.

We will use a package named @nestjs-modules/mailer it’s a package for NestJS that provides email-sending capabilities, built on top of the popular library Nodemailer.

Dependencies installation:

npm i @nestjs-modules/mailer nodemailer

Update the app.module to configure the Mailer Module:

import { MailerModule } from '@nestjs-modules/mailer';
import { BullModule } from '@nestjs/bull';
import { Module } from '@nestjs/common';
import { AppController } from './app.controller';
import { AppService } from './app.service';

@Module({
  imports: [
    BullModule.forRoot({
      redis: {
        host: 'localhost',
        port: 6379,
      },
    }),
    BullModule.registerQueue({
      name: 'emailSending',
    }),
    MailerModule.forRoot({
      transport: {
        host: 'smtp.example.com',
        port: 587,
        auth: {
          user: 'username',
          pass: 'password',
        },
      },
    }),
  ],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}

From lines 18 to 27, we have added a new object which is MailerModule to make sure it works you have to update the code above with your SMTP server credentials.

Once the package is set up, it will provide a MailService class that has a method named sendEmail that will receive some parameters and then send the mail. But first, we need to create the mail template and for that, we will use handlebars. It will allow us, to create HTML templates in which we can pass dynamics parameters, such as the user name or its information, and write inline CSS to style our email.

Handlebars installation:

npm i handlebars

Then we need to create a folder inside srcin which our template will be located

Once it’s done we have to update two files the nestjs-cli.json and the app.module to notify the Mailer Module where it can find the template it needs to use.

{
  "$schema": "https://json.schemastore.org/nest-cli",
  "collection": "@nestjs/schematics",
  "sourceRoot": "src",
  "compilerOptions": {
    "assets": ["templates/**"],
    "deleteOutDir": true
  }
}

We have added the 6th line and then let’s update the app.module.

import { MailerModule } from '@nestjs-modules/mailer';
import { HandlebarsAdapter } from '@nestjs-modules/mailer/dist/adapters/handlebars.adapter';
import { BullModule } from '@nestjs/bull';
import { Module } from '@nestjs/common';
import { join } from 'path';
import { AppController } from './app.controller';
import { AppService } from './app.service';

@Module({
  imports: [
    BullModule.forRoot({
      redis: {
        host: 'localhost',
        port: 6379,
      },
    }),
    BullModule.registerQueue({
      name: 'emailSending',
    }),
    MailerModule.forRoot({
      transport: {
        host: 'smtp.example.com',
        port: 587,
        auth: {
          user: 'username',
          pass: 'password',
        },
      },
      template: {
        dir: join(__dirname, 'templates'),
        adapter: new HandlebarsAdapter(),
      },
    }),
  ],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}

Now everything is setup we can update our job queue process to send emails:

import { MailerService } from '@nestjs-modules/mailer';
import { Process, Processor } from '@nestjs/bull';
import { Job } from 'bull';
import { Mail } from './mail.interface';

@Processor('emailSending')
export class EmailProcessor {
  constructor(private readonly mailService: MailerService) {}

  @Process('welcome')
  async sendWelcomeEmail(job: Job<Mail>) {
    const { data } = job;

    await this.mailService.sendMail({
      ...data,
      subject: 'Welcome',
      template: 'welcome',
      context: {
        user: data.user,
      },
    });
  }

  @Process('reset-password')
  async sendResetPasswordEmail(job: Job<Mail>) {
    const { data } = job;

    await this.mailService.sendMail({
      ...data,
      subject: 'Reset password',
      template: 'reset-password',
      context: {
        user: data.user,
      },
    });
  }
}

First, we injected the mailService provided by our package on the email.processor which allows us the use the built-in method sendEmail.

That method requires some parameters to send the email and those parameters are declared in our mail interface.

After setting the data, we need to set the template as well. The remaining attribute named context is used if you want to pass some variables to your template for customization or to display specific information.

We have come a long way, the last thing to do here for a testing purpose is to create an endpoint that will trigger an email sending.

import { Body, Controller, Post } from '@nestjs/common';
import { AppService } from './app.service';
import { Mail } from './dto/mail.interface';

@Controller()
export class AppController {
  constructor(private readonly appService: AppService) {}

  @Post()
  async sendEmail(@Body() data: Mail) {
    await this.appService.sendWelcomeEmail(data);
  }
}

Finally, every time a user will call this endpoint with the data, the controller’s method will call the service by passing it the data needed, and then the service will add the job inside the queue and the last one of our process according to the job type, will get the job and then execute (send the email) it.

You can find the whole code for this article here.

Conclusion

In conclusion, leveraging a job queue to handle email sending in your NestJS application offers several benefits. By decoupling the email-sending process from the request-response cycle, you can improve the responsiveness and performance of your application. The job queue enables asynchronous processing of email tasks, allowing your application to quickly respond to user requests while the email jobs are handled in the background.

I hope you enjoy this article as much as I enjoyed writing it for you.

Writing Unit Tests for your Nestjs Rest API

Marius Vincent NIEMET — Tue, 11 Apr 2023 18:32:58 +0000

Software test automation refers to using software tools to automate the execution of tests and compare actual results with expected results, resulting in more efficient and accurate testing processes. There are multiple tests, including unit testing, integration testing, and end-to-end (E2E) testing. In this article, we will focus on unit tests.

Unit Testing

Unit testing involves testing individual units or components of the software in isolation from the rest of the system to ensure that it performs as expected and meets its design requirements. They use test cases that include a set of inputs and expected outputs, and they can be run automatically and frequently to ensure that changes to the code do not introduce new defects.

Requirement

For the next part of this article, we need a functional nestjs project, in a previous article we built a Rest API that handles users, we will use the same project, and you can find it here.

npm install

In the introduction, we mention that software automation refers to using software tools to automate testing, according to the project language and the type of tests you want to write there is a multitude of packages that can be used.

In our case, we have a javascript app for which we want to write unit tests, we can use Jest, Mocha, Jasmine and Chai.

For this article we are going to use Jest, because is the most popular one also when you create a new nestjs project, Jest is installed by default, so we don’t need to install other dependencies.

Unit Tests writting

Since we used a framework, we can assume that its internal working is well tested and then as a developer we are responsible for testing the code that we have written. For the project that will be ours during this article, we can notice that the code written by the developers is focused on the users.controller and the users.service so we need to write unit tests for each of them.

Testing the service layer

hen using the nestjs cli to generate a new resource, it generates files like the users.service.ts and users.controller.ts for each file, it also generated a file with the same name but with .spec.ts as extension, those files are for writing tests. So when you wanted to write a unit test you can edit those files instead of creating a new file on your own.

Let’s check what is the users.service.spec.ts content:


import { Test, TestingModule } from '@nestjs/testing';
import { UsersService } from './users.service';

describe('UsersService', () => {
  let service: UsersService;

  beforeEach(async () => {
    const module: TestingModule = await Test.createTestingModule({
      providers: [UsersService],
    }).compile();

    service = module.get<UsersService>(UsersService);
  });

  it('should be defined', () => {
    expect(service).toBeDefined();
  });
});

As you can see, we have a bunch of code that might not be familiar to you, we are going to explain every part of this code before moving forward.

The describe annotation is used to group related test cases and provide a description of what the group of tests is intended to test in our case we want to test the UsersService class. Then it declares a new variable service of type UsersService it’s an instance of the class we want to test.

The Testing Module is provided by @nestjs/testing it’s similar to the users.module.ts that we have in the project. Its goal is to provide every dependency that we need to create an instance of UserService.

To create a new Testing Module, we use the Test.createTestingModule which is also provided by @nestjs/testing that takes as parameters the dependencies, providers, controllers, and so on.

Once the Testing module is created we retrieve an instance of the UsersService class from the module and then we assign that to the variable declared above.

Here there is an important thing we need you to pay attention to, when you create a testing module, you need to provide, the class for which you are writing the tests and all dependencies that are injected in that class. The default createTestingModule receives only one parameter UsersService itself but the userRepository is missing, which means Nest can’t resolve the dependencies to create a new instance, to verify it, you can run the command npm run test you will receive the error below

As you can see Nest can’t resolve dependencies, that means before moving forward we need to provide all dependencies that are needed to create an UserService instance.

The it annotation (also called a test case) is used to define an individual test case.

Now we have reviewed the test file’s default content, we need to edit that according to our needs.

Let me show you the users.service.ts code for which we will write tests.

import { CreateUserDto } from './dto/create-user.dto';
import { UpdateUserDto } from './dto/update-user.dto';
import { User } from './entities/user.entity';

@Injectable()
export class UsersService {
  constructor(
    @InjectRepository(User)
    private userRepository: Repository<User>,
  ) {}

  async create(createUserDto: CreateUserDto) {
    return await this.userRepository.save(createUserDto);
  }

  async findAll() {
    return await this.userRepository.find();
  }

  async findOne(id: number) {
    return await this.userRepository.findOne({ where: { id } });
  }

  async update(id: number, updateUserDto: UpdateUserDto) {
    const toUpdate = await this.userRepository.findOne({ where: { id } });

    const updated = Object.assign(toUpdate, updateUserDto);

    return await this.userRepository.save(updated);
  }

  async remove(id: number) {
    return await this.userRepository.delete(id);
  }
}

As you can see, we have a dependency injected by using dependency injection and a bunch of methods, so writing unit test for this class, means we need to write tests for each method.

I have a few steps that I follow before writing tests, I suggest you to do the same:

First I check if the class for which I'm writing tests interacts with external dependencies, it’s simple to know that, you have just to check if there is dependency injection in the constructor, in this case, we have @InjectRepository(User) private userRepository: Repository<User> that means we need to mock that dependency before moving forward.

Second, I create a test case for every method inside the class, to insure I don’t forget to test something.

Mocking external dependencies: mocking is a technique used in software testing to create fake or substitute objects that mimic the behavior of real objects or dependencies. This is done to isolate the code being tested from its dependencies so that it can be tested in isolation and with greater control.

For the users.service class, we have to mock the userRepository class with its method.

const mockUserRepository = {
    save: jest.fn(),
    find: jest.fn(),
    findOne: jest.fn(),
    delete: jest.fn(),
  };

We have created an object named mockUserRepository that has attributes named in the same way as the method we want to mock. Every method receives jest.fn() that creates a new, empty mock function to mimic the behavior of a real function.

Once the mock object is created we need also to update the Testing module creation.

beforeEach(async () => {
    const module: TestingModule = await Test.createTestingModule({
      providers: [
        UsersService,
        {
          provide: getRepositoryToken(User),
          useValue: mockUserRepository,
        },
      ],
    }).compile();

    service = module.get<UsersService>(UsersService);
  });

Since we have mocked all external dependencies and provided them if you run again the command npm run test the test will pass.

Creating test cases for each method: usually, I create test cases for each method.

import { Test, TestingModule } from '@nestjs/testing';
import { getRepositoryToken } from '@nestjs/typeorm';
import { User } from './entities/user.entity';
import { UsersService } from './users.service';

describe('UsersService', () => {
  let service: UsersService;

  const mockUserRepository = {
    save: jest.fn(),
    find: jest.fn(),
    findOne: jest.fn(),
    delete: jest.fn(),
  };

  beforeEach(async () => {
    const module: TestingModule = await Test.createTestingModule({
      providers: [
        UsersService,
        {
          provide: getRepositoryToken(User),
          useValue: mockUserRepository,
        },
      ],
    }).compile();

    service = module.get<UsersService>(UsersService);
  });

  it('should be defined', () => {
    expect(service).toBeDefined();
  });

  it('create', () => {});

  it('findAll', () => {});
  it('findOne', () => {});
  it('update', () => {});
  it('remove', () => {});
});

If you have followed every step with me, your test file should look like this. We have all dependencies set, and all test cases created, then we can finally write the test code.

While writing tests, I always follow the Arrange-Act-Assert pattern, it’s a pattern that helps to make your tests maintainable and understandable.

Let’s see what it looks like:

 it('create => Should create a new user and return its data', async () => {
    // arrange
    const createUserDto = {
      firstname: 'Chadwick',
      lastname: 'Boseman',
      email: 'chadwickboseman@email.com',
    } as CreateUserDto;

    const user = {
      id: Date.now(),
      firstname: 'Chadwick',
      lastname: 'Boseman',
      email: 'chadwickboseman@email.com',
    } as User;

    jest.spyOn(mockUserRepository, 'save').mockReturnValue(user);

    // act
    const result = await service.create(createUserDto);

    // assert
    expect(mockUserRepository.save).toBeCalled();
    expect(mockUserRepository.save).toBeCalledWith(createUserDto);

    expect(result).toEqual(user);
  });

The Arrange-Act-Assert pattern suggests you to break your test into three parts:

Arrange: In this part, you arrange all data that you will need for your test, for the UsersService.create method, as you can see it receives a parameter of type CreateUserDto so we need to create a variable for it. the create method also returns a user after calling a method of an external dependency, so we need to create the returned data and then we need to mock the result of that external method, remember we want to test that method in isolation which means you can’t refer to an external dependency. To mock the result returned by the external method we use jest.spyOn
Act: in this part, we call the function we are testing by giving it the data created in the arrange part, we also store its result.
Assert: In the last part we can make our assertions, the goal is to check if the result is equal to what we expect and if the function behaves as we want. First, we check if the mocked function has been called, then we check if it has been called with the right data and finally we check if the result is equal to the mocked value.

Overall, our test for the UsersService.create method, helps us to be sure that the method behaves as expected, we are sure that the create method of the external dependency will always be called, it will be called with the data received as a parameter by the UsersService.create and if everything goes well, we will receive a brand new user with an id.

Now we can do the same for the remaining methods

import { Test, TestingModule } from '@nestjs/testing';
import { getRepositoryToken } from '@nestjs/typeorm';
import { CreateUserDto } from './dto/create-user.dto';
import { User } from './entities/user.entity';
import { UsersService } from './users.service';

describe('UsersService', () => {
  let service: UsersService;

  const mockUserRepository = {
    save: jest.fn(),
    find: jest.fn(),
    findOne: jest.fn(),
    delete: jest.fn(),
  };

  beforeEach(async () => {
    const module: TestingModule = await Test.createTestingModule({
      providers: [
        UsersService,
        {
          provide: getRepositoryToken(User),
          useValue: mockUserRepository,
        },
      ],
    }).compile();

    service = module.get<UsersService>(UsersService);
  });

  it('should be defined', () => {
    expect(service).toBeDefined();
  });

  it('create => Should create a new user and return its data', async () => {
    // arrange
    const createUserDto = {
      firstname: 'Chadwick',
      lastname: 'Boseman',
      email: 'chadwickboseman@email.com',
    } as CreateUserDto;

    const user = {
      id: Date.now(),
      firstname: 'Chadwick',
      lastname: 'Boseman',
      email: 'chadwickboseman@email.com',
    } as User;

    jest.spyOn(mockUserRepository, 'save').mockReturnValue(user);

    // act
    const result = await service.create(createUserDto);

    // assert
    expect(mockUserRepository.save).toBeCalled();
    expect(mockUserRepository.save).toBeCalledWith(createUserDto);

    expect(result).toEqual(user);
  });

  it('findAll => should return an array of user', async () => {
    //arrange
    const user = {
      id: Date.now(),
      firstname: 'Chadwick',
      lastname: 'Boseman',
      email: 'chadwickboseman@email.com',
    };
    const users = [user];
    jest.spyOn(mockUserRepository, 'find').mockReturnValue(users);

    //act
    const result = await service.findAll();

    // assert
    expect(result).toEqual(users);
    expect(mockUserRepository.find).toBeCalled();
  });
  it('findOne => should find a user by a given id and return its data', async () => {
    //arrange
    const id = 1;
    const user = {
      id: 1,
      firstname: 'Chadwick',
      lastname: 'Boseman',
      email: 'chadwickboseman@email.com',
    };

    jest.spyOn(mockUserRepository, 'findOne').mockReturnValue(user);

    //act
    const result = await service.findOne(id);

    expect(result).toEqual(user);
    expect(mockUserRepository.findOne).toBeCalled();
    expect(mockUserRepository.findOne).toBeCalledWith({ where: { id } });
  });
  it('remove => should find a user by a given id, remove and then return Number of affected rows', async () => {
    const id = 1;
    const user = {
      id: 1,
      firstname: 'Chadwick',
      lastname: 'Boseman',
      email: 'chadwickboseman@email.com',
    };

    jest.spyOn(mockUserRepository, 'delete').mockReturnValue(user);

    //act
    const result = await service.remove(id);

    expect(result).toEqual(user);
    expect(mockUserRepository.delete).toBeCalled();
    expect(mockUserRepository.delete).toBeCalledWith(id);
  });
});

Your final code should look like the code above, we have used the same pattern for each method. Now if you run the npm run test commande you will the result below:

Our six tests have passed 🔥

Testing the controller layer

Testing the controller layer is almost the same as testing a service class, the main difference here is when we try to create a Testing Module, take a look at the default code in the users.controller.spec.ts:

import { Test, TestingModule } from '@nestjs/testing';
import { UsersController } from './users.controller';
import { UsersService } from './users.service';

describe('UsersController', () => {
  let controller: UsersController;

  beforeEach(async () => {
    const module: TestingModule = await Test.createTestingModule({
      controllers: [UsersController],
      providers: [UsersService],
    }).compile();

    controller = module.get<UsersController>(UsersController);
  });

  it('should be defined', () => {
    expect(controller).toBeDefined();
  });
});

As you can see on line 10, we have an attributed name controllers that takes an array of controllers, and another attribute named provider that takes an array of providers. Here we need to do the distinction because we are testing a controller and every class that behaves as a controller is always prefixed by the @Controller annotation but the remaining stay the same, we have an external dependency here UsersService that we need to mock as we already did in the previous part.

import { Test, TestingModule } from '@nestjs/testing';
import { UsersController } from './users.controller';
import { UsersService } from './users.service';

describe('UsersController', () => {
  let controller: UsersController;

  const mockUsersService = {
    create: jest.fn(),
    findAll: jest.fn(),
    findOne: jest.fn(),
    update: jest.fn(),
    remove: jest.fn(),
  };

  beforeEach(async () => {
    const module: TestingModule = await Test.createTestingModule({
      controllers: [UsersController],
      providers: [
        {
          provide: UsersService,
          useValue: mockUsersService,
        },
      ],
    }).compile();

    controller = module.get<UsersController>(UsersController);
  });

  it('should be defined', () => {
    expect(controller).toBeDefined();
  });
});

We have created a mocked object and provided it as a dependency to create the testing module. Now let’s create test cases for each method.

import { Test, TestingModule } from '@nestjs/testing';
import { CreateUserDto } from './dto/create-user.dto';
import { UpdateUserDto } from './dto/update-user.dto';
import { User } from './entities/user.entity';
import { UsersController } from './users.controller';
import { UsersService } from './users.service';

describe('UsersController', () => {
  let controller: UsersController;

  const mockUsersService = {
    create: jest.fn(),
    findAll: jest.fn(),
    findOne: jest.fn(),
    update: jest.fn(),
    remove: jest.fn(),
  };

  beforeEach(async () => {
    const module: TestingModule = await Test.createTestingModule({
      controllers: [UsersController],
      providers: [
        {
          provide: UsersService,
          useValue: mockUsersService,
        },
      ],
    }).compile();

    controller = module.get<UsersController>(UsersController);
  });

  it('should be defined', () => {
    expect(controller).toBeDefined();
  });

  it('create => should create a new user by a given data', async () => {
    // arrange
    const createUserDto = {
      firstname: 'Chadwick',
      lastname: 'Boseman',
      email: 'chadwickboseman@email.com',
    } as CreateUserDto;

    const user = {
      id: Date.now(),
      firstname: 'Chadwick',
      lastname: 'Boseman',
      email: 'chadwickboseman@email.com',
    } as User;

    jest.spyOn(mockUsersService, 'create').mockReturnValue(user);

    // act
    const result = await controller.create(createUserDto);

    // assert
    expect(mockUsersService.create).toBeCalled();
    expect(mockUsersService.create).toBeCalledWith(createUserDto);

    expect(result).toEqual(user);
  });

  it('findAll => should return an array of user', async () => {
    //arrange
    const user = {
      id: Date.now(),
      firstname: 'Chadwick',
      lastname: 'Boseman',
      email: 'chadwickboseman@email.com',
    };
    const users = [user];
    jest.spyOn(mockUsersService, 'findAll').mockReturnValue(users);

    //act
    const result = await controller.findAll();

    // assert
    expect(result).toEqual(users);
    expect(mockUsersService.findAll).toBeCalled();
  });

  it('findOne => should find a user by a given id and return its data', async () => {
    //arrange
    const id = '1';
    const user = {
      id: 1,
      firstname: 'Chadwick',
      lastname: 'Boseman',
      email: 'chadwickboseman@email.com',
    };

    jest.spyOn(mockUsersService, 'findOne').mockReturnValue(user);

    //act
    const result = await controller.findOne(id);

    expect(result).toEqual(user);
    expect(mockUsersService.findOne).toBeCalled();
    expect(mockUsersService.findOne).toBeCalledWith(+id);
  });

  it('update => should find a user by a given id and update its data', async () => {
    //arrange
    const id = '1';
    const updateUserDto = {
      firstname: 'Chadwick',
      lastname: 'Boseman',
      email: 'chadwickboseman@email.com',
    } as UpdateUserDto;
    const user = {
      id: 1,
      firstname: 'Chadwick',
      lastname: 'Boseman',
      email: 'chadwickboseman@email.com',
    };

    jest.spyOn(mockUsersService, 'update').mockReturnValue(user);

    //act
    const result = await controller.update(id, updateUserDto);

    expect(result).toEqual(user);
    expect(mockUsersService.update).toBeCalled();
    expect(mockUsersService.update).toBeCalledWith(+id, updateUserDto);
  });
  it('remove => should find a user by a given id, remove and then return Number of affected rows', async () => {
    const id = '1';
    const user = {
      id: 1,
      firstname: 'Chadwick',
      lastname: 'Boseman',
      email: 'chadwickboseman@email.com',
    };

    jest.spyOn(mockUsersService, 'remove').mockReturnValue(user);

    //act
    const result = await controller.remove(id);

    expect(result).toEqual(user);
    expect(mockUsersService.remove).toBeCalled();
    expect(mockUsersService.remove).toBeCalledWith(+id);
  });
});

We have created tests for the UsersController methods by always mocking external dependency and using the AAA pattern to ensure maintainability and readability.

Now if we run the npm run test command we have:

Twelve tests have passed.

You can also check the code coverage by running npm run test:cov . Test coverage is a measurement of how much of your code is being tested by your automated tests. It measures the percentage of code that is executed during a test suite run.

Conclusion

Throughout this article, we have learned a bunch of things, the definition of software automation and unit testing, how to unit test a class, by mocking its external dependencies, test coverage, and some cool annotations provided by Jest.

You can find the final code here.

I hope you have enjoyed reading this article as I enjoyed writing it for you.

Building your first Rest API with Nestjs and TypeORM and test it with Postman

Marius Vincent NIEMET — Mon, 27 Mar 2023 12:49:55 +0000

Rest API

A REST API is a web API that uses HTTP requests to allow clients to access and manipulate resources, identified by URIs, using standard HTTP methods. It's designed to be stateless and is commonly used to expose data and functionality from a server-side application to client-side applications or third-party developers.

NestJS

NestJS is a Node.js framework for building scalable and efficient server-side applications, with built-in features and modules for dependency injection, middleware, routing, and more, as well as support for multiple databases and testing tools.

TypeORM

TypeORM is an Object-Relational Mapping (ORM) library for TypeScript and JavaScript that provides a simplified interface for working with relational databases using object-oriented programming techniques. It allows developers to define database schemas using TypeScript classes and decorators, supports multiple databases, and provides features such as database migrations.

Let's get started

Firstly you have to install the Nestjs CLI globally

npm i -g @nestjs/cli

and then create a new project

nest new rest-api

Before creating the project, nestjs will ask which package manager you use I choose npm but you have the choice between npm, yarn, or pnpm.

So let’s see what Nestjs has generated for us.

node_modules: a folder for our dependencies test: is where nestjs suggest you put your end-to-end tests.
boilerplate files: for package management, typescript configuration, and static code checking.
src: a folder that contains several core files it’s where we will write our code.

Once the installation is complete you can run the following command to run your app.

npm run start

This command will run your app on the port indicated in the main.ts file. You can also run

npm run start:dev

This command will run your app and reload every time you edit a file.

Now our app is created, we know how to run it, let’s build. We want an app that handles users and basic crud operations. Those users should be stored in a database.

Database set up

We are going to use TypeORM and MySQL for storage. Nestjs provides tight integration with TypeORM out-of-the-box with the @nestjs/typeorm package. We just have to install the packages.

npm install --save @nestjs/typeorm typeorm mysql2

and then connect the database by using the TypeOrmModuleprovide by @nestjs/typeormalongside the different database params (host, user name, user password, database name):

import { Module } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';
import { AppController } from './app.controller';
import { AppService } from './app.service';
import { UsersModule } from './users/users.module';

@Module({
  imports: [
    TypeOrmModule.forRoot({
      type: 'mysql',
      host: 'localhost',
      port: 3306,
      username: 'root',
      password: '',
      database: 'restapi',
      entities: ['src/**/**.entity{.ts,.js}'],
      synchronize: true,
    }),
    UsersModule,
  ],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}

Now our database is connected to the project.

Resource handling

Nestjs manage resources, a resource is a folder or module that represents a part of your app’s domain. For each resource of your app, you have a controller, service, dto, and entity. In our case we want to manage users, so users are our resource.

To create a new resource we tap the command below by adding the name of the resource we are creating.

nest g resource users

The cli will ask you two questions:

What transport layer do you use?: choose REST API
Would you like to generate CRUD entry points?: Yes

And then the following folder and files will be created.

Entity

Now we have all the boilerplate set up, we need to define the user entity, to do that we will edit the user.entity.ts file and use the decorator provide by typeOrm.

import { Column, Entity, PrimaryGeneratedColumn } from 'typeorm';

@Entity()
export class User {
  @PrimaryGeneratedColumn()
  id: number;

  @Column('text')
  firstname: string;

  @Column('text')
  lastname: string;

  @Column('text')
  email: string;
}

First, we need to add the @Entity decorator before the class definition to tell typeOrm that the next class will be an entity and then for the attributes you have two decorators, @PrimaryGeneratedColumn to create a primary and auto-increment key, and @Column to create other table columns. To define column type, we have just to define it in the @Columndecorator, in our case the three attributes are text, but you can use number, boolean, and so on.

DTO

When the user's resources have been generated for us, there had a folder called dto with two files create-user.dto.ts and update-user.dto.ts.

Dto is a class that shapes the data we want to receive while creating a new entry for our database. The create-user.dto.ts will contain the attribute we want to receive to create a new user and if those attributes aren’t available the request won’t reach the service layer. In this case, we have only three attributes we want to receive so let’s update the create-user.dto.ts.

export class CreateUserDto {
  public firstname: string;
  public lastname: string;
  public email: string;
}

Service

The service layer is the layer in which we implement our business logic, its also this layer that communicates with the layer that interacts with the database mostly called the Repository layer and the controller layer.

Typically, the service layer receives the data that has been sent by the user from the controller, the service layer applies the business logic on those data and then calls the repository layer to store it in the database.

When you generate a new resource through the cli, you have a service that is created with few methods.

import { Injectable } from '@nestjs/common';
import { CreateUserDto } from './dto/create-user.dto';
import { UpdateUserDto } from './dto/update-user.dto';

@Injectable()
export class UsersService {
  create(createUserDto: CreateUserDto) {
    return 'This action adds a new user';
  }

  findAll() {
    return `This action returns all users`;
  }

  findOne(id: number) {
    return `This action returns a #${id} user`;
  }

  update(id: number, updateUserDto: UpdateUserDto) {
    return `This action updates a #${id} user`;
  }

  remove(id: number) {
    return `This action removes a #${id} user`;
  }
}

We will edit this class. In the introduction above we say that the service layer needs to call the repository layer to interact with the database.

Since we are using TypeOrm, it provides a built-in class that comes with the method to handle the data related to a given entity.

Nestjs favor dependency injection to allow two different class to interact, dependency injection is an inversion of control (IoC) technique wherein you delegate the instantiation of dependencies to the IoC container, instead of doing it in your own code imperatively. Let’s edit the user service.

constructor(
    @InjectRepository(User)
    private userRepository: Repository<User>,
  ) {}

To inject the repository we have to do that in the constructor, by using the InjectRepository decorator, and declaring a private variable that has the type Repository provide by typeOrm with the entity we want to manage.

But every time we use an external dependency we need to import it, so that will be available in the service we wanted to use it. To import an external dependency we have to edit the module of the resource we are working with.

import { Module } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm/dist';
import { User } from './entities/user.entity';
import { UsersController } from './users.controller';
import { UsersService } from './users.service';

@Module({
  imports: [TypeOrmModule.forFeature([User])],
  controllers: [UsersController],
  providers: [UsersService],
})
export class UsersModule {}

Once it’s done the userRepository variable will be available in every function of that class.

Now the repository is set up, we can use it to read, create, update, and delete the users.

import { Injectable } from '@nestjs/common';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository } from 'typeorm';
import { CreateUserDto } from './dto/create-user.dto';
import { UpdateUserDto } from './dto/update-user.dto';
import { User } from './entities/user.entity';

@Injectable()
export class UsersService {
  constructor(
    @InjectRepository(User)
    private userRepository: Repository<User>,
  ) {}

  async create(createUserDto: CreateUserDto) {
    return await this.userRepository.save(createUserDto);
  }

  async findAll() {
    return await this.userRepository.find();
  }

  async findOne(id: number) {
    return await this.userRepository.findOne({ where: { id } });
  }

  async update(id: number, updateUserDto: UpdateUserDto) {
    const toUpdate = await this.userRepository.findOne({ where: { id } });

    const updated = Object.assign(toUpdate, updateUserDto);

    return await this.userRepository.save(updated);
  }

  async remove(id: number) {
    return await this.userRepository.delete(id);
  }
}

We have converted all methods to async methods since the repository’s methods return Promises and we have to handle those promises.

Now we have set up the service, inject the repository and each method within the service communicates with the repository to store the data it receives, we can move to the next layer.

Controller

The controller layer is the layer in which we handle the user request and send data to the service layer. Each method in the controller will be converted to an endpoint with which the final user will interact.

We have used the cli to generate the user's resource, so we already have a controller ready to be used without any changes. Let’s see how it looks:

import { Controller, Get, Post, Body, Patch, Param, Delete } from '@nestjs/common';
import { UsersService } from './users.service';
import { CreateUserDto } from './dto/create-user.dto';
import { UpdateUserDto } from './dto/update-user.dto';

@Controller('users')
export class UsersController {
  constructor(private readonly usersService: UsersService) {}

  @Post()
  create(@Body() createUserDto: CreateUserDto) {
    return this.usersService.create(createUserDto);
  }

  @Get()
  findAll() {
    return this.usersService.findAll();
  }

  @Get(':id')
  findOne(@Param('id') id: string) {
    return this.usersService.findOne(+id);
  }

  @Patch(':id')
  update(@Param('id') id: string, @Body() updateUserDto: UpdateUserDto) {
    return this.usersService.update(+id, updateUserDto);
  }

  @Delete(':id')
  remove(@Param('id') id: string) {
    return this.usersService.remove(+id);
  }
}

Since the controller has to interact with the service layer, we follow the same method as in the service layer, we inject the service class by using the dependency injection.

Each decorator before each method defines the method that will be used to call the endpoint.

The Body decorator in the update and create method are used to validate the request body. You can provide a type, here we provide the DTO class that we have created in the first part of the article to be sure that the data we receive is shaped as we want.

The decorator Param is used to getting the request’s params by defining the name of the param with its type.

Since the endpoint method is defined, and the request params and body are validated, we call the service method that will apply our business logic on it and then call the repository to interact with the database.

Testing the API

We will use postman to test our API.

Postman is a tool for building, testing and documenting APIs. It simplifies the process of making HTTP requests and inspecting responses, allows for easy collaboration among team members, and supports automated testing and integration with CI/CD pipelines.

Get the users list

To get the users list, we have to set the endpoint URL and the GET method.

Create a new user

To create a new user we use the same URL, but we change the method to POST, we also need to add a Content-Type attribute in the header of the request to indicate that the data being sent in the request body is in JSON format.

Once the content-type attribute is set up. we can edit the request body to send our data.

The request body is where we set the user’s data and as you can see we receive a 201 status which means the user has been successfully created and we also receive the user data with his id.

Delete a user

To delete a user we have to add his id to the URL and change the method to DELETE.

Edit a user

To edit a user we have to add his id to the URL, change the method to PATCH, and set the value we wanted to edit in the request body.

As we can see in the image above, the user’s email has been changed.

Conclusion

In conclusion, building a REST API with NestJS, TypeORM, and MySQL can be a powerful combination for creating robust and scalable web applications. With NestJS, you can take advantage of its modular architecture and dependency injection to build highly testable and maintainable code. TypeORM simplifies the process of working with databases by providing a rich set of tools and features for handling data access and manipulation. Postman provides a user-friendly interface to test your Rest API.

I hope this article has been helpful in guiding you through the process of building a REST API with these technologies.

Graph data structure explained

Marius Vincent NIEMET — Tue, 06 Dec 2022 12:41:14 +0000

A graph is a non-linear data structure consisting of a node (vertex) and edge. The nodes are sometimes also referred to as vertices and edges are lines or arcs that connect two nodes in the graph.

Graph terminology

Adjacency

A vertex is said to be adjacent to another vertex if there is an edge connecting them:

Path

It’s a sequence of edges that allows you to go from vertex a to vertex b. In the illustration below, the path between vertex and vertex is marked in red.

Directed Graph

A graph in which there is only one direction between two vertices. An arrow represents the direction.

Undirected Graph

A graph in which we can have two directions between two vertices, it’s represented by a line without an arrow.

Unweighted or weighted Graph

if the edges in your graph have weights then your graph is said to be a weighted graph, if the edges do not have weights, the graph is said to be unweighted. Weight is a numerical value attached to each individual edge. In a weighted graph relationships between nodes have a magnitude and this magnitude is essential to the relationship we’re studying.

Graph representation

Adjacency Matrix

An adjacency matrix is a way of representing a graph as a matrix of booleans. A finite graph can be represented in the form of a square matrix the boolean value of the matrix is indicated if there is a direct path between two vertices.

If the cell at row i and column j have the value 1, it means that the node i and j are adjacent.

Space complexity: O(n²)
Time complexity: Check if two nodes are adjacent takes O(1). Getting all adjacent nodes to a given node takes O(n).

Adjacency Linked List

It's a way to represent a graph as an array of linked lists but since I mostly use javascript, I use the built-in Map and Set data structure instead of a linked list.

Space complexity: O(n+e)
Time complexity: Check if two nodes are adjacent takes O(n). Getting all adjacent nodes to a given node takes O(1).

Graph Traversal

Like Tree, there is two way to traverse a graph, Breadth-first and Depth-first traversal. To keep this article as short as possible, those two traversals will be tackled in the different articles that will be published soon.

Conclusion

Well, we have learned a lot of things in this one, I hope you enjoy reading this as much as I enjoyed the writing process.

Depth-First Search of a Binary Tree

Marius Vincent NIEMET — Sat, 03 Dec 2022 12:43:26 +0000

In this previous blog post we learned what a binary tree is, how it can be represented, and the different types of binary trees, in conclusion, we’ve said that a binary tree, unlike other data structures, can be browsed in two different ways breath-first search and depth-first search.

This post will be focused on depth-first search.

What’s depth-first search Tree traversal

The depth-first search is an algorithm that allows you to traverse an entire Tree. The logic is that you keep moving forward until you reach a leaf (a node that hasn’t left or right child) then backtrack according to the method you are using.

Depth-first search Tree traversal can be written in three ways: Pre Order, In Order, and Post Oder traversal. Each way can be written recursively and iteratively. The way you choose influences the result. Let’s me show you the difference:

Pre Order

The hint is on the name, Pre-order, when you are writing a pre-order traversal, you access the parent node then the left and the right child. If you traverse the tree above with the Pre Order method your result should be:

Let’s write the recursive and iterative implementation with javascript

Recursive

function dfsPreOrderRecursive(node, arr = []){
  if(node){
    arr.push(node.val); 
    if(node.left) dfsPreOrderRecursive(node.left, arr);
    if(node.right) dfsPreOrderRecursive(node.right, arr);
  }
}

Explanation since you know in which order you have to access the nodes, recursive methods are mostly straightforward. Here We have to access the parent node and then its children. so we push the current node’s value into the result array, and then check if it has a left or right child, if so we call the function by giving the child node and the result array.

Iterative

function dfsPreOrderIterative(root){
  const stack = [root]
  const result = []; 
  let current;

  while(stack.length){
    current = stack.pop(); 
    result.push(current.val);

    if(current.right) stack.push(current.right); 
    if(current.left) stack.push(current.left); 
  }

  return result; 

  }

Explanation: the pre-order traversal is the straightforward method, if there isn’t any constraint you should prior the method. Since we don’t need to check if a node has any children, we push the current node’s value to the result and then check if it has child, if so they will be added into the stack. We push the right left before the right because we are using a stack and it follows the FIFO order.

In Order

For the in-order traversal, you have to access the left child before backtracking to the parent and then the right child. You have to keep moving forward while your current node has a left child. if you traverse the tree above with the In-Order method your result should be:

Let’s write the recursive and iterative implementation with javascript.

Recursive

function dfsInOrderRecursive(node, arr = []){ 
  if(node){
    if(node.left) dfsInOrderRecursive(node.left, arr);
    arr.push(node.val); 
    if(node.right) dfsInOrderRecursive(node.right, arr);
  }

  return arr; 

 }

Explanation: since you know in which order you have to access the nodes, recursive methods are mostly straightforward. Here We have to access the left child before backtracking to the parent and then the right child, so we check if the current node has a left child if so, we call the function by giving the child, once it's done we push the node’s value into the result array and finally we check if a right child exists, if so we call the function by giving the right child.

Iterative

function dfsInOrderIterative(root){
  const stack = []; 
  const result = [];
  let current = root; 

  while(stack.length || current){
    while(current){
      stack.push(current); 
      current = current.left; 
    }
    current = stack.pop();
    result.push(current.val); 
    current = current.right;
  }

  return result;
}

Explanation: For the in-order traversal, we have to push the left child, the parent and then the right child. That means we have to keep moving forward since there is left child. For the iterative implementation, the current loop while browse all the left child and then if there’s no left child, the current node’s value will be pushed into the result.

Post Order

For the post-order traversal, you have to access children before backtracking to the parent. You have to keep moving forward while your current node has children. if you traverse the tree above with the post-Order method your result should be:

Let’s write the recursive and iterative implementation with javascript.

Recursive

function dfsPostOrderRecursive(node, arr = []){
  if(node){
    if(node.left) dfsPostOrderRecursive(node.left, arr);
    if(node.right) dfsPostOrderRecursive(node.right, arr);
    arr.push(node.val);
  }

  return arr;
}

Explanation: since you know in which order you have to access the nodes, recursive methods are mostly straightforward. Here We have to access children before the parent, so we check if the current node has a left or right child if so, we call the function by giving the child and then we push the value into the result array.

Iterative

function dfsPostOrderIterative(root){
  const s1 = [root];
  const s2 = [];
  const result = []; 
  let current;

  while(s1.length){
    current = s1.pop();

    if(current.left) s1.push(current.left);
    if(current.right) s1.push(current.right);

    s2.push(current.val);
  }

  return s2.reverse();

}

Complexity analysis

Every approach that we have seen in this post has the same time and space complexity O(n) even the recursive approach because each recursive function use stack under the hood, to track the elements that you are traversing. Your choice will depend on the question you have to solve.

Binary Heap in javascript

Marius Vincent NIEMET — Wed, 12 Oct 2022 07:42:24 +0000

A binary Heap is a binary tree like data structure that is always a complete binary tree. A complete binary tree is a binary tree in which all the levels are completely filled except possibly the lowest one, which is filled from the left. There are two types of binary heap: max heap and min heap.

Max Heap

A Max heap is a binary heap in which the parent’s value is always greater than the value of its children.

Min Heap

A Max heap is a binary heap in which the parent’s value is always less than the value of its children.

Representation

Since a binary tree is always a complete binary tree, it can be also represented by using an array.

The traversal that is used to build the array is Level order traversal. When a binary heap is represented by using an array you can find the parent, right or left child by using the following:

Array[0]: root node
Array[(i*2)+1]: the left child of the node that’s at the position i
Array[(i*2)+2]: the right child of the node that’s at the position i
Array[(i-1)/2]: the parent of the node that’s at the position i

Let’s see how to create a binary Heap.

Add new value

While you are trying to add a new value to a binary heap, this new value should be added at the end of the binary heap to keep it as a complete binary tree. Once the value is added, you have to compare it with its parent, if you are created a max heap you have to check if the new value is greater than its parent, if so you have to swap them, and continue with this process until the value is at the right place (greater than its children). In case it's a min heap you have if the new value is less than its parent, if so you follow the same process as a max heap (swap child and parent). The illustration below will help to visualize:

Javascript implementation

function add(val){
  //add at the end of the array
  this.arr.push(val)

  let i = this.arr.length - 1; 

  while(this.arr[i] >  this.arr[Math.round((i-1)/2)]){

    if(this.arr[i] >  this.arr[Math.round((i-1)/2)]){
      let temp = this.arr[i]; 
      this.arr[i] = this.arr[Math.round((i-1)/2)];
      this.arr[Math.round((i-1)/2)] = temp;

     i = Math.round((i-1)/2);
   }
  }

Explanation

First of all we had the new value at the end of the array by using the javascript method push and then we save the new value position since the value is at the end its position is the array length minus 1. While the new value is greater than its parent we will adjust the tree. Inside of the loop, we check if the value is greater than its parent, if so we swap them and save its new position. Since the value has changed a position we need to compare it again with its new parent to be sure it has the right position. The number of swaps depends on the Tree length and the Tree length is equal to O(log n) so the time complexity to add a new value into a binary Heap is O(log n).

Deletion

While trying to delete a node from a binary heap, you can only delete the root element, when it’s a max heap, you retrieve the first largest value and when it’s a min heap, you retrieve the first smallest value. Once the node is deleted, you have to push the last node of the binary heap to the root position, to keep the tree complete. Then you can start to adjust the tree, first, you have to compare the root’s children, once you have to greater of them, you can compare it with the parent, if it is greater than the parent you have to swap their positions, continue this process until the node it at the right position (greater than its children). The illustration below will help to visualize:

Javascript implementation

// if its a max binary heap we retrieve the  largest element 
// if its a min binary heap we retrieve the smallest element 
function delete(){
  let result = this.arr[0]; // retrieve a the first element 
  this.arr[0] = this.arr.pop(); // push the last element at the first place to keep the binary heap full 

  // adjust the binary tree by comparing each parent with its children 

  i = 0;
  while(!(this.arr[i] > this.arr[2*i+1] && this.arr[i] > this.arr[2*i+2])){

  if(this.arr[2*i+1] > this.arr[2*i+2]){
    if(this.arr[i] < this.arr[2*i+1]){
      let temp = this.arr[i]; 
      this.arr[i] = this.arr[2*i+1]; 
      this.arr[2*i+1] = temp; 

      i = 2*i+1; 
    }
    }else{
       if (this.arr[i] < this.arr[2*i+2]){
        let temp = this.arr[i]; 
        this.arr[i] = this.arr[2*i+2]; 
        this.arr[2*i+2] = temp; 

        i = 2*i+2; 
      }
    }
  }

  return result;
}

Explanation

First of all we save the first element and push the last element at its position and we save the position 0 into a variable. Now we have to adjust the binary Heap, so to adjust after a deletion we need to check if the root element is at the right place, and to check that, you need to first compare it to its children, to know which is greater and then compare the greater child with the parent, if the child is greater we have to swap them, and so on the process continue until the root element is greater than its children. Time complexity is O(Log n) because the number of swaps depends on the tree length.

Create a new Binary Heap

Most time we will create a heap from another data structure, now we gonna see how to make a max heap from an array.

function createMaxHeapFromArray(arr) {
  let binaryHeap = [];

  for (let j = 0; j < arr.length; j++) {
    binaryHeap.push(arr[j]);
    let i = binaryHeap.length - 1;

    while (binaryHeap[i] > binaryHeap[Math.round((i - 1) / 2)]) {
      if (binaryHeap[i] > binaryHeap[Math.round((i - 1) / 2)]) {
        let temp = binaryHeap[i];
        binaryHeap[i] = binaryHeap[Math.round((i - 1) / 2)];
        binaryHeap[Math.round((i - 1) / 2)] = temp;

        i = Math.round((i - 1) / 2);
      }
    }
  }

  return binaryHeap;
}

The code above shows you how to create a max heap from an array, basically, you have to create a new array called maxHeap, traverse the entire array, and at each lap add the current value into the max heap by following to code we have written in the first part of the article. The time complexity of this is O(nlogn) since insertion into a heap takes O(logn) and we do that for all the array’s values (n). The space complexity is O(n).

Well, now we know what a binary heap, the types of binary heap, how to add, delete and even create a binary heap from a given array but we don’t know in which case a binary heap can be useful during an interview. A binary heap is useful when you face questions like Kth Largest Element in an Array or Kth Smallest Element in a Sorted Matrix because you can create a max heap or min heap from the given data structure and then retrieve the first kth element of your binary heap.

Conclusion

Well, we have learned a lot of things in this one, I hope you enjoy reading this as much as I enjoyed the writing process.