DEV Community: Shubham Yadav

microservice and monolithic architectures.

Shubham Yadav — Fri, 25 Nov 2022 15:48:05 +0000

Before microservices, let's see what a monolithic architecture is.

A monolithic architecture is like a huge container within which all software components of an application are assembled, and all the components share a same database.

👍 advantages of monolithic architecture:

easy deployment.
easier to develop.
end to end testing can be performed faster.
easy debugging.

👎 Disadvantages of monolithic architecture:

Scalability: Individual components cannot be scaled.
Reliability: if one feature goes down, the entire system goes down
Adoption of technology is restricted.
Lack of flexibility.

🚀 Microservice architecture

A microservices architecture is an architectural solution that is based on a collection of independently deployable services. These services have their own business logic and database, and they serve a specialised purpose.

👍 Advantages of Microservices

Flexible scaling
Continuous deployment: frequent and faster release cycles.
Highly maintainable and testable.
Independently deployable.
High reliability: one failure does not bring down the entire application.

👎 Disadvantages of Microservices

Difficult to manage a large number of services.
Each microservice has its own set of logs, making debugging more difficult.

Components of Microservices Architecture:

🔵 Identity Provider: These client requests are subsequently forwarded to identity providers, who authenticate the requests and relay them to API Gateway. The queries are subsequently routed to internal services using a well-defined API Gateway.

🔵 Static Content. Following internal communication, the Microservices distribute static material to a cloud-based storage service that may provide it directly to clients via Content Delivery Networks (CDNs).

🔵 Management: This component is responsible for balancing the services on nodes and identifying failures.

🔵 Service Discovery: it keeps a list of services on which nodes are situated, it acts as a guide for Microservices to discover the path of communication between them.

Follow me on twitter

JWT (JSON Web Token)

Shubham Yadav — Tue, 16 Aug 2022 19:07:00 +0000

What JWT is used for?

A JWT is used for authorisation, authorisation is making sure that the user that sends a request to your server is the same user that actually logged in during the authentication process.
it's authorizing that a user is an access to this particular system and the way is usually done is by using session.

for example, you have a session ID that you send down in the cookies of the browser and every time the client makes a request they send that session ID up to the server and the server checks its memory that what the user has that session ID it finds that user and it does the authorisation to make sure the user has access.

In JWT instead of cookies, it uses a JSON web token which is what it stands for.

first, we're gonna take look at the traditional user login system that uses sessions and cookies to store the user. so the first that happens is the user actually logs in from the client by posting their email and password for example as soon as it gets to the server, the server is going to the authentication to make sure that the user is correct with the email and password it stores that user inside the session which stored in the memory of the server and it gets a unique ID and it sends the ID back to the browser using a cookie so that the browser always has that session ID that it sends up to the server every time It makes a request.
for example, the client makes another request that the session ID gets sent along with the cookie and the server makes the calculations and checks the session memory and checks based on the ID it verifies the user and sends the response back to the browser.

The other form of authentication is JWT it works very similarly in the beginning, we make a POST request with email and password to the server just like before but instead of storing information on the server inside the session memory what happens is that the server creates a JSON Web Token (JWT) and it signs it with its own secret key so the server knows if you change it then it's invalid it can check that with its secret key.

The main difference here is nothing is stored on the server, the server doesn't store the user, this JWT has all the information about the user built into it. so the server sends JWT back down to the browser and it can choose to store that. for example, it can do cookie storage and it works similarly.

Know the client sends a request to the server and it includes JSON Web Token so it knows what user s authenticating with it and the server checks the token with its own secret key and it verifies that this web token has not been changed. If the client changed the JSON Web Token and changed the user information then it can say it's invalid but if nothing is changed with JWT and the user is authorized to use that resource it sends the response back to the client.

Why use JWT?

Now that we talked about how JWT works, and what it is now let's see why would you want to use JWT.
let's take a look at a very simple one of the common use cases of JWT.

Here we have two different serves, we have a bank that owns a server that runs all their banking applications and their banking website and all the baking information, but they also own a separate server and this takes care of all the retirement plans they allow people to invest and do retirement plans on a completely separate web application, but they want their users that login into the bank to also be able to automatically log into their retirement account, so the switch from bank to retirement server they don't want their user to log back in so that it makes it seamless and looks like they are on the same application. This is very common in large-scale industries

What happens is that when you use a normal session-based server your session is stored in the bank server and not inside the retirement server so the users need to log back in because they need to have their session stored in the retirement server because the session ID from the client is not found in the retirement server.

But when you use JWT you share the same secret key between the bank and the retirement server and all you need to do is send the same JWT from the client to both of them and you'll be authenticated both times without having to re-log back in.

SO the important thing about JWT is, no matter how many different servers you have no matter how many different applications, or load balancers you have it doesn't matter, the user can authenticate with any of those servers as long has the same secret key between them.

app.post('/api/login', async (req, res) => {
    const { username, password } = req.body
    const user = await User.findOne({ username }).lean()

    if (!user) {
        return res.json({ status: 'error', error: 'Invalid username/password' })
    }

    if (await bcrypt.compare(password, user.password)) {

        const token = jwt.sign(
            {
                id: user._id,
                username: user.username
            },
            JWT_SECRET
        )

        return res.json({ status: 'ok', data: token })
    }

    res.json({ status: 'error', error: 'Invalid username/password' })
})

above is an example of how you can use JWT for logging a user in the application using a post request in nodejs.

Kubernetes Overview

Shubham Yadav — Wed, 08 Jun 2022 21:15:00 +0000

kubernetes

Kubernetes is a container orchestration tool, developed by Google in 2014. in simple terms, it manages containers from docker containers or from other technology.

Kubernetes basically helps you manage containerized applications made up of hundreds or maybe thousands of containers in different environments like physical, virtual, or cloud environments.

If you are not familiar with containers, check out my another blog.

What features do orchestration tools provide?

High availability: In simple words, high availability means that your application does not have any downtime, so it is always accessible by the users.
Scalability: This means that the application has high performance, it loads fast and users have a very high rate of response rate from the application.
Disaster Recovery: it basically means that if the infrastructure has any problems like data is lost or the servers explode. the infrastructure has to have some kind of mechanism to pick up the data and restore it to the latest state.
Some other features of Kubernetes are scheduling, scaling, load balancing, fault tolerance, deployment, automated rollouts, rollbacks, etc.

Kubernetes Architecture

Overview

Kubernetes architecture consists of two parts a master node and a worker node or node which is basically a simple server a physical or virtual machine, all the nodes join together to form a cluster.

Components of master node

The master node is responsible for managing the cluster and it basically monitors and manages the nodes and the pods in a cluster so whenever there is a failure of a node it will move the workload to the other working node and take care of the overall cluster.

API server

It is the front-end for the Kubernetes control panel. The API server is responsible for the communications, it exposes APIs so that the users can interact with the API server.
The Users, management devices, and the command line interface all talk to the API server to interact with the Kubernetes cluster.

scheduler

it is responsible for scheduling the pods over multiple nodes, It can read the configuration from a configuration file, and accordingly, it selects the nodes where the pods can be scheduled.
It watches newly created pods that have no node assigned and selects a node for them to run on.
it obtains all this information from configuration files and indirectly from etcd datastore

Control Manager

there 4 types of controllers behind control manager:

Node controller: it is responsible for noticing and responding when the nodes go down.
Replication Controller: It is responsible for maintaining the correct numbers of pods for every replication controller object in the system.
Endpoints Controller: It is responsible for all the services and communication.
Service account and token controller: It is responsible for maintaining accounts and API access.

etcd

It is a distributed key-value store used to hold and manage the critical information that distributed systems need to keep running.

Components of worker node

A worker node is a simple server physical or virtual machine where containers are deployed.

Kublet

It is an agent running on each node, it communicates with the components from the master node.
kubelet make sure that containers are running and healthy in a pod. In case Kubelet notices any issues with the pods running on the worker nodes, it tries to restart the Pod on the same node or a different node.

the kubelet takes a set of PodSpecs that are provided through various mechanisms and ensures that the containers described in those PodSpecs are running and healthy.

Kube-Proxy

It is a network agent which runs on each node and is responsible for maintaining network configuration and rules. It also exposes services to the outside world. Kube-Proxy is the core networking component in Kubernetes

Pods

A pod is the smallest unit of Kubernetes. So Pod is basically an abstraction over a container, it creates a running environment or a layer on top of the container, Pods is usually meant to run one application container inside of it, you can run multiple containers inside one pod but it's only the case if you have one application container and a helper container or some service that has to run inside of that pod,
We interact and manage the containers using Pods.
Each Pod can communicate with each others.

Containers

Containers are Runtime Environments for containerized applications. We run container applications inside the containers. For more detailed information, check out my blog on Docker

Service and Ingress

Service is a permanent IP address that can be attached to each pod.
The lifecycles of service and pods are not connected so even if the pod dies the service and its IP address will stay.

Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the cluster. Traffic routing is controlled by rules defined on the Ingress resource.

Use/Installation

online Kubernetes labs

The following are the online platforms/playgrounds in case you test something on Kubernetes:

Kubernetes playground - https://www.katacoda.com/courses/kubernetes/playground
Play with K8s - https://labs.play-with-k8s.com/
Play with Kubernetes classroom - https://training.play-with-kubernetes.com/

Kubernetes Installation tools

- Minikube

minikube is a very quick way to install Kubernetes on your system especially if you have limited resources minikube will be very ideal to install and setup Kubernetes on the system, here we will be using a single system for master and for worker node so the same system it will have master instance and the worker instance.

- Kubeadm

Cloud-based Kubernetes services

GKE: Google Kubernetes Engine.
AKS - Azure Kubernetes Service.
Amazon EKS

usage

$ brew install minikube

The above command also installs kubectl as its dependency.

minkube start

after this command, you can use the following command to see the nodes.

 $ kubectl get nodes
NAME       STATUS   ROLES                  AGE   VERSION
minikube   Ready    control-plane,master   64s   v1.20.7

the minikube has 'master' as its role.

**So there are 2 ways to create a pod **

*1. directly :
*

apiVersion: v1
kind: Pod
metadata:
  name: nginx-pod
spec:
  containers:
  - name: nginx
    image: nginx:1.14.2
    ports:
    - containerPort: 80

To create the Pod shown above, run the following command:

~ kubectl create -f nginx-pod.yaml
pod/nginx-pod created

list all the pods

~ kubectl get pods 
NAME        READY   STATUS    RESTARTS   AGE
nginx-pod   1/1     Running   0          9m25s

delete the pod

~ kubectl delete pod nginx-pod
pod "nginx-pod" deleted

2. Using deployment

The pod is the smallest unit in the Kubernetes cluster but usually, in practice, you are not creating pods or you are not working with pods directly. There is an abstraction layer over the pods that is called deployment, so this is what we create and that creates the pods underneath.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-depl
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80

usage:

~ kubectl apply -f nginx-depl.yaml
deployment.apps/nginx-depl created

List all the deployments

~ kubectl get deployment
NAME         READY   UP-TO-DATE   AVAILABLE   AGE
nginx-depl   1/1     1            1           77m

List all the Pods.

~ kubectl get pods
NAME       READY   STATUS    RESTARTS   AGE
nginx-depl   1/1     Running   0          31m

~ Executing a Pod

~ kubectl exec -it nginx-depl -- /bin/sh
# ls
bin  boot  dev etc  home  lib lib64  media  mnt  opt proc  root  run  sbin  srv  sys  tmp  usr  var
# hostname
nginx-depl
# exit

delete deployment

 kubectl delete deployment nginx-depl

What is the difference between Pods and Deployment?

Pods:

Runs a single set of containers
Good for occasional dev purposes
Rarely used directly in production

Deployment:

Runs a set of identical pods
Monitors the state of each pod, updating as necessary
Good for dev
Good for production

linux commands🐧

Shubham Yadav — Tue, 04 Jan 2022 13:34:58 +0000

Types of shells

Bourne shell (sh shell)
C shell (csh shell)
Z shell (zsh shell)
Bourne again shell (bash shell)

linux commands

basic commands

echo - print a string
ls - list files in a directory
cd - change directory
pwd - print working directory
mkdir - make a directory
rmdir - remove a directory
rm - remove a file
cp - copy a file
mv - move a file
cat - concatenate files and print on the standard output
grep - search for a pattern in a file

User accounts

whoami - print the name of the current user
id - print the user ID of the current user
sudo - run a command as root
su - run a command as another user

Download files

curl <url> - download a file from the internet
wget <url> - download a file from the internet

Check OS version

uname -a - print operating system information
lsb_release -a - print operating system information

Package managers

RPM: Red Hat Package Manager

rpm -i <package> - install a package
rpm -e <package> - remove a package
rpm -q <package> - query a package

YUM: Yum Package Manager

yum install <package> - install a package
yum remove <package> - remove a package
yum list <package> - query a package
yum repolist - list available repositories

Services

service <service> start - start a service
systemctl start <service> - start a service
systemctl stop <service> - stop a service
systemctl status <service> - check the status of a service
systemctl enable <service> - enable a service to start at boot
systemctl disable <service> - disable a service to start at boot

VI editor

vi <file> - open a file in vi editor
to switch to insert mode: i
to switch to command mode: esc
to switch to visual mode: v
to switch to replace mode: r
to delete a character: x
to delete a line: dd
to delete a word: dw
to copy a line: yy
to paste a line: p
to save a file: :wq
to quit vi: :q
to quit vi without saving: :wq!
to quit vi and exit: :q!

Networking

ifconfig - print network interface configuration
route - print routing table
netstat - print network statistics
ping <host> - send ICMP echo request to a host
traceroute <host> - trace the route to a host
dig <host> - query DNS information
nslookup <host> - query DNS information
nmap <host> - scan a network
ss - print network statistics

Routing

route add -net <network> <gateway> - add a route
route del -net <network> - delete a route
route add -host <host> <gateway> - add a route

Docker Made Easy🐳

Shubham Yadav — Mon, 13 Dec 2021 16:00:00 +0000

What is Docker?

Docker is a tool for running applications in an isolated environment.
It is similar to Virtual Machine but it's much faster and does not require a lot of memory and an entire operating system to operate.
The cool thing about docker is that your application runs in exact same environment for eg. if it works on my machine it will definitely work on your machine, or if it works on the staging environment it will also work in a production environment.

What can I use Docker for?

Docker provides the ability to package and run an application in an isolated environment called containers. Containers are great for continuous integration and continuous deployment (CI/CD) workflows. Using docker you can significantly reduce the delay between writing code and running it in production.

Containers vs VM

Containers

containers are an abstraction at the app layer that packages code and dependencies together. Multiple containers can run on the same machine and share the OS kernel with other containers, each running as an isolated process in userspace.

Virtual Machine

An application on a VM requires a guest OS and thus an underlying hypervisor to run. The hypervisor is used to create multiple machines on a host operating system and it manages virtual machines. These virtual machines have their own operating system and do not use the host’s operating system. They have some space allocated.

Docker Architecture

The Docker daemon

The docker daemon listens to the API requests and manages Docker objects such as images, containers, network, and volumes.

The Docker client

The docker client is the primary way to interact with Docker. When you use the command using docker, the client sends these commands to deamon, which carries them out. The Docker client can communicate with more than one daemon.

Docker Registries

A Docker registry stores Docker images. Docker Hub is a public registry that anyone can use. When you pull the image Docker by default look for it on Docker Hub.

Docker Objects

Images

An image is a read-only template with instructions for creating a Docker container. Often an image is based on another image, with some additional customization.
You can create your own images or can use those created and published in a registry. To build your own image, you create a Dockerfile with a simple syntax for defining the steps needed to create the image and run it. Each instruction in a Dockerfile creates a layer in the image.

Image is a template for creating an environment of your choice. It is also a snapshot.
Images are immutable. Once built, the files making up an image do not change. Images can be stored locally or in remote locations like hub.docker.com.

Containers

A container is a runnable instance of an image. You can create, start, stop, move, or delete a container using CLI.

Docker Compose

Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration. It is a declarative way of describing the components of a Docker application.

Docker Volume and Bind Mounts

By default, all the files inside the container are stored on a writable container layer.
The data does not persist when the container is no longer running.
Docker has two ways to persist data.
1. Volumes
2. Bind Mounts

Volumes

Volumes are a way to persist data between containers.
volumes are stored in a part of the filesystem which is managed by Docker.
a given volume can be mounted into multiple containers.
when no running container is using the volume, it is still available.

Bind Mounts

Bind mounts are stored anywhere on the host system.
bind mounts are used to mount a volume from a container into another container.
In Bind Mounts, the file or directory is referenced by its full path in the host system.

To see basic commands like pulling building images etc.click here

Now that you know what docker is you can now create your application with tons of containers that interact with each other so you can imagine the effort to manage hundreds of containers manually, every time a container crashes or the communication between them is not working you have to manually fix these problems. Here is where the container-orchestration system comes into play, It helps in automating application deployment, scaling, and management. One of the very widely used COE is Kubernetes.

connect :twitter

Docker Commands 🐳

Shubham Yadav — Thu, 05 Aug 2021 17:46:57 +0000

docker images commands

Shows the help for the docker images command.

docker images --help

pulls an image from a registry.

 docker pull [image] `

Pulls an image from a registry and tag it.

 docker pull [image]:[tag]

Lists all images in the local registry.

 docker images

list image id only

 docker images -q

list dangling images

 docker images -f "dangling=true"

Runs an image in a new container.

 docker run [image]

Lists all running containers.

docker ps

Runs an image in a new container and names it.

 docker run --name [container] [image]

Runs an image in a new interactive container.

 docker run -it [image]

Runs an image in a new interactive container and names it.

 docker run -it --name [container] [image]

Inspects a container.

 docker inspect [container]

Removes an image from the local registry.

 docker rmi [image]

Removes an image from the local registry and force remove it.

 docker rmi -f [image]

Shows the history of an image.

 docker history [image]

docker container commands

Lists all running containers.

 docker ps

Runs an image in a new container.

 docker run [image]

Starts a container.

 docker start [container-name] / [container-id]

Stops a container.

 docker stop [container-name] / [container-id]

Pauses a container.

 docker pause [container-name] / [container-id]

Unpauses a container.

 docker unpause [container-name] / [container-id]

Restarts a container.

 docker restart [container-name] / [container-id]

Shows the running processes of a container.

 docker top [container-name] / [container-id]

Shows the resource usage of a container.

 docker stats [container-name] / [container-id]

Inspects a container.

 docker inspect [container-name] / [container-id]

Attaches to a container.

 docker attach [container-name] / [container-id]

docker attach usage : Use docker attach to attach your terminal’s standard input, output, and error (or any combination of the three) to a running container using the container’s ID or name. This allows you to view its ongoing output or to control it interactively, as though the commands were running directly in your terminal.

Kills a container.

 docker kill [container-name] / [container-id]

Removes a container.

 docker rm [container-name] / [container-id]

Shows the history of a container.

 docker history [container-name] / [container-id]

dockerfile commands

Builds an image from a Dockerfile.

 docker build [path]

Builds an image from a Dockerfile and tag it.

 docker build -t [image] [path]

docker compose commands

Shows the configuration of a service.

 docker compose config [service]

Starts services.

 docker-compose up [service]

Starts services and keep them running.

 docker-compose up -d [service]

Stops services.

 docker-compose down [service]

Lists all running services.

 docke compose ps

Useful Git Commands.💻

Shubham Yadav — Tue, 01 Jun 2021 07:25:58 +0000

Git is an example of a distributed version control system commonly used for open source and commercial software development.

Let's go :
initalize git in files/directories.

git init

1.Clone the repository in your local system.

git clone https://github.com/<your-user-name>/<repo-name>

2.Connect to remote

git remote add origin <url>

3.To check the current status of the repository.

git status

4.To add specific file to the staging area.

git add <file-name>

5.To add all changed file to staging area.

git add .

6.To unstage a certain file

git restore --stagged <filename>

7.To see recent changes in the repository.

git diff

8.To give a message and commit.

git commit -m "your-message"

9.To see the commit history.

git log

10.To see last specific commits (eg. Last 3 commits).

git log -3

11.To discard the specific commit.

git revert <commit-token>

12.To undo the commit and bring back changes to staging area.

git reset --soft HEAD <no._of_commit_to_revert>

13.To show remote URLs

git remote -v

14.To fetch the changes from origin to your local system.

git pull origin

15.To create a branch named branch-name.

git branch <branch-name>

16.To make changes in the specific branch.

git checkout <branch-name>

17.To merge sub branch to main branch.

git merge <branch-name>

18.To delete a specific branch.

git branch -d <branch-name>

19.To push the recent commits.

git push origin <branch-name>

20.Connect to remote

git remote add origin <url>

21.To push the current branch and set the remote as upstream

git push --set-upstream origin master

22.To rename branch name.

git branch -m <branch-name>

23.change url for git remote repo

git remote set-url origin