DEV Community: Madhavam Saxena

Persistent Volumes and Persistent Volumes Claims

Madhavam Saxena — Thu, 19 Jan 2023 18:12:44 +0000

Persistent Volumes:
Persistent volumes is the concept that comes into play when disadvantages of hostpath are observed. Persistent volumes, unlike hostpath and emptydir, are simply independent of pods and slave / worker Node, hence, data stored in persistent volumes is available even after a pod or even a slave / worker node crashes.

Persistent Volume Claim:
PVC is present on the slave / worker node which is basically connected to a persistent volume. With the help of this connection, worker / slave nodes and pods are able to write into a persistent volume.
A single claim can be attached to multiple persistent volumes, also, different volume claims on different volumes, attached with different pods on different node, i.e. you’ve full flexibility with the volumes and data remains available.

Defining Persistent Volume:

apiVersion: v1
kind: PersistentVolume
metadata:
    name: host-pv
spec:
    capacity:
        storage: 4Gi
        volumeMode: Filesystem
        accessMode:
            - ReadWriteOnce
            # - ReadOnlyMany
            # - ReadWriteMany
    hostPath:
        path: /data
        type: DirectoryOrCreate

spec.capacty: Defines the capacity of the volumes.
spec.capacity.storage: Here we define the size of persistent volume in Gi format which basically is defining a Gb.
spec.capacity.volumeMode: It is of two types:
Filesystem
Block
spec.capacity.accessMode: As the name suggests, it defines the mode of access. Since I’ve takes example of hostPath, therefore, I only can use ReadWrtiteOnce access mode as hostPath allows different pods within same slave / worker node to write into the volume, therefore only these pods can access it and ReadWriteOnce access mode defines that Different number of pods can access a persistent volume that are part of same salve / worker node.

Persistent Volumes Claims:
Persistent volume claims are used to map the pods with the persistent volumes, hence this claim needs to be defined in the pod that needs to access the persistent volume.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
    name: host-pvc
spec:
    volumeName: <name_of_the_volume_that_is_to_be_mapped>
                host-pv
    accessModes:
        - ReadWriteOnce
    resources:
        requests:
            storage: <counter_part_of_volume_Defined_in_deployment.yml>
                     1Gi

spec.resources: defines the resource
spec.resources.requests: requests the resource
spec.resources.requests.storage: maximum as that is defined in the deployment.yml

Adding this Persistent Volume claim in the pods:

apiVersion: apps/v1
kind: Deployment
metadata:
    name: story_deployment
spec:
    replicas: 1
    selector:
        matchLabels:
            app: story
    spec:
        template:
            metadata:
                labels:
                    app: story
            containers:
                - name: story
                  image: deocker_repo/<image_name>
                  volumeMounts:
                    - mountPath: /app/story
                      name: story-volume
            volumes:
                - name: story-volumes
                  persistentVOlumeClaim:
                    claimName: <name_of_the_persistent_volume_claim>

Volumes in Kubernetes - Part 2 (hostPath)

Madhavam Saxena — Tue, 17 Jan 2023 17:50:06 +0000

hostPath:
The emptyDir volume is a very basic volume and has a few downsides as well. Let's suppose if the count of pods running is more than one, and data is stored in these emptyDir volumes and if a pod crashes, the data stored in that volume also gets lost.
Here comes hostPath driver in the action.
hostPath driver basically allows us to map a path on the worker / slave node on which the pods are running to a specific path inside the pods, i.e. the data from inside of this path will be exposed to multiple pods, therefore, multiple pods can use same path on the host (worker) machine to store the data instead of specific path for specific pods.
hostPath is very much like the bind mount in docker, similar to docker we can perform read / write actions in this hostPath.

Syntax:

apiVersion: apps/v1
kind: Deployment
metadata:
    name: <name_of_deployment>
spec:
    replicas: 1
    selector:
        matchLabels:
            a: b
    spec:
        template:
            metadata:
                labels: 
                    a: b
            containers:
                - name: <name_of_container>
                  image: deocker_repo/<image_name>
            volumes:
                - name: <name_of_volumes>
                  hostPath:
                      path: <path_on_the_host_machone_where_data_should_be_stored>
                      type: <declares_how_the_above_declare_path_is_to_be_handled>
                            DirectoryOrCreate

type: Directory defines the path that already exists.
Creates defines that the path does not exist but needs to be created.

Disadvantage of hostPath:
Let's suppose there are multiple pods, running on multiple slaves / worker nodes, now defining hostpath lets you map a volume on the slave / worker node to the N number of pods running on that particular slave / worker node i.e. all other pods that are running on other slaves / worker nodes will not be able to access the volume defined in this slave / worker node.

Volumes in Kubernetes- Part 1 (emptyDir)

Madhavam Saxena — Mon, 16 Jan 2023 18:04:04 +0000

Volumes:
State in kubernetes:
Data created and used by your application which must not be lost.
Types of data
1. User generated data
2. Intermediate data (either memory or temp. database)

Volumes in kubernetes are used to store the data of running pods. There are situations in live production environments when production pods get terminated, data inside those pods is important, now to avoid the loss of this data volumes are used in kubernetes.
Kubernetes can mount volumes in containers. We can set an instruction in the pod template which basically is part of deployment.yml that mounts a volume to the container that gets created with the pod which came up via deployment.yml.
Kubernetes provides various types of volumes:
Local Node: A directory on the worker node where the pod is running.
Cloud Provider Specific

Lifetime of a volume by default depends on the lifetime of a pod as volumes are part of the pod.

In the deployment.yml file:
We need to add the volume key inside the spec.template.spec. It basically is a list of volumes. We can give names to every volume that we create and these volumes will be accessible by all the containers inside that pod.
Under the name, we have to set up the volume type.

apiVersion: apps/v1
kind: Deployment
metadata:
    name: <name_of_deployment>
spec:
    replicas: 1
    selector:
        matchLabels:
            a: b
    spec:
        template:
            metadata:
                labels: 
                    a: b
            containers:
                - name: <name_of_container>
                  image: deocker_repo/<image_name>
            volumes:
                - name: <name_of_volumes>
                  emptyDir: {}

emptyDir: {}
EmptyDir is a type of volume that is used in kubernetes to store data.
This means we want to use all the default values and no configuration is declared instead default is used as it.
It simply creates a new empty directory whenever the pod starts and keeps the directory and writes the data till the lifespan of the pod. Containers can easily perform Write operation over it but if the pod dies, emptyDir dies with the pod i.e. this volume also terminates, and when a pod is created again a new empty directory is created.

Now, after creating a volume we need to bind the volume with the container, therefore we need to make it available inside of the container in container configuration with the volumeMounts keyword.
It takes the list. It takes mountPath i.e. which is the container internal path where the volume is mounted .
name keyword is used to mount the volume/s that we created above to the mountPath i.e. path internal of containers.

Liveness Probes in kubernetes

Madhavam Saxena — Tue, 10 Jan 2023 16:06:49 +0000

Suppose you’ve created a service and a deployment with the replica count as ‘n’, this means that there will be at least ‘n’ number of pods running which will be part of the service we create. Now you need to run your application on these pods, but are you sure if the pods and containers are healthy or not?
The answer is NO and to get sure for the same we use livenessprobe. Now this livenessprobe is defined inside the deployment.yml file.

Syntax:

livenessProbe:
httpGet: 
        path: /status
        port: 8080
    periodSeconds: 10
    initalDelaySeconds: 30

Example:

apiVersion: apps/v1
kind: Deployments
metadata:
      name: first-dep
      labels: 
           app: hello-world
spec:
    replicas: 3
    selector:
        matchLabels:
            app: hello-world
    template:
        metadata:
            name: first-pod
            labels: 
                app: hello-world
        spec:
            containers:
                -   name:
                    image:                              
                    livenessProbe:
                        httpGet: 
                            path: /status
                            port: 8080
                        periodSeconds: 10
                        initalDelaySeconds: 30

path: it is used to check if the desired output is being received at defined path
port: it is the port number of the container i.e. port number at which container is exposedRunning.
periodSeconds: This is used to define how often the health check will be performed
initialDelaySeconds: this defined time is used as delay between the two simultaneous health checks.

Rollback and Updating Deployment

Madhavam Saxena — Mon, 09 Jan 2023 17:29:08 +0000

Scaling a deployment:

kubectl scale <object_type> <object name> --replicas=<count_of_objects_needed>

Updating a deployment:

kubectl get deployments
kubectl set image deployment <name_of_deployment> <name_of_new_image_that_you_want_to_give / old_image_name>=<actual_name_of_image(full path of repository)>:<tag>

Now what it does is, this command basically looks for the actual_name_of_image on docker repository, and downloads it from there. Now after downloading it updates the current image of deployment with this downloaded image, and that's how a deployment is updated.

Note: Images are differentiated on the basis of tags, i.e K8s re-downloads the image if and only if a new tag is passed to it, if the tag is not mentioned while updating a deployment, it will not update the image. Hence, passing a tag while executing a set image command is necessary.

To check status of updation, one can execute:

kubectl rollout status deployment <name_of_deployment> 
                          OR
kubectl rollout status deployment/<name_of_deployment>

Now, what if the tag passed doesn’t actually exists? I.e.

kubectl set image deployment/<name_of_deployment> <old_image_name>=<repo_name/image_name>:<some_random_value>

It will still show that the deployment is updated although the deployment has not actually updated. To check if the deployment is actually updated, one can test it via:

kubectl rollout status deployment/<name_of_deployment>

This will lead to an infinite loop of updation of deployment as, k8s follows rolling strategy, but since the tag passed doesn’t exists, the image will not be downloaded, and hence the new pod that's trying to spinnup will not get live, and the old pod will not terminate, and will be stuck in this.
In such cases, we need to rollback to the previous deployment by the following command:

kubectl rollout undo deployment/<name_of_deployment>
kubectl rollout status deployment/<name_of_deployment>

We can check the history of deployment by following command:

kubectl rollout history deployment/<name_of_deployment>

This wil list out all the deployments in the form of revisions.

To particularly inspect a deployment, we can use:

kubectl rollout history deployment/<name_of_deployment> --revision=<nth_deployment>

This also prints the image which was used in that deployment.

Declarative approach for creating a service

Madhavam Saxena — Sun, 08 Jan 2023 16:21:44 +0000

Service.yml:

apiVersion: v1
kind: Service
metadata:
  name: <name_of_service>
spec:
  selector:
    <labelkey_of_pod_that’s_supposed_to_be_part_of_service>: <labelvalue_of_pod_that’s_supposed_to_be_part_of_service>
  ports: <value_entered_is_as_list>
    protocol: ‘TCP’
    port: 
    targetPort:
  type: 
protocol: <by_default_value_is_TCP>
port: <port_number_at_which_service_will_be_exposed> i.e. outside_world_port
targetPort: <port_at_which_container_will_be_exposed> i.e. port_inside_container
  type: <clusterIP / Nodeport / Loadbalancer>

//refer “Exposing a deployment in kubernetes” blog for getting detailed idea of type.

.kind: Defines the type of k8s object being created
.metedata.name: Assigns the name to the service object.
.spec: Defines the specification of service object
.spec.selector: Here, the value of selector is always set to “Matchlabels” by default and hence, we don't define it.
What is does is, it basically looks for the pods that have the labelkey and labelvalue pair as defined under selectors, and then controls those pods under this service.
So, for e.g. if key value pair is,

spec:
  selector:
    app:backend

So, it will look for all the pods having this key,value pair as label. Now, if backtracking to the origin of pods, we find that pods are created via deployment. Therefore, now on joining the dots we can conclude that, the value in selector should be same as:
.spec.selector.matchLabels

.spec.ports: The value accepted here is in list format as we can enter more than one value and hence its defined as portS.
.spec.ports.protocol: By default is ‘TCP’
.spec.ports.port:
port_number_at_which_service_will_be_exposed

To apply this service, run:

kubectl -f apply service.yml

Exposing a deployment in kubernetes

Madhavam Saxena — Sat, 07 Jan 2023 18:35:35 +0000

Exposing a deployment:

kubectl expose deployment <name_of_deployment> --type=LoadBalancer/NodePort/ClusterIP(Default) --port=<port_no_at_which_deployment_needs_to_be_exposed>

type= LoadBalancer is only available for those environments that support load balancer like AWS.
Now via above command what we are doing is, we are creating a service that will be exposed on the assigned port number.
Now, What is service?

Service:
Service basically help us to reach the pods and container running in a pod. Another k8s object responsible for exposing pod within cluster and outside it.
If you’ll see, Pods do have an IP address but we can’t access them from outside of cluster Via this IP, also, whenever the pod gets terminated and another pod spinnup’s, the IP address of the pod will change.
Service, basically groups these pods together and assign a particular IP address to that group of pods. Benefit of this is, pods might get terminated but the service will never terminate, and hence, pods inside the service will be reachable. Also, when on exposing this service, we make pods reachable from outside of cluster. Without service, it becomes very difficult to reach out these pods and hence we need service.

--type=NodePort :
Actually means that this deployment should be exposed with the help of worker/slave node on which its running, which makes this deployment accessible from outside, but a more better approach is loadbalancer.

--type=LoadBalancer :
This not only helps to assign IP to the service, but also, routes the load in the balanced way to the service.

---type=ClusterIP is default type and lets you expose the deployment via cluster IP.

To see the running services

kubectl get services

This is basically an imperative approach of running or basically creating a service.
For declarative approach, stay tunned.

Deployment in Kubernetes

Madhavam Saxena — Fri, 06 Jan 2023 17:49:26 +0000

What is Deployment in Kubernetes?
Deployment basically is a higher level Kubernetes object which is used to manage POD's and their version.

Creating a deployment
There are basically two methods to create a deployment:

Imperative
Declarative We are going to learn about Declarative method. Attached below is the snippet for creating a deployment in k8s.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: <Name Of Deployment that you want to assign>
  labels: <Labels that you want to attach with deployment>
spec:
  replicas: <Defines the total number of pods expected to run>
  selector: <Selects the pods to control on the basis of 
             selector.Matchlabel is the mostly used type>
    matchLabels:
      <labelkey>: <labelvalue>
  template: <Template is basically a defined structure for 
             containers running inside the pod>
    metadata:
      labels: <if needed>
      name: <if needed>
    spec:
      containers: <Values assigned is in the form of list>
        - name: <Name of container that you want to give>
          image: <Image of container>
          ports: <port num. on which its expected to listen
                  This value is also given in form of list>
            - containerPort:

So what is happening above?
We have created an kubernetes object whose kind is deployment. Name of this deployment can be defined at .metadata.name and labels can be attached to this deployment at .metadata.labels .
Once the deployment is created, it generates a replicaset (rs) which is responsible to create the declared number of pods. Now, after defining replicas, we have declared selectors. So, what's the use of selector?

A selector is used to basically select the pods that needs to be controlled via deployment which we are creating. Now the selection of pods is done on the basis of labels i.e. labels mentioned in matchlabels and that .metadata.label should be same, else it might can be cause of misbehavior or Error.
.spec.template is supposed to be a defined structure for the pod following which the pod('s) will spin up.
.spec.template.spec defines the container/'s . Value if container is given in the form of list.

How to run this file?

kubectl apply -f deployment.yml

How to see deployments?

kubectl get deployments

How to delete a specific deployment?

kubectl delete deployment <name_of_deployment>

Basic flow / working of k8s

Madhavam Saxena — Wed, 04 Jan 2023 17:50:28 +0000

*Master Node *:
When a slave node needs some action to be performed it directly communicates with API server.
Now API server forwards the request of slave node to Controller Manager which then makes a check via API server to etcd Cluster of present scenario of requesting slave node.
Now after this, the Controller Manager checks if the present state is the same as the desired state or not.
If a match is not established, the Controller Manager instructs the Kube scheduler to do the needful.

*Slave Node *:
First of all a container is created with the help of a container engine inside a POD.
For creation of a POD kubelet makes a request to the API server and then the Master node plays its role.
After creation of POD, Kubeproxy assigns it IP address so that a POD can be uniquely identified.

Brief Overview of Higher Level Kubernetes Objects

Madhavam Saxena — Tue, 03 Jan 2023 17:24:30 +0000

They can be understood as Kubernetes plugins that allow us to maintain the versions of PODS and help us to rollback to the previous version in case of POD crash.

Replication set :
Provides us with the facility of auto scaling and auto healing.
Deployment :
Another K8s higher level object used for maintaining the versions of PODs which in long term helps in the process of rollback and recovery.
Services :
This helps in assigning static IP addresses to PODs and perform other networking actions over them.
Volume :
This provides ephemeral storage to PODS which even persist after crashing of PODS.

NOTE :
For single cloud like AWS commands generally start with kubectl
For on premises command starts with kubeadm
For hybrid clouds, kubefed is used, which stands for kube federation.
A container can be declared in both imperative and declarative form.
Declarative is when you declare the requirements of a container in a yaml file whereas Imperative is when you manually create a container using CLI commands.

Architecture of Kubernetes(Slave Node)

Madhavam Saxena — Mon, 02 Jan 2023 18:13:45 +0000

Kubelet :
Kubelet is responsible for controlling the PODs, i.e it conveys the message to the API server which then forwards it to the Controller Manager to check if the desired state is maintained or not.
It always listens to Master Node, on port number 10255 which can also be changed according to the need.
Provides the feedback of success or failure to master.
KubeProxy :
Kubeproxy is responsible for assigning dynamic IP addresses to PODs and establishing communication between PODs.
PODs are not allowed to communicate directly with each other, therefore KubeProxy is needed.
It runs on each slave node and makes sure that each POD get its own IP.
Container Engine :
The Container Engine is not supposed to be the internal part of K8s. Also, the major role of the Container Engine is to create containers in POD.
Generally it's seen, Docker is used as a Container Engine but ContainerD, and Rocket are some Container Engines that can be used as well.

Responsible for pulling container images, exposing the container on the ports that are mentioned in manifest.
Also, looks after initialization and termination of containers.
POD :
POD is supposed to be the atomic / control unit of K8s.
POD’s can not communicate directly with each other.
POD’s have containers in them.
Ideally a POD is supposed to have one container in it but we can keep more than one container as well, in such case, all the containers will be tightly coupled i.e. if any of the container starts malfunctioning, other containers that are connected with it will also get affected and at the end, that POD will be deleted.
Every time a new POD is created meaning if a POD has more than one container out of which a container of POD is not created then, Control Manger will create a whole new POD which will be assigned a whole new ip address.
Either the entire request of creating n number of PODs is fulfilled or not even a single container associated to that request is created.
Multi Container PODs share access to the same volume and memory space.

Limitations of POD :
By default no facility of autohealing and autoscaling, for that higher objects of K8s are added.
In case of crashing of PODs, there is a dependency over Higher Level Kubernetes objects for rollback.

Architecture of Kubernetes(Slave Node)

Architecture of Kubernetes(Master Node)

Madhavam Saxena — Sun, 01 Jan 2023 08:57:24 +0000

Components of Master Node

API Server :
API server is supposed to be the front face that communicates with nodes and the other components of the control plane. Every node provides its requirements to the API server (i.e. Manifest file) which are then forwarded by API server to Controller Manager.
Controller Manager :
Controller Manager checks if all the requirements that are asked by the slave node are getting fulfilled or not i.e. if the actual state is equivalent to desired state or not. It's responsible to maintain balance b/w the two states.
Here comes the role of the etcd cluster.
etcd Cluster :
If K8s is on cloud, then Cloud-Control Manager will be in role and if K8s is on premises, then Kube-Control Manager is found in action.
Etcd is responsible for storing metadata and status of a cluster in key-value pair format.
It's consistent and is highly available.
Source of truth for cluster i.e. you can find all the information of cluster with etcd.
Scheduler :
Kube scheduler responds to those requests which are generated by the user for creation and management of POD.
If the manifest user has not mentioned the node on which it wants POD to be created, in that case, Kube Scheduler finds the best Salve node on which POD can be created and then creates the POD on that node.
It gets information for hardware configuration from configuration files and schedules the POD’s on the slave nodes accordingly.

Architecture Diagram of Kubernetes
(https://dev-to-uploads.s3.amazonaws.com/uploads/articles/frqk1mrp4d81dyn41v2h.png)