Kubernetes: Designing and Building Applications

This article provides various aspects of application design and development in Kubernetes, covering topics such as container images, running jobs and cron jobs, building multi-container pods, utilizing Init containers, leveraging volumes and their types, and utilizing persistent volumes.

Container Image

An image is a self-contained file that encapsulates all the essential software and executables required to execute a container. It serves as a compact and portable package that enables you to deploy and operate your application across numerous containers.
Docker is one tool that you use to create your own images. A Dockerfile defines what is contained in the image. The docker build command builds an image using the Dockerfile.
Checkout to my article "Docker: Image Creation, Management, and Registry" to create and manage Docker images.

Running Jobs and CronJobs

Jobs are specifically designed to execute containerized tasks and ensure their successful completion. It can also be used to run multiple Pods in parallel.
A sample manifest of a Job:

apiVersion: batch/v1
kind: Job
metadata:
  name: example-job
spec:
  template:
    spec:
      containers:
      - name: my-container
        image: my-image:latest
        command: ["echo", "Hello, Kubernetes!"]
      restartPolicy: Never
  backoffLimit: 3
  activeDeadlineSeconds: 10

The Job object is in Kubernetes "batch" APIs. The restartPolicy is set to "Never" to ensure the Job completes without retries. The backoffLimit is set to 3, meaning the Job will be retried at most 3 times in case of failures. The activeDeadlineSeconds is the maximum time Kubernetes allows to run the job.

CronJobs are designed to execute Jobs at regular intervals based on a predefined schedule. One CronJob object is like one line of a crontab (cron table) file on a Unix system.
A sample manifest of a CronJob:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: example-cronjob
spec:
  schedule: "*/5 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: my-container
            image: my-image:latest
            command: ["echo", "Hello, Kubernetes CronJob!"]
          restartPolicy: OnFailure
      backoffLimit: 3
      activeDeadlineSeconds: 10

The schedule field uses a cron expression to define the desired schedule for running the Job. It represents minute, hour, day of the month, month, and day of the week (from left to right).

The restartPolicy for a Job or CronJob Pod must be OnFailure or Never.

Building Multi-Container Pods

Multi-Container Pods are Pods that consist of multiple containers collaborating within a single unit. They involve the inclusion of more than one container within a shared Pod.

An example manifest of a multi-container Pod:

apiVersion: v1
kind: Pod
metadata:
  name: example-pod
spec:
  volumes:
    - name: shared-volume
      emptyDir: {}
  containers:
    - name: main-container
      image: main-image:latest
      volumeMounts:
        - name: shared-volume
          mountPath: /shared-data
    - name: sidecar-container
      image: sidecar-image:latest
      volumeMounts:
        - name: shared-volume
          mountPath: /shared-data

Multi-container Pods are recommended for scenarios where containers require tight coupling and the need to share resources like network and storage volumes.

It is advisable to utilize multi-container Pods selectively, focusing on situations where close collaboration and resource sharing between containers are necessary.

Multi-Container Design Patterns

In the Sidecar pattern, a sidecar container works alongside the main container to provide assistance or perform complementary tasks. One example is when the main container serves files from a shared volume, and the sidecar container periodically updates those files.
In the Ambassador pattern, an ambassador container acts as a proxy for network traffic to and/or from the main container. One example is when the main container needs to communicate with a database, and the ambassador container proxies the traffic to different databases based on the environment.
In the Adapter pattern, an adapter container is responsible for transforming the output of the main container in some way. One example is when the main container produces log data in a non-standard format without timestamps, and the adapter container transforms the data by adding timestamps and converting it into a standard format.

Using Init containers

An Init container is designed to fulfill a specific task before the main container within a Pod is initiated. It operates on a one-time basis, executing its designated job and then concluding its execution. Once the Init container completes its task, the main container within the Pod is initiated and begins its execution.
An example manifest of a Pod using Init container:

apiVersion: v1
kind: Pod
metadata:
  name: example-pod
spec:
  containers:
    - name: main-container
      image: main-image:latest
      command: ["sh", "-c", "echo Main container started; sleep 10; echo Main container completed"]
      ports:
        - containerPort: 8080
          protocol: TCP
  initContainers:
    - name: init-container
      image: init-image:latest
      command: ["sh", "-c", "echo Init container started; sleep 5; echo Init container completed"]

A few use-cases of Init containers are:

• Separate image: Init containers offer the capability to execute start-up tasks using a distinct image, which may involve software not included or required by the main container's image.
• Delay startup: Init containers provide a means to defer the start of the main container until specific prerequisites are satisfied.
• Security: Init containers offer a secure approach to executing sensitive start-up processes, such as consuming secrets, in isolation from the main container.

Exploring Volumes

A volume serves as a storage solution external to the container's file system, offering additional storage capabilities for containers.
Volume vs VolumeMounts
Volume: Specified within the Pod configuration, the volume defines the specifics regarding the storage location and configuration for data within a Pod.
VolumeMounts: Configured within the container specification, volumeMounts associate a volume with a particular container and determine the directory path at which the volume data will be accessible during runtime.

Volume types

The volume type determine where and how data storage is handled. There are a lot of volume types but a few important ones are:
hostPath
Data is stored in a specific location directly on the host file system on the Kubernetes node where the Pod is running.
An example manifest of Pod using hostPath volume:

apiVersion: v1
kind: Pod
metadata:
  name: hostpath-pod
spec:
  containers:
    - name: example-container
      image: example-image:latest
      volumeMounts:
        - name: data-volume
          mountPath: /app/data
  volumes:
    - name: data-volume
      hostPath:
        path: /data
        type: Directory

hostPath volume has further four types:

• Directory: Mounts an existing directory on the host
• DirectoryOrCreate: Mounts an existing directory on the host and creates if does not exist.
• File: Mounts an existing single file on the host
• FileOrCreate: Mounts an existing single file on the host and creates if it does not exist.

emptyDir
Data is stored in an automatically managed location on the host file system. Data is deleted if the Pod is deleted.
An example manifest of a Pod using emptyDir volume:

apiVersion: v1
kind: Pod
metadata:
  name: emptydir-pod
spec:
  containers:
    - name: example-container
      image: example-image:latest
      volumeMounts:
        - name: data-volume
          mountPath: /app/data
  volumes:
    - name: data-volume
      emptyDir: {}

persistentVolumeClaim
Data is stored using a persistentVolume.

Using PersistentVolume

A PersistentVolume provides the ability to abstract storage details from Pods, treating storage as a consumable resource without directly exposing the underlying storage implementation.

PersistentVolume vs PersistentVolumeClaim
A PersistentVolume (PV) defines an abstract storage resource that is available for consumption by Pods. It encompasses specific information regarding the type and capacity of storage that it provides.
A PersistentVolumeClaim (PVC) declares the need for storage and specifies the required characteristics, such as storage type. It automatically binds to an available PersistentVolume (PV) that fulfills the defined requirements. Once bound, the PVC can be mounted within a Pod like any other volume.

An example of a Pod manifest YAML file that includes a PersistentVolumeClaim (PVC) and PersistentVolume (PV):

apiVersion: v1
kind: Pod
metadata:
  name: pvc-pod
spec:
  containers:
    - name: example-container
      image: example-image:latest
      volumeMounts:
        - name: data-volume
          mountPath: /app/data
  volumes:
    - name: data-volume
      persistentVolumeClaim:
        claimName: my-pvc
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: my-pv
spec:
  capacity:
    storage: 5Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: /data

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

This example demonstrates the usage of PVC and PV in a Pod. The PVC is bound to an available PV that meets the specified requirements, and the PV is mounted as a volume within the Pod.

By focusing on container images, running jobs and cron jobs, building multi-container pods, utilizing Init containers, exploring volumes and their types, and leveraging persistent volumes, developers can create robust and resilient applications in the Kubernetes ecosystem. Adopting these approaches and understanding the underlying concepts will empower developers to unlock the full potential of Kubernetes for their application deployments.

Comments

[Benoit COUETIL 💫]

Don't you think that presenting something as central as deployment would be a good addition to your article ?

No one creates solo pod in real-life applications 😉

[Yasir Rehman]

Thanks for your feedback.

I wrote this article specifically for development perspective how to design different types of workloads of an application to various Kubernetes objects. There are multiple things to be considered in running and managing these objects in Kubernetes. "Deployments" is one of those objects that be used to get more benefits (along with other objects) from Kubernetes.

I will plan another article to extend it including necessary considerations for running applications in Kubernetes. :)

[Benoit COUETIL 💫]

You presented job/cronjob for this purpose, which is great (even if you presented pod after them, which is a lower level resource, so a bit weird).

Looking forward to your next article 😉