DEV Community

Aisalkyn Aidarova
Aisalkyn Aidarova

Posted on

KUBERNETES CONTINUOUS DEPLOYMENT

Goal

After reading this material, you should understand what happens after CI creates a Docker image and how a DevOps engineer deploys, verifies, troubleshoots, and rolls back an application in Kubernetes.

==================================================

1. FROM CI TO CD

Continuous Integration (CI) prepares and validates the application.

Typical CI flow:

Developer

GitHub

GitHub Actions / Jenkins

Lint

Unit Tests

SonarQube

Security Scanning

Trivy

Docker Build

AWS ECR

At the end of CI, we have a tested and scanned Docker image stored in a container registry.

Example:

restaurant-company:

AWS ECR is a container registry.

ECR stores Docker images.

ECR does NOT deploy the application.

After the image is ready, Continuous Delivery / Continuous Deployment begins.

==================================================

2. WHAT IS CONTINUOUS DEPLOYMENT?

The purpose of CD is to take an approved artifact and deploy it into an environment.

In our project:

Artifact:
Docker Image

Container Registry:
AWS ECR

CD Tool:
Jenkins

Container Platform:
Kubernetes

Managed Kubernetes Service:
AWS EKS

Our CD flow:

AWS ECR

Docker Image

Jenkins

AWS Authentication

AWS EKS

Kubernetes Manifests

Deployment

ReplicaSet

Pods

Containers

Service

Ingress / Load Balancer

Users

IMPORTANT:

Jenkins does not run our application containers.

Jenkins automates the deployment process.

Kubernetes is responsible for running and maintaining the application.

==================================================

3. WHAT DOES JENKINS DO?

In our project, Jenkins performs tasks such as:

  1. Checkout the CD repository

  2. Verify required tools

AWS CLI
kubectl
Git

  1. Authenticate to AWS

  2. Connect to the EKS cluster

  3. Create or verify the Kubernetes namespace

  4. Apply application configuration

  5. Apply Kubernetes Deployment

  6. Apply Kubernetes Service

  7. Apply additional production resources

  8. Verify the Kubernetes rollout

  9. Verify that the application resources are healthy

Without Jenkins, a DevOps engineer could perform these commands manually.

Jenkins allows us to automate the same process consistently.

==================================================

4. KUBERNETES DESIRED STATE

One of the most important Kubernetes concepts is:

DESIRED STATE

We describe what we want.

Kubernetes continuously works to make the actual environment match that desired state.

Example:

replicas: 3

This means:

Desired State = 3 Pods

Current state:

Pod 1 → Running
Pod 2 → Running
Pod 3 → Running

Now imagine Pod 2 crashes.

Actual State = 2 Pods

Desired State = 3 Pods

Kubernetes detects:

Actual State != Desired State

Kubernetes creates another Pod.

This behavior is called:

SELF-HEALING

==================================================

5. DEPLOYMENT, REPLICASET, POD, CONTAINER

Understand this relationship:

Deployment

ReplicaSet

Pod

Container

Deployment

Defines the desired application state and deployment strategy.

ReplicaSet

Maintains the required number of Pods.

Pod

The smallest deployable Kubernetes workload.

Container

Runs the actual application process inside the Pod.

Jenkins does NOT directly create and maintain individual Pods.

Jenkins applies the Deployment configuration.

Kubernetes controllers handle the rest.

==================================================

6. NAMESPACE

Namespaces logically separate Kubernetes resources.

Examples:

restaurant-dev
restaurant-qa
restaurant-staging
restaurant-prod

Our production namespace:

restaurant-prod

To see Pods in that namespace:

kubectl get pods -n restaurant-prod

If you run:

kubectl get pods

without specifying a namespace, kubectl normally checks the default namespace.

Therefore:

No resources found in default namespace

does NOT necessarily mean that your application does not exist.

You may simply be looking in the wrong namespace.

==================================================

7. DOCKER IMAGE AND ECR

The Kubernetes Deployment references a Docker image.

Example:

image: ACCOUNT_ID.dkr.ecr.REGION.amazonaws.com/restaurant-company:

The basic process is:

Kubernetes creates a Pod

Scheduler selects a Node

Node receives the workload

Node contacts ECR

Node pulls Docker image

Container is created

Application starts

The Docker image is NOT pushed directly into Kubernetes.

The image is stored in ECR.

Kubernetes Nodes pull the required image from ECR.

==================================================

8. WHY IMMUTABLE IMAGE VERSIONS MATTER

For learning, you may see:

restaurant-company:latest

In production, immutable version tags are generally better.

Example:

restaurant-company:a84f12c

The value could represent a Git commit SHA.

This gives us traceability.

Git Commit
a84f12c

CI Pipeline

Docker Image
restaurant-company:a84f12c

ECR

CD

Kubernetes

If a production incident happens, we can identify exactly which code version was deployed.

==================================================

9. IMAGEPULLBACKOFF

A common Kubernetes error is:

ImagePullBackOff

This means Kubernetes is having trouble pulling the container image.

Possible causes include:

Wrong image repository

Wrong image tag

Image does not exist

Registry permissions

Authentication problem

Incorrect AWS account or region

CPU architecture incompatibility

The first command:

kubectl get pods -n restaurant-prod

Then investigate the Pod:

kubectl describe pod POD-NAME -n restaurant-prod

Pay special attention to:

Events

Events may show:

Failed to pull image

Image not found

Access denied

Authentication error

No match for platform in manifest

==================================================

10. THE ARCHITECTURE PROBLEM FROM OUR LAB

During our lab, we encountered:

no match for platform in manifest

The Docker image was built for:

linux/arm64

The Kubernetes worker environment required:

linux/amd64

This can happen when building containers on an Apple Silicon Mac.

We rebuilt the image for AMD64:

docker buildx build \
--platform linux/amd64 \
-t IMAGE \
--push .

The important lesson is NOT:

ImagePullBackOff always means ARM64.

It does not.

The correct lesson is:

ImagePullBackOff

Investigate

kubectl describe

Read Events

Find the actual root cause

==================================================

11. CRASHLOOPBACKOFF

Another common error is:

CrashLoopBackOff

This is different from ImagePullBackOff.

ImagePullBackOff:

Kubernetes tries to pull image

Image cannot be pulled

Container cannot start

CrashLoopBackOff:

Image successfully pulled

Container created

Container starts

Process fails

Container exits

Kubernetes restarts it

Process fails again

CrashLoopBackOff

This usually means we need to investigate what is happening INSIDE the container.

==================================================

12. KUBECTL LOGS

Check container logs:

kubectl logs POD-NAME -n restaurant-prod

If the container already restarted:

kubectl logs POD-NAME -n restaurant-prod --previous

This is especially useful for:

CrashLoopBackOff

Application exceptions

Configuration problems

Permission errors

Startup failures

A useful troubleshooting idea:

kubectl get pods

What is happening?

kubectl describe pod

What does Kubernetes know about the problem?

kubectl logs

What is the application/container reporting?

==================================================

13. BASIC TROUBLESHOOTING PROCESS

Do not randomly change YAML files when something fails.

Use a systematic process.

STEP 1

Check resources:

kubectl get pods -n restaurant-prod

STEP 2

Identify the status.

STEP 3

Describe the failing resource:

kubectl describe pod POD-NAME -n restaurant-prod

STEP 4

Read Events.

STEP 5

Check application logs:

kubectl logs POD-NAME -n restaurant-prod

STEP 6

For restarting containers:

kubectl logs POD-NAME -n restaurant-prod --previous

STEP 7

Identify the root cause.

STEP 8

Fix the configuration or application.

STEP 9

Deploy again.

STEP 10

Verify the rollout and application health.

==================================================

14. STARTUP PROBE

Startup Probe answers:

HAS THE APPLICATION FINISHED STARTING?

Some applications need time to initialize.

They may need to:

Load configuration

Connect to dependencies

Initialize caches

Start application services

Startup Probe protects slow-starting applications from being treated as unhealthy too early.

==================================================

15. READINESS PROBE

Readiness Probe answers:

IS THIS POD READY TO RECEIVE USER TRAFFIC?

A container may be running while the application is not yet ready.

Example:

Container started

Application initializing

Database connection starting

Application not ready

During this time, Kubernetes should not send user traffic to that Pod.

Once readiness succeeds:

Pod becomes Ready

Service can send traffic to it

==================================================

16. LIVENESS PROBE

Liveness Probe answers:

IS THE APPLICATION STILL HEALTHY?

Imagine:

Container = Running

but

Application = Frozen

The process may exist but no longer work correctly.

Liveness Probe can detect this condition.

After repeated failures, Kubernetes can restart the container.

Remember:

Startup Probe
→ Did the application finish starting?

Readiness Probe
→ Can it receive traffic?

Liveness Probe
→ Is it still healthy?

==================================================

17. SERVICE

Pods are temporary.

A Pod can be deleted and recreated with:

A new Pod name

A new Pod IP

Therefore, applications should not depend on individual Pod IP addresses.

Kubernetes Service provides a stable network endpoint.

Traffic flow:

User

Load Balancer / Ingress

Service

Pod 1
Pod 2
Pod 3

Service discovers Pods using:

LABELS AND SELECTORS

Example Pod label:

app: restaurant-company

Example Service selector:

app: restaurant-company

These values must match.

If they do not match:

Service exists

Pods exist

But Service cannot find the Pods

==================================================

18. CONFIGMAP

ConfigMap stores non-sensitive application configuration.

Examples:

APP_ENV=production

LOG_LEVEL=info

API_URL=...

The purpose is to separate configuration from the application image.

==================================================

19. SECRET

Secrets are used for sensitive configuration.

Examples:

Database password

API token

API key

Credentials

Important:

Do not assume that simply creating a Kubernetes Secret solves all production secret-management requirements.

Production environments may integrate Kubernetes with systems such as:

AWS Secrets Manager

HashiCorp Vault

External Secrets solutions

==================================================

20. SERVICEACCOUNT

A ServiceAccount provides an identity for workloads running inside Kubernetes.

Think about it as:

Pod

ServiceAccount

Workload identity / permissions

Applications should receive only the permissions they actually require.

This follows the principle of:

LEAST PRIVILEGE

==================================================

21. ROLLING UPDATE

Imagine production currently runs:

Pod 1 → Version 1

Pod 2 → Version 1

Pod 3 → Version 1

A new Version 2 is released.

We do not want to delete every Version 1 Pod at the same time.

That could cause downtime.

Rolling Update gradually replaces the old Pods.

Example:

v1
v1
v1

v1
v1
v1
v2

v1
v1
v2

v1
v2
v2

v2
v2
v2

This allows the application to remain available while the release is being deployed.

==================================================

22. MAXSURGE AND MAXUNAVAILABLE

Example:

maxUnavailable: 0

maxSurge: 1

maxSurge: 1

means Kubernetes can temporarily create one additional Pod during the rollout.

If desired replicas = 3:

Normal:

3 Pods

During rollout:

Up to 4 Pods

maxUnavailable: 0

means the rollout is configured to avoid intentionally reducing available replicas below the desired count during the update, subject to the Pods actually becoming Ready.

==================================================

23. ROLLING UPDATE LAB COMMANDS

Restart the Deployment:

kubectl rollout restart deployment/restaurant-company \
-n restaurant-prod

Watch the Pods:

kubectl get pods -n restaurant-prod -w

Observe:

New Pod created

Container starts

Readiness succeeds

New Pod becomes Ready

Old Pod terminates

Process continues

==================================================

24. MANUAL SCALING

Suppose the Deployment currently has:

3 replicas

We can manually scale it:

kubectl scale deployment restaurant-company \
--replicas=5 \
-n restaurant-prod

Then:

kubectl get pods -n restaurant-prod

Kubernetes creates additional Pods to reach the desired state.

==================================================

25. HPA

HPA means:

Horizontal Pod Autoscaler

Instead of manually changing:

3 → 5 → 10 → 5 → 3

HPA can automatically adjust the number of replicas based on supported metrics.

Simplified example:

Application load increases

Metrics increase

HPA calculates more replicas are needed

Deployment replica target increases

Kubernetes creates additional Pods

When demand decreases, HPA can scale the workload down according to its configuration.

==================================================

26. PDB

PDB means:

PodDisruptionBudget

PDB helps protect application availability during voluntary disruptions.

Example:

3 application Pods

PDB:

minAvailable: 2

During supported voluntary disruptions, Kubernetes considers this budget when evicting Pods.

Important:

PDB does NOT guarantee that Pods can never fail.

It does not protect the application from every type of outage.

==================================================

27. NETWORKPOLICY

NetworkPolicy controls allowed network communication between workloads when supported by the cluster networking implementation.

Without appropriate controls:

Many workloads may be able to communicate with each other.

Production goal:

Only allow required communication.

Example:

Frontend

Backend

Database

We may want:

Frontend → Backend

Backend → Database

but NOT:

Random Pod → Database

This supports the principle of least privilege.

==================================================

28. ROLLOUT STATUS

After deployment, do not assume success simply because:

kubectl apply

completed successfully.

Check the rollout:

kubectl rollout status deployment/restaurant-company \
-n restaurant-prod

Then check Pods:

kubectl get pods -n restaurant-prod

Then check Services:

kubectl get services -n restaurant-prod

A production deployment should be verified.

==================================================

29. ROLLBACK

Imagine:

10:00 AM

Version 2 deployed

10:05 AM

Customers report that the application is failing.

First investigate the impact.

Check rollout:

kubectl rollout status deployment/restaurant-company \
-n restaurant-prod

Check history:

kubectl rollout history deployment/restaurant-company \
-n restaurant-prod

If the new release is causing production impact and rollback is the appropriate recovery action:

kubectl rollout undo deployment/restaurant-company \
-n restaurant-prod

Then verify:

kubectl rollout status deployment/restaurant-company \
-n restaurant-prod

The priority during a serious production incident is often:

RESTORE SERVICE FIRST

then

INVESTIGATE ROOT CAUSE

==================================================

30. COMMON KUBERNETES STATUS TROUBLESHOOTING

PENDING

Think about:

Scheduling

Worker Nodes

CPU

Memory

Taints / Tolerations

Node selectors / affinity

Storage requirements


IMAGEPULLBACKOFF

Think about:

Image repository

Image tag

Registry

ECR permissions

Authentication

Architecture


CRASHLOOPBACKOFF

Think about:

Application logs

Configuration

Environment variables

Permissions

Startup command

Dependencies

Use:

kubectl logs POD-NAME -n restaurant-prod

kubectl logs POD-NAME -n restaurant-prod --previous


RUNNING BUT 0/1 READY

Think about:

Readiness Probe

Application health

Port

Health endpoint

Dependencies


SERVICE EXISTS BUT APPLICATION IS NOT REACHABLE

Think about:

Service selector

Pod labels

Service port

targetPort

Endpoints

Ingress / Load Balancer

==================================================

31. COMMANDS YOU SHOULD KNOW

kubectl get nodes

kubectl get namespaces

kubectl get pods -n restaurant-prod

kubectl get pods -n restaurant-prod -o wide

kubectl get deployments -n restaurant-prod

kubectl get replicasets -n restaurant-prod

kubectl get services -n restaurant-prod

kubectl get ingress -n restaurant-prod

kubectl describe pod POD-NAME -n restaurant-prod

kubectl logs POD-NAME -n restaurant-prod

kubectl logs POD-NAME -n restaurant-prod --previous

kubectl get events -n restaurant-prod

kubectl rollout status deployment/restaurant-company -n restaurant-prod

kubectl rollout history deployment/restaurant-company -n restaurant-prod

kubectl rollout undo deployment/restaurant-company -n restaurant-prod

Do NOT only memorize these commands.

Understand:

WHEN to use them

WHY to use them

WHAT information they provide

==================================================

32. COMPLETE PROJECT ARCHITECTURE

Developer

GitHub

CI Pipeline

Lint

Unit Tests

SonarQube

Security Scanning

Trivy

Docker Build

AWS ECR

==============================

CONTINUOUS DEPLOYMENT

Jenkins

AWS Authentication

AWS EKS

Namespace

ConfigMap / Secret

ServiceAccount

Deployment

ReplicaSet

Pods

Containers

Service

Ingress / Load Balancer

Users

==================================================

33. INTERVIEW QUESTION

QUESTION:

Walk me through how you deploy an application to Kubernetes.

SAMPLE ANSWER:

Our CI pipeline validates the application through testing, code-quality checks, and security scanning.

It then builds the Docker image and publishes the approved image to our container registry.

Our CD pipeline authenticates to the target environment and deploys the Kubernetes manifests to EKS.

The Deployment defines the desired replicas, image, resource requirements, health probes, and rollout strategy.

Kubernetes creates and manages the ReplicaSets and Pods, while Services provide stable networking.

We monitor the rollout and verify Pod and application health after deployment.

If the deployment fails, I check the Pod status, Kubernetes events, container logs, configuration, probes, and image information to identify the root cause.

If a new release is causing production impact, we can roll back to the previous stable version and verify service health before continuing the root-cause investigation.

==================================================

34. QUESTIONS YOU SHOULD BE ABLE TO ANSWER

  1. What is CI?

  2. What is CD?

  3. What does Jenkins do in our CD pipeline?

  4. What does AWS ECR do?

  5. Does ECR deploy the application?

  6. What does a Kubernetes Deployment do?

  7. What is desired state?

  8. What is a ReplicaSet?

  9. What is a Pod?

  10. What happens if a Pod dies?

  11. What is Kubernetes self-healing?

  12. What is a Service?

  13. How does a Service find Pods?

  14. What is the difference between ConfigMap and Secret?

  15. What is a ServiceAccount?

  16. What is Startup Probe?

  17. What is Readiness Probe?

  18. What is Liveness Probe?

  19. What is ImagePullBackOff?

  20. What is CrashLoopBackOff?

  21. How do you check Pod logs?

  22. Why would you use kubectl describe?

  23. What is Rolling Update?

  24. What is maxSurge?

  25. What is maxUnavailable?

  26. What is HPA?

  27. What is PDB?

  28. What is NetworkPolicy?

  29. How do you check rollout status?

  30. How do you roll back a bad deployment?

==================================================

35. HOMEWORK

Do not just read this material.

Open your Kubernetes YAML files and identify where each concept exists.

Find:

namespace.yaml

deployment.yaml

service.yaml

configmap.yaml

secrets.yaml

serviceaccount.yaml

hpa.yaml if available

pdb.yaml

networkpolicy.yaml

ingress.yaml

Jenkinsfile

For each file, be able to explain:

  1. What is this resource?

  2. Why do we need it?

  3. What problem does it solve?

  4. What happens if it is configured incorrectly?

  5. How would you verify that it is working?

Finally, practice explaining this complete flow without reading:

Developer

GitHub

CI

Docker Image

ECR

Jenkins CD

EKS

Deployment

ReplicaSet

Pods

Service

Ingress / Load Balancer

User

HOMEWORK GOAL:

Do not memorize Kubernetes.

Understand how the components work together.

A DevOps engineer must be able to:

DEPLOY

VERIFY

TROUBLESHOOT

ROLL BACK

and explain WHY each step is necessary.

Top comments (0)