DEV Community

Cover image for The Importance of Kubernetes Probes: Liveness & Readiness
Kelvyn Thai
Kelvyn Thai

Posted on

The Importance of Kubernetes Probes: Liveness & Readiness

#k8s #redis #docker

Deploying an application to a Kubernetes cluster is only the first step. The real challenge is ensuring your application remains healthy, available, and self-healing when unexpected failures occur.

In production, your application may encounter issues such as:

  • Deadlocks causing the application to stop responding.
  • Out of Memory (OOM) errors.
  • CPU or memory resource exhaustion.
  • External dependency failures (Redis, MySQL, Kafka, etc.).
  • Slow startup during deployment.

These problems don't always cause the application process to crash. Sometimes the container is still Running, but it can no longer serve user requests.

For example:

  • Your Node.js application enters a deadlock and every request hangs indefinitely.
  • Your application depends on Redis for caching configuration, user sessions,... If Redis becomes unavailable, your application may no longer be able to process requests correctly.

Without proper health checks, Kubernetes assumes the Pod is healthy simply because the container process is still running.

This is where Kubernetes Probes become essential.

  • Liveness Probe detects whether an application is still alive. If the application becomes unresponsive, Kubernetes automatically restarts the container.
  • Readiness Probe determines whether an application is ready to receive traffic. If a dependency such as Redis becomes unavailable, Kubernetes removes the Pod from the Service endpoints until it becomes healthy again.

In this article, we'll walk through two practical examples using a Next.js application:

  1. Liveness Probe — Automatically recover from an application deadlock.
  2. Readiness Probe — Prevent traffic from reaching an application when Redis is unavailable.

By the end of this article, you'll understand not only how to configure these probes, but more importantly why they are critical for building resilient Kubernetes applications.

1. Deadlock (Without vs With Liveness Probe)

Prerequisites

  • Create a /deadlock endpoint that blocks the event loop for 30 seconds.
  • Prepare two deployment manifests:
    • Without livenessProbe
    • With livenessProbe
  • Expose the application using a NodePort Service.

Scenario 1: Without Liveness Probe

Step 1. Verify the application is healthy 

kubectl get svc nextjs-app

curl http://<NodeIP>:<NodePort>/healthz

curl -o /dev/null -s -w "%{http_code}\n" http://<NodeIP>:<NodePort>/healthz
Enter fullscreen mode Exit fullscreen mode

Expected

  • /healthz returns HTTP 200
  • Pod status is Running (1/1)

📷 Screenshot: Healthy application

Step 2. Simulate a deadlock

src/app/deadlock/route.ts · develop-beta · Kelvyn Labs / Next.js CICD Template · GitLab

GitLab.com

favicon gitlab.com

Open the following endpoint:

http://<NodeIP>:<NodePort>/deadlock
Enter fullscreen mode Exit fullscreen mode

This endpoint blocks the Node.js event loop for 30 seconds.

Step 3. Observe the application

Watch the Pod status:

kubectl get pods -w -l app=nextjs-app
Enter fullscreen mode Exit fullscreen mode

Monitor the health endpoint:

while true; do
  curl -sS --max-time 2 -w "\n" http://<NodeIP>:<NodePort>/healthz \
  || echo "Health check timeout"
  sleep 1
done
Enter fullscreen mode Exit fullscreen mode

Result

  • Pod remains Running (1/1)
  • Restart count stays 0
  • /healthz starts timing out
  • Users cannot access the application for 30 seconds

📷 Screenshot: Pod still Running while health checks timeout

Conclusion

Without a Liveness Probe, Kubernetes cannot detect that the application is hung because the container process is still alive.

Scenario 2: Enable Liveness Probe

Add the following configuration:

livenessProbe:
  httpGet:
    path: /healthz
    port: 3000

  initialDelaySeconds: 5 # First check after 5s
  periodSeconds: 5       # Check every 5s
  timeoutSeconds: 1      # Fail after 1s
  failureThreshold: 3    # Restart after 3 failures
Enter fullscreen mode Exit fullscreen mode

Redeploy the application.

Step 1. Trigger the same deadlock

Open:

http://<NodeIP>:<NodePort>/deadlock
Enter fullscreen mode Exit fullscreen mode

Step 2. Observe Kubernetes

Check Pod events:

kubectl describe pod <pod-name>
Enter fullscreen mode Exit fullscreen mode

Expected events:

Liveness probe failed
Container will be restarted
Enter fullscreen mode Exit fullscreen mode

📷 Screenshot: Liveness probe failed

Check the Pod status:

kubectl get pods -w -l app=nextjs-app
Enter fullscreen mode Exit fullscreen mode

Expected:

READY   STATUS    RESTARTS

1/1     Running   1
Enter fullscreen mode Exit fullscreen mode

📷 Screenshot: Restart count increased

Result

Without Liveness With Liveness
Pod stays Running Pod restarted automatically
Restart count = 0 Restart count increases
Health endpoint times out Health endpoint recovers
Users wait until app recovers Kubernetes recovers automatically

Key Takeaway

A Liveness Probe enables Kubernetes to detect when an application is alive but no longer responsive. Instead of leaving users waiting for the application to recover on its own, Kubernetes automatically restarts the container and restores the service.

2. Redis Goes Down (Without vs With Readiness Probe)

Prerequisites

  • Create a /cache?key=... endpoint that reads data from Redis.
  • Configure the application to return HTTP 503 when Redis is unavailable.
  • Prepare two deployment manifests:
    • Without readinessProbe
    • With readinessProbe
  • Expose the application using a NodePort Service.

Scenario 1: Without Readiness Probe

Step 1. Verify the application is healthy 

kubectl get pods -l app=nextjs-app

curl http://<NodeIP>:<NodePort>/healthz

curl -w "\n" -X POST http://<<NodeIP>:<NodePort>/api/cache -H "Content-Type: application/json" -d '{"key":"test","value":"Hello Kubernetes!"}'

curl http://<NodeIP>:<NodePort>/api/cache?key=test

curl -o /dev/null -s -w "%{http_code}\n" http://<NodeIP>:<NodePort>/api/cache?key=test/
Enter fullscreen mode Exit fullscreen mode

Expected

  • Pod status is Running (1/1)
  • /healthz returns HTTP 200
  • /api/cache returns HTTP 200

📷 Screenshot: Application healthy

Step 2. Stop Redis 

kubectl scale deployment redis --replicas=0
Enter fullscreen mode Exit fullscreen mode

or

docker stop redis
Enter fullscreen mode Exit fullscreen mode

📷 Screenshot: Redis stopped

Step 3. Observe the application

Check Pod status:

kubectl get pods -w -l app=nextjs-app
Enter fullscreen mode Exit fullscreen mode

Test the API:

curl http://<NodeIP>:<NodePort>/api/cache?key=test
Enter fullscreen mode Exit fullscreen mode

or

curl -o /dev/null -s -w "%{http_code}\n" \
http://<NodeIP>:<NodePort>/api/cache?key=test
Enter fullscreen mode Exit fullscreen mode

Result

  • Pod still shows Running (1/1)
  • Restart count = 0
  • /api/cache returns 503 Service Unavailable
  • Kubernetes still routes traffic to this Pod

📷 Screenshot: Pod Ready but API returns 503

Conclusion

Without a Readiness Probe, Kubernetes only knows that the container process is running.

It does not know that Redis is unavailable, so the Pod continues receiving user traffic even though it cannot successfully serve requests.

Scenario 2: Enable Readiness Probe

Add the following configuration:

readinessProbe:
  httpGet:
    path: /healthz
    port: 3000

  initialDelaySeconds: 5 # First check after 5s
  periodSeconds: 5       # Check every 5s
  timeoutSeconds: 1      # Fail after 1s
  failureThreshold: 3    # Mark Pod NotReady after 3 failures
Enter fullscreen mode Exit fullscreen mode

Redeploy the application.

Step 1. Stop Redis again 

kubectl scale deployment redis --replicas=0
Enter fullscreen mode Exit fullscreen mode

Step 2. Observe Kubernetes

Watch Pod status:

kubectl get pods -w -l app=nextjs-app
Enter fullscreen mode Exit fullscreen mode

Expected:

READY

0/1
Enter fullscreen mode Exit fullscreen mode

📷 Screenshot: Pod becomes NotReady

Check Service endpoints:

kubectl get endpoints nextjs-app
Enter fullscreen mode Exit fullscreen mode

Expected:

<none>
Enter fullscreen mode Exit fullscreen mode

or the affected Pod is removed from the endpoint list.

📷 Screenshot: Endpoint removed

Access the application again:

curl http://<NodeIP>:<NodePort>/api/cache?key=test
Enter fullscreen mode Exit fullscreen mode

Expected:

curl: (7) Failed to connect
Enter fullscreen mode Exit fullscreen mode

depending on whether another healthy Pod exists.

Result

Without Readiness With Readiness
Pod Ready (1/1) Pod NotReady (0/1)
Traffic still routed Traffic stopped
Users receive 503 Unhealthy Pod removed from Service
Kubernetes thinks Pod is healthy Kubernetes waits until dependencies recover

Key Takeaway

A Readiness Probe does not restart your application.

Instead, it tells Kubernetes whether the Pod is ready to receive traffic.

When an external dependency such as Redis becomes unavailable, Kubernetes automatically removes the Pod from the Service endpoints, preventing users from sending requests to an unhealthy application.

Top comments (0)