Production incident. Pod is failing. Alerts are firing.
The instinctive response is often:
"Restart the pod and see if it comes back."
Sometimes that restores service — but it can also destroy useful evidence.
Before restarting, check these five things.
1. Describe the pod
kubectl describe pod <pod-name> -n <namespace>
Look at:
- Container state
- Restart count
- Last termination reason
- Probe failures
- Events
This often tells you whether the problem is in the application, resources, scheduling, storage, or networking.
2. Check the previous container logs
If the container has restarted:
kubectl logs <pod-name> -n <namespace> --previous
This is especially useful for CrashLoopBackOff.
The current container may already be running again while the actual failure evidence exists only in the previous logs.
3. Check why the container terminated
For example:
OOMKilled
Error
Completed
OOMKilled does not automatically mean "increase the memory limit."
First understand whether the workload has a memory leak, unrealistic limits, an unexpected traffic spike, or node pressure.
4. Check probes
A running container can still be unavailable because of:
Readiness probe failed
Liveness probe failed
Startup probe failed
A bad probe configuration can create an incident even when the application itself is healthy.
5. Follow the evidence before restarting
A simple troubleshooting habit:
Observe → Capture → Isolate → Recover
Capture the evidence first.
Then apply the safest fix.
A restart may restore service, but understanding the failure is what stops the same incident from returning.
I turned this workflow into the OPSFORGED Kubernetes Incident Troubleshooting Playbook V1.3 — a 37-page visual field guide covering CrashLoopBackOff, OOMKilled, probes, DNS, networking, rollout issues and more.
₹499 (about US$6).
If you work with Kubernetes, I'd be interested to know which production issue wastes the most troubleshooting time for you.
Top comments (0)