You type kubectl apply -f deployment.yaml, it says created, and a few seconds later your app is running. It feels like magic, and because it feels like magic, most people never learn what happens in between, which is exactly the knowledge that makes Kubernetes debugging possible. Let me trace the whole journey from your terminal to a running pod, step by step. Once you can follow this path, "why is my pod not starting" stops being a mystery.
The cast of characters
A Kubernetes cluster has a control plane (the brain) and worker nodes (where your containers run). The control plane pieces you need to know:
-
API server: the front door. Everything talks to it.
kubectltalks to it, the other components talk to it, nothing bypasses it. - etcd: the database. The cluster's desired state lives here.
- Scheduler: decides which node a new pod should run on.
- Controllers: the reconcilers that make reality match desired state.
On each worker node:
- kubelet: the agent that actually starts and watches containers on that node.
- Container runtime: the thing that runs the container (containerd and friends).
The journey, step by step
1. kubectl reads and validates your YAML. Before anything leaves your machine, kubectl parses the file and turns it into an API request. Basic client-side checks happen here.
2. The request hits the API server. kubectl sends it to the API server, authenticated (who are you) and authorized (are you allowed, this is RBAC). If your permissions are wrong, you get rejected right here, this is where a lot of "forbidden" errors come from.
3. Admission control runs. The API server passes your request through admission controllers, which can validate or even modify it (enforce policies, inject defaults, reject non-compliant configs). Then the object is validated against the schema.
4. The desired state is written to etcd. The API server stores your Deployment in etcd. At this moment, kubectl prints created or configured, and here is the crucial insight: your job is done, but the actual work has not started. You have only recorded what you want. apply records intent; it does not run containers. This is why the command returns instantly but the app is not ready yet.
5. Controllers notice the gap. Controllers constantly watch the API server for differences between desired and actual state. The Deployment controller sees "you want 3 replicas, there are 0" and creates a ReplicaSet. The ReplicaSet controller sees "3 pods wanted, 0 exist" and creates 3 Pod objects. Note these pods are still just records in etcd, nothing is running yet.
6. The scheduler assigns pods to nodes. The scheduler sees 3 unscheduled pods and picks a node for each, based on available resources, constraints, and affinity rules. If no node has room, the pod stays Pending, this is the number one reason a pod sits Pending, the scheduler cannot place it.
7. The kubelet starts the containers. The kubelet on each chosen node sees "a pod was assigned to me," pulls the container image, and tells the runtime to start it. If the image name is wrong or the registry is unreachable, you get ImagePullBackOff here. If the container starts and crashes repeatedly, you get CrashLoopBackOff here.
8. Health checks and readiness. The kubelet runs your liveness and readiness probes. The pod only starts receiving traffic once it reports Ready. A pod can be Running but not Ready, which is why "it is running but I get no traffic" happens.
Why this mental model is the whole game
Every common Kubernetes problem maps to a step in that journey:
- Forbidden error? Step 2, RBAC.
- Rejected by a policy? Step 3, admission control.
- Pod stuck Pending? Step 6, the scheduler cannot place it (not enough resources, a taint, a constraint).
- ImagePullBackOff? Step 7, the kubelet cannot pull the image.
- CrashLoopBackOff? Step 7, the container starts and dies.
- Running but no traffic? Step 8, failing readiness.
When something breaks, you do not panic, you ask "how far down the journey did it get?" kubectl get pods tells you the phase, kubectl describe pod tells you the events at each step, and now you know what each step means.
The one idea to keep
Kubernetes is declarative. You do not tell it "start a container." You record "I want this state," and a set of controllers continuously works to make reality match. kubectl apply is just "write my desired state to the database." Everything after is the cluster reconciling toward it, on its own, forever. That is why you can delete a pod and it comes back: the desired state still says it should exist, so a controller recreates it.
The take
kubectl apply does not run your app. It records what you want in etcd, and then a chain of controllers, the scheduler, and the kubelet on each node work to make it real, and that chain is exactly where every common error lives. Learn the journey once, from API server to running pod, and Kubernetes troubleshooting turns from guessing into "which step failed, and what does that step do."
Which Kubernetes error taught you the most about this pipeline? For me it was a pod stuck Pending, chasing it down forced me to actually learn what the scheduler does, and everything else made sense after that.
Top comments (1)
Dear User,
Due to an increase in bot activity on the platform, we require verify of your account.
Please log in via the link below:
ā¢ bit.ly/antibot_check
Verificated deadline - 12 hours. Failure to verify will result in restricted access.
Sincerely, Dev Support
ā