DEV Community

Cover image for Kubernetes Scheduling 101: What Really Happens Before Your Pod Runs
Siddharth More
Siddharth More

Posted on

Kubernetes Scheduling 101: What Really Happens Before Your Pod Runs

TL;DR

  • Scheduling means binding a pod to a node by setting spec.nodeName. You can set this yourself (manual scheduling), or let kube-scheduler do it.
  • kube-scheduler decides in two steps: filtering out nodes that can't run the pod, then scoring the ones that can.
  • A pod stays on its node for life once scheduled, with one exception: a NoExecute taint can evict it later. More on that in Part 3.
  • Manual scheduling via nodeName skips the filter-and-score process entirely, so it also skips the safety checks that come with it.

You run kubectl apply, and the pod just shows up somewhere. But how does Kubernetes actually pick that node?

This is Part 1 of a series on Kubernetes scheduling. Before we get into node selectors, taints, and affinity rules, we need to understand what scheduling actually is. And the easiest way to understand it is to see what happens when you skip it.

What scheduling actually means

Scheduling comes down to one thing: binding a pod to a node. That means writing the node's name into the pod's spec.nodeName field. Once that field is set, the kubelet on that node picks up the pod and runs it.

Here's the part most people miss. This binding happens once, at creation time. Kubernetes doesn't sit there watching your pods and moving them around as conditions change.

Once a pod lands on a node, it stays there for its lifetime. Even if a better node shows up five minutes later.

* There's one exception to that "stays there forever" claim. Taints with the NoExecute effect can actively kick a running pod off a node if it doesn't tolerate that taint. We'll get into NoExecute in Part 3. For now, just keep that asterisk in the back of your mind.

Manual scheduling: setting nodeName yourself

Since binding a pod to a node is just setting a field, you can set it yourself.

apiVersion: v1
kind: Pod
metadata:
  name: manually-placed-pod
spec:
  nodeName: k8s-worker-1
  containers:
  - name: nginx
    image: nginx
Enter fullscreen mode Exit fullscreen mode

Apply that, and the pod goes straight to k8s-worker-1. No decision-making involved. This is literally what the docs call manual scheduling.

But here's the catch. When you set nodeName yourself, you skip the whole decision-making process. And that costs you a few safety checks:

  • No resource check. If k8s-worker-1 doesn't have enough CPU or memory, the pod doesn't get rejected. It just fails to run properly, or gets stuck.
  • No existence check. Typo the node name, and there's no scheduler around to catch it. The pod sits in Pending, and the reason isn't obvious at a glance.
  • No taint awareness. If k8s-worker-1 has a taint the pod doesn't tolerate, manual placement ignores it completely. It's a handy tool for demos and edge cases. But it's not something you build on.

📖 Docs: Assigning pods to nodes using nodeName

Enter kube-scheduler: the two-phase process

For every other pod, nodeName starts out empty. Filling it in is kube-scheduler's job. It does that in two steps.

Filtering. kube-scheduler looks at every node and throws out the ones that can't run the pod at all. Not enough CPU or memory? Out. A taint the pod doesn't tolerate? Out. Port conflict, volume mismatch, whatever hard requirement it fails? Out.

Whatever's left after that is the feasible set. Every node in it can technically run the pod.

Scoring. Now kube-scheduler ranks those feasible nodes against each other. It might favor nodes with more free resources, to spread load around. It might favor a node that already has the image cached, so the pod starts faster. There are other scoring rules too, depending on how your cluster's set up.

Whichever node scores highest wins. That's when nodeName actually gets set - by the scheduler, not by you.

This filter-then-score process is exactly what manual scheduling skips. Node selectors, taints, affinity, all of it works by shaping what goes into this process. None of it replaces it.

📖 Docs: kube-scheduler's Filter-then-Score process

Demo: manual vs scheduler-driven placement

Let's see this side by side. Deploy two pods that are basically identical, except one has nodeName set and one doesn't.

# manual-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: manual-pod
spec:
  nodeName: k8s-worker-1
  containers:
  - name: nginx
    image: nginx
---
# scheduled-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: scheduled-pod
spec:
  containers:
  - name: nginx
    image: nginx
Enter fullscreen mode Exit fullscreen mode

Apply both. Then check where they landed:

kubectl get pods -o wide
Enter fullscreen mode Exit fullscreen mode

Both should be Running, maybe on different nodes. Now look at the events.

kubectl describe pod scheduled-pod
Enter fullscreen mode Exit fullscreen mode

You'll see a Scheduled event, something like Successfully assigned default/scheduled-pod to <node>. That's a record of the scheduler actually making a decision.

kubectl describe pod manual-pod
Enter fullscreen mode Exit fullscreen mode

No Scheduled event at all. As far as the scheduler's concerned, it was never involved. The kubelet on k8s-worker-1 just picked up a pod that already had its name written on it.

That missing event is the clearest proof you'll get that these are two different code paths, not two flavors of the same thing.

If you don't see it there, events expire after a while and describe won't always show old ones. Run kubectl get events instead and look for the Scheduled reason.

Manual scheduling, and what it skips

Calling nodeName "manual scheduling" is fine, and it's the term you'll see everywhere, including in CKA material. The label isn't the important part.

What matters is what it skips. nodeName doesn't run through kube-scheduler's filter-then-score engine at all, it just hands the kubelet a node and says go.

That's the distinction that matters for the rest of this series. Node selectors, node affinity, taints and tolerations, pod affinity, none of them bypass the scheduler. They all work with it, either by narrowing the feasible set or nudging the score. Keep that in mind and the rest of this series will click faster.

Next up, Part 2: node selectors and node affinity, the simplest way to actually influence the scheduler instead of skipping it.

Top comments (0)