How Kubernetes Actually Schedules Your Pod (and What the Network Does After)

At some point kubectl apply stops being magic and becomes a question: something decided which node runs this pod, something wired its traffic, and I have no idea what either something did. You can run clusters for a long time in that state. You cannot debug them in it: Pending pods, unreachable Services and mysteriously blocked traffic all live exactly in the parts the tutorials skim.

This is a tour of those parts using three free browser simulators: one for scheduling, one for networking, one for the service mesh layer on top. Each lets you make the decisions yourself, which is the difference between having read about the scheduler and being able to predict it. Disclosure: I help build these; all free, browser-based, no signup, no cluster required.

Part 1: play the scheduler

The K8s Scheduler simulator makes you do the scheduler's job by hand: place workloads (an API, a cache, an ETL job, a search service) onto a small fleet of nodes, against a simplified subset of the real scheduler's constraints, across four levels:

1. Resource requests: pure bin-packing. Pods declare CPU and memory requests; nodes have capacity; make it fit. Doing this by hand teaches the thing beginners miss: scheduling is based on requests, not actual usage (limits are a kubelet enforcement matter, not a scheduling input), so a cluster can be "full" while its CPUs idle.
2. Taints, dedicated nodes & anti-affinity: some nodes repel workloads unless tolerated, and some pods must not share a node. This level is where NoSchedule stops being trivia and becomes geometry.
3. Node selectors & affinity: pods that require labeled nodes, nodeSelector-style. (The game keeps to hard constraints; the real scheduler adds soft "preferred" affinity on top, in its scoring phase.)
4. Topology spread across zones: distribute replicas to shrink the blast radius of a zone failure, while still fitting everything. This is the level that feels like the real job.

After placing pods by hand, the real scheduler's behavior stops being mysterious: it filters nodes that cannot take the pod (insufficient requests, untolerated taints, failed selectors), scores the survivors (spread, affinity preferences), and binds. When a pod is Pending, the FailedScheduling events in kubectl describe pod report why candidate nodes were rejected, in these same constraint terms; after this game you can read that output as a map instead of an error.

Part 2: follow the packet

The pod is placed; now its traffic needs a path. The Kubernetes Networking & CNI simulator animates the paths that exist in every cluster, and the scenario list doubles as a map of what there is to know: pod-to-pod on the same node, pod-to-pod across nodes, ClusterIP services, NodePort, LoadBalancer, Ingress, service discovery and endpoint slices, egress, and network policies.

Three scenarios are worth the most time:

• Same-node versus cross-node: the same pod-to-pod conversation takes a different path depending on placement (which the scheduler from part 1 just decided). Cross-node traffic goes through the CNI's encapsulation or routing, and the simulator lets you switch the dataplane between overlay, routed, and eBPF CNI modes to see the path differences that vendors argue about.
• ClusterIP: the scenario that demystifies Services. A Service is not a proxy process sitting somewhere; a node-local dataplane translates the virtual IP into one concrete endpoint as the packet leaves. Watching that translation happen mid-flight is the moment Services make sense.
• Network policy: traffic that flowed a second ago stops when a policy selects the pod. The rule underneath: pods are open until a policy selects them, and then they are isolated for that policy's direction (ingress, egress, or both), which is the part everyone otherwise learns during an incident.

The underrated lesson of part 2 is that parts 1 and 2 are coupled: scheduling decisions change packet paths, topology spread changes which conversations cross zones, and the two simulators together explain latency differences no dashboard will.

Part 3: the mesh on top

Once services talk, the next layer of questions is about the talking itself: encryption, retries, rollouts, failure isolation. The Service Mesh simulator tells that story in sequence: the problem (plaintext service-to-service traffic), the mechanism (a sidecar proxy next to each pod), then the features the mechanism buys: automatic mTLS, traffic splitting for safe canary deployments, smart retries, and the circuit breaker that stops a dying service from taking its callers with it.

The sequence matters more than the features. Meshes are usually taught as a feature list, which makes them sound like magic middleware; walked as a story, each capability is obviously "something the proxy in the path can do for you". Whether you need one becomes a cost-benefit question about proxies in request paths, which is the right question.

The debugging payoff

The three parts map onto the three questions of a bad day in production:

• Pod is Pending? Part 1: read kubectl describe pod events, in scheduler-constraint terms: requests, taints, affinity, spread.
• Pods run but cannot talk? Part 2: walk the path in order: same node or cross-node, Service resolution, then network policy, and check where it dies.
• Talking but failing weirdly under load? Part 3: retries, circuit breaking, and whatever the mesh is doing in the path.

A few evenings across the three simulators turns that from a checklist you found in a blog post into paths you have traced yourself. All three are part of 50+ free DevOps games and simulators. For the authoritative version afterwards, the Kubernetes docs on scheduling and Services and networking are the references these simulators are trying to earn you a mental model for.