DEV Community

Cover image for K8s: The Affinity Club
George Michalakis
George Michalakis

Posted on

K8s: The Affinity Club

Introduction

If you have fiddled with Kubernetes long enough to worry about which node your Pods are actually running on, you can probably relate.

First: what does affinity even mean?

Affinity: a strong feeling that you understand or like someone or something; a close relationship between people or things with similar qualities.


Okay. Fair enough.

In Kubernetes, affinity is a family of scheduling rules:

  • Node affinity defines hard requirements and soft preferences for which nodes can run a Pod.
  • Pod affinity lets us place a Pod near other Pods.
  • Pod anti-affinity lets us keep Pods away from other Pods.

So, loosely, if I were a Pod, I would say:

“I have an affinity for being near or away from these Pods.”

But why care?

At the end of the day, I can always scale to more replicas. I’m safe… right?

Asking are you sure about that?

Let’s see it in action

Let’s take the worst-case scenario.

Imagine a cluster with two worker nodes. We have a critical service, but all of its Pods happen to be scheduled on worker-1.

Traffic grows, so we scale the Deployment. But every new replica still lands on worker-1.

Lab 01 placement visual

After scaling, two things become obvious:

  • worker-1 is becoming stressed.
  • If worker-1 fails, every replica fails with it—and the application is down.

The scheduler is not random, but without a placement rule, it has no obligation* to spread our replicas across workers.

In this scenario, worker-2 is right there... empty..

Ok. Can we force Kubernetes to keep replicas of the same service on different workers?


Yes. That is exactly what pod anti-affinity is for.

Below, we apply pod anti-affinity to our critical service’s Deployment.

Before: no placement rule

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
  namespace: drainlab
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
        - name: nginx
          image: nginx:1.27-alpine
          resources:
            requests:
              cpu: 250m
              memory: 128Mi
          ports:
            - containerPort: 80
Enter fullscreen mode Exit fullscreen mode

After: hard pod anti-affinity

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
  namespace: drainlab
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      affinity: # <-Magic starts to happen here
        podAntiAffinity: 
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchLabels:
                  app: web
              topologyKey: kubernetes.io/hostname # <-finishes here
      containers:
        - name: nginx
          image: nginx:1.27-alpine
          resources:
            requests:
              cpu: 250m
              memory: 128Mi
          ports:
            - containerPort: 80
Enter fullscreen mode Exit fullscreen mode

This tells the scheduler:

requiredDuringScheduling “A new app: web Pod cannot be scheduled onto a node that already runs another matching app: web Pod.”

If every eligible worker already runs one, the new Pod stays Pending rather than placing two replicas on the same node.

More specifically:

  • requiredDuringScheduling: if no valid worker exists, the new Pod stays Pending.
  • IgnoredDuringExecution: Kubernetes does not evict an already-running Pod merely because later changes violate the anti-affinity condition.
  • topologyKey: kubernetes.io/hostname: each worker node is treated as a separate failure domain.

01lab-02


But wait… if I have two eligible worker nodes and need four replicas, do the other two remain Pending?

Shocked Cat

Before anti-affinity, all four Pods could run on worker-1, even though one node failure would take down the entire service.

Now, Kubernetes protects us from that false sense of safety but hard pod anti-affinity limits us to one matching Pod per worker.

So we gained failure isolation, but we gave up that easy scaling?

No, This is where topology spread constraints come in.


*Kubernetes may already prefer to spread Pods using soft scheduling preferences. Exact placement depends on available resources and scheduler configuration. This example demonstrates why relying on a preference is different from declaring a hard availability rule.

Labs / Visualizations for this article can be found here

Top comments (0)