DEV Community: The Cyber Sidekick

Add a streaming sidecar so kubectl logs can see a legacy app (CKA)

The Cyber Sidekick — Fri, 07 Aug 2026 11:50:09 +0000

Add a streaming sidecar so kubectl logs can see a legacy app (CKA)

Here is a pod that is perfectly healthy, doing real work, and completely silent as far as Kubernetes is concerned. kubectl logs returns nothing at all. Today's CKA task is to fix that without rewriting the application: add a second container alongside it that streams the app's log file to standard output, and share a volume between the two so it can actually read that file. This is the streaming sidecar pattern, and it is one of the most quietly useful things on the exam. Let's run it on a live cluster.

🎥 Watch the video: https://www.youtube.com/watch?v=JDQgrcbmFH0

This is a CKA Workloads & Scheduling walkthrough. Every command below is real output from a live cluster, and you can reproduce the whole thing yourself (scripts at the end).

The scenario

Here is the setup. A legacy application has been packaged into a Deployment called report-engine, in the legacy namespace. It writes everything it does to a log file inside its container and prints nothing to standard output, so it is invisible to the cluster's logging architecture. Three things are asked of you. Add a second, co-located container to the same pod, named log-stream, using the busybox image, whose command streams that log file to standard output. Give both containers a shared volume so the file is visible to the new container. And change nothing else about the container that is already there.

Deployment 'report-engine' (ns 'legacy') logs to a FILE, not stdout
1. Add a co-located container 'log-stream' (busybox) that streams that file
2. Share a volume between both containers so the file is visible
3. Change nothing else about the existing 'app' container

How pod logging actually works

One fact explains this whole question. Kubernetes has a single logging path: the kubelet captures what a container writes to standard output and standard error, and kubectl logs replays that. It does not know about files, it does not look inside your container's filesystem, and there is no configuration to point it at one. So an app that appends to a log file is not badly configured, it is simply outside the pipeline. The documented fix is a streaming sidecar: put a second container in the same pod whose only job is to tail that file and echo it to its own standard output, where the pipeline picks it up. That works because containers in a pod can share volumes. Mount one volume, an emptyDir is plenty, into both containers at the directory holding the log file, and both processes see the same file.

The empty kubectl logs

Let's see the problem before fixing it. The Deployment is running, one pod, READY one of one, no restarts. Nothing is broken. Now ask for its logs, and look carefully: there is no output. Not an error, not a warning, just nothing, which is the most confusing failure mode there is. The app is definitely producing output though. Exec into the container and tail the file it writes, and there are the log lines, timestamped, several of them, sitting in the container's own filesystem where kubectl cannot reach them.

$ kubectl get deployments,pods -n legacy
NAME                            READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/report-engine   1/1     1            1           8s

NAME                                 READY   STATUS    RESTARTS   AGE
pod/report-engine-7d75b577d9-cs5nz   1/1     Running   0          8s

$ kubectl -n legacy logs deploy/report-engine
(no output at all)

$ kubectl -n legacy exec deploy/report-engine -- tail -n 3 /var/log/report-engine.log
2026-07-31 10:58:03 report-engine: batch complete
2026-07-31 10:58:08 report-engine: batch complete

Add the sidecar

Now the edit. In the exam you would run kubectl edit on the Deployment, or dump it to a file, change it, and re-apply; both are fine. Three things go in. A volumeMount on the existing container, pointing at the directory that holds the log file. A second container, named log-stream, image busybox, whose command tails that same file, plus the identical volumeMount. And the volume itself, an emptyDir, which sits at the pod spec level next to containers, not inside them. Watch the indentation there, because that is where this question is usually lost. Notice what did not change: the app container keeps its image and its command exactly as they were.

$ vi report-engine.yaml
...
        app: report-engine
    spec:
      containers:
        - name: app
          image: busybox:1.36
          command:
            - sh
            - -c
            - >-
              mkdir -p /var/log;
              while true; do
              echo "$(date '+%F %T') report-engine: batch complete" >> /var/log/report-engine.log;
              sleep 5; done

$ (after editing)
...
              sleep 5; done
          volumeMounts:
            - name: logs
              mountPath: /var/log
        - name: log-stream
          image: busybox:1.36
          command: ["sh", "-c", "tail -n+1 -F /var/log/report-engine.log"]
          volumeMounts:
            - name: logs
              mountPath: /var/log
      volumes:
        - name: logs
          emptyDir: {}

Rollout to 2/2

Apply it. Changing the pod template starts a rolling update, so the Deployment builds a replacement pod with both containers and retires the old one. Wait for rollout status to report success, and then look at the pod list. The READY column now says two of two: two containers in a single pod, both running, sharing a network namespace and, more importantly for us, sharing a volume. One pod, not two, and that distinction is the whole point of a sidecar.

$ kubectl apply -f report-engine.yaml
deployment.apps/report-engine configured

$ kubectl -n legacy rollout status deployment/report-engine
Waiting for deployment "report-engine" rollout to finish: 0 out of 1 new replicas have been updated...
Waiting for deployment "report-engine" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "report-engine" rollout to finish: 1 old replicas are pending termination...
deployment "report-engine" successfully rolled out

$ kubectl get pods -n legacy
NAME                             READY   STATUS    RESTARTS   AGE
report-engine-764cb77fd9-bgmqs   2/2     Running   0          33s

The payoff

And here is the payoff. Ask for the logs again, this time naming the container with -c log-stream, and the app's log file comes streaming out of kubectl. Same lines, same timestamps we saw hiding in that file a minute ago, except now they are flowing through the cluster's normal logging path, which means anything that collects logs from this cluster picks them up too. The application was never touched. Note the -c: once a pod has more than one container, kubectl needs to know which one you mean.

$ kubectl -n legacy logs deploy/report-engine -c log-stream --tail=4
2026-07-31 10:58:31 report-engine: batch complete
2026-07-31 10:58:36 report-engine: batch complete
2026-07-31 10:58:41 report-engine: batch complete
2026-07-31 10:58:46 report-engine: batch complete

One volume, two containers

It is worth proving that this really is one shared volume and not a coincidence. Exec into the sidecar, the second container, and list the log file: it is right there, owned by root, growing, even though a completely different container is the one writing to it. Then read the mounts back from the Deployment. Both containers mount the same volume, named logs, at the same path. That is the mechanism in two lines of output: same volume, same mount path, two processes, one file.

$ kubectl -n legacy exec deploy/report-engine -c log-stream -- ls -l /var/log/report-engine.log
-rw-r--r--    1 root     root           400 Jul 31 10:58 /var/log/report-engine.log

$ kubectl -n legacy get deployment report-engine -o jsonpath='{range .spec.template.spec.containers[*]}{.name}{" mounts "}{.volumeMounts[0].name}{" at "}{.volumeMounts[0].mountPath}{"\n"}{end}'
app mounts logs at /var/log
log-stream mounts logs at /var/log

Nothing else moved

Last check, and it is the one part three grades. Read the original container back: same name, same image, same command it started with. The only thing that changed about it is the volume mount it needed to share the file, which the task explicitly allows. And the volume itself is a plain emptyDir, created with the pod and thrown away with it, which is exactly right for a log relay. No PersistentVolumeClaim, no storage class, nothing to provision.

$ kubectl -n legacy get deployment report-engine -o jsonpath='{.spec.template.spec.containers[0].name}{" / "}{.spec.template.spec.containers[0].image}{"\n"}{.spec.template.spec.containers[0].command[2]}'
app / busybox:1.36
mkdir -p /var/log; while true; do echo "$(date '+%F %T') report-engine: batch complete" >> /var/log/report-engine.log; sleep 5; done

$ kubectl -n legacy get deployment report-engine -o jsonpath='{.spec.template.spec.volumes}'
[{"emptyDir":{},"name":"logs"}]

Exam tips

Things to carry into the exam. The volume mount goes in every container that needs the file, and there are two of them; forgetting the app container is the classic mistake, because then the sidecar tails a file that will never appear. The volume itself lives at the pod spec level, beside containers, not inside one. An empty emptyDir, written as `emptyDir: {}, is all you need; search the docs page foremptyDirand copy the shape. Mount at the directory, not the file. Once the pod has two containers,kubectl logsand kubectl exec both need-cto say which one. And one modern footnote: Kubernetes does have native sidecars now, an init container with arestartPolicy: Always`, but a question that says co-located container in the same pod means a plain second entry under containers, so do the simple thing.

volumeMounts go in BOTH containers; the app one is the one people forget
The volume itself sits at pod-spec level, beside 'containers', not inside it
'emptyDir: {}' is enough; it is born and dies with the pod
Mount the DIRECTORY that holds the log file, not the file
Two containers means kubectl logs / exec need -c
Native sidecars (initContainer + restartPolicy: Always) exist, but this asks for a plain container

Recap

kubectl logs reads stdout only, so a file-logging app is invisible
Added 'log-stream' (busybox) tailing the file to stdout, in the SAME pod
emptyDir 'logs' mounted at /var/log in both containers: one file, two processes
READY 2/2, logs streaming, original container untouched; subscribe + dev.to writeup

Reproduce this yourself

The entire scenario is scripted on a throwaway kind cluster: https://github.com/The-Cyber-Sidekick/TCS_CKA_2026_Exam_Scenarios

`bash
git clone https://github.com/The-Cyber-Sidekick/TCS_CKA_2026_Exam_Scenarios.git
cd TCS_CKA_2026_Exam_Scenarios/learning/scenarios/scenario14-sidecar-logging
./setup.sh # creates the cluster AND arms the scenario

solve it by hand, or:

./solution.sh # apply the answer key and verify
`

If this helped, subscribe to The Cyber SideKick on YouTube for more CKA drills, and grab the newsletter at https://thecybersidekick.beehiiv.com.

Add a WaitForFirstConsumer StorageClass and hand it the default (CKA)

The Cyber Sidekick — Thu, 06 Aug 2026 16:41:33 +0000

Add a WaitForFirstConsumer StorageClass and hand it the default (CKA)

There is a version of this CKA task that takes ninety seconds, and a version that quietly breaks the cluster you are being graded on. Today: add a StorageClass, set its binding mode so volumes wait for the pod that needs them, and make it the cluster default. The interesting part is that last word. Default is an annotation, only one class is allowed to carry it, and you are told not to touch anything that already exists. Let's run it on a live cluster.

🎥 Watch the video: https://www.youtube.com/watch?v=5Pbq42ykbn4

This is a CKA Storage walkthrough. Every command below is real output from a live cluster, and you can reproduce the whole thing yourself (scripts at the end).

The scenario

Here is the setup. The cluster already runs a dynamic provisioner, and there are already StorageClasses using it, one of which is marked as the default. Three things are asked of you. Add a class called fast-local that reuses that same provisioner and binds with WaitForFirstConsumer. Make fast-local the cluster's default, which means exactly one class ends up carrying the marker. And do all of that without editing any existing Deployment or PersistentVolumeClaim, because there is a workload here already using storage and points come off if you disturb it.

1. New StorageClass 'fast-local': reuse the EXISTING provisioner, WaitForFirstConsumer
2. Make it the cluster default (and only one class may be the default)
3. Do NOT modify any existing Deployment or PVC
A running workload in ns 'ledger' is already bound through the old default

How binding modes and the default class work

Two mechanics to have straight before typing. First, binding mode. Immediate means the provisioner creates a volume as soon as the claim appears, before anyone knows which node the pod will land on. WaitForFirstConsumer means the claim sits Pending on purpose until a pod that mounts it gets scheduled, and only then is the volume created, on that pod's node. For node-local storage that is the correct choice, and it is what this task asks for. Second, the default. It is not a field in the spec, it is an annotation, storageclass.kubernetes.io/is-default-class, set to the string true. It only affects claims that name no class at all, and a cluster with two of them is misconfigured. So the order matters: take it off the old class first, then give it to the new one.

What you are handed

Read the cluster before changing it. There are two classes already. Both use the same provisioner, rancher.io/local-path, which is the one the task tells you to reuse, so there is nothing to install. Standard carries the default marker, and it binds with WaitForFirstConsumer; bulk-hdd binds Immediately and is not the default. Note which one is default, because that is the class you will have to take it away from. Then look at what you must not break: the ledger namespace has a claim that is already Bound through standard, with a Deployment running on top of it.

$ kubectl get storageclass
NAME                 PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
bulk-hdd             rancher.io/local-path   Delete          Immediate              false                  4m24s
standard (default)   rancher.io/local-path   Delete          WaitForFirstConsumer   false                  4m24s

$ kubectl get pvc,pods -n ledger
NAME                                STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
persistentvolumeclaim/ledger-data   Bound    pvc-8bf9e803-20a7-4015-aa77-5d3879dcf33d   128Mi      RWO            standard       <unset>                 4s

NAME                              READY   STATUS    RESTARTS   AGE
pod/ledger-api-6bff9497c6-jw27j   1/1     Running   0          4s

Create the class

Now the class itself. In the exam, open the StorageClass page in the Kubernetes docs, copy the example, and delete everything the task did not ask for: no parameters, no mount options, no reclaim policy. What is left is four lines that matter. The name, fast-local. The provisioner, copied exactly from the class that already exists, because reusing it is the whole point. And volumeBindingMode set to WaitForFirstConsumer. Apply it, and the class is created. Notice there is no default annotation on it yet; that is deliberate, and it is the next step.

$ cat fast-local.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast-local
provisioner: rancher.io/local-path
volumeBindingMode: WaitForFirstConsumer

$ kubectl apply -f fast-local.yaml
storageclass.storage.k8s.io/fast-local created

Unset the old default

Before handing the title over, take it away from the current holder. A one line patch sets the annotation on standard to the string false. kubectl edit gets you to the same place if you prefer an editor, and in the exam either is fine. Look at the list now: nothing carries the default marker. That is a perfectly valid state, and it is much safer than the alternative, because if two classes claim to be the default, Kubernetes cannot choose between them and an unqualified claim will not get a class at all.

$ kubectl patch storageclass standard -p '{"metadata":{"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'
storageclass.storage.k8s.io/standard patched

$ kubectl get storageclass
NAME         PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
bulk-hdd     rancher.io/local-path   Delete          Immediate              false                  4m25s
fast-local   rancher.io/local-path   Delete          WaitForFirstConsumer   false                  0s
standard     rancher.io/local-path   Delete          WaitForFirstConsumer   false                  4m25s

Hand over the default

Same patch, other direction: set the annotation to true on fast-local. And now read the list the way a grader would. Exactly one class shows the default marker, it is ours, the provisioner matches the one that was already in the cluster, and the binding mode says WaitForFirstConsumer. Every requirement of the task is visible in that single line of output. If you had baked the annotation into the YAML when you created the class, you would still have had to unset standard, so the order of operations is the thing to remember, not which tool you used.

$ kubectl patch storageclass fast-local -p '{"metadata":{"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
storageclass.storage.k8s.io/fast-local patched

$ kubectl get storageclass
NAME                   PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
bulk-hdd               rancher.io/local-path   Delete          Immediate              false                  4m25s
fast-local (default)   rancher.io/local-path   Delete          WaitForFirstConsumer   false                  0s
standard               rancher.io/local-path   Delete          WaitForFirstConsumer   false                  4m25s

Pending on purpose

Let's prove both halves for real, without touching anything we were told to leave alone. Here is a throwaway claim in its own namespace with no storageClassName at all. Create it, and look at the class column: fast-local. Nothing told it that; it picked up the default, which is exactly what the annotation is for. And the status is Pending. That is not a failure, and this is the part people panic about in the exam. Describe the claim and the event says it plainly: waiting for first consumer to be created before binding. The volume does not exist yet because no pod has asked for it yet.

$ kubectl apply -f scratch-claim.yaml
persistentvolumeclaim/scratch-claim created

$ kubectl -n scratch get pvc
NAME            STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
scratch-claim   Pending                                      fast-local     <unset>                 4s

$ kubectl -n scratch describe pvc scratch-claim | grep -A3 'Events:'
Events:
  Type    Reason                Age   From                         Message
  ----    ------                ----  ----                         -------
  Normal  WaitForFirstConsumer  5s    persistentvolume-controller  waiting for first consumer to be created before binding

The first consumer binds it

Now create the first consumer, a single pod that mounts that claim. The scheduler places the pod, the provisioner creates the volume on that node, and the claim goes straight to Bound with a real volume behind it. That is the entire point of WaitForFirstConsumer: the storage follows the pod instead of the pod being dragged to wherever the storage happened to land. And the last check is the one the task actually grades you on for part three. The ledger namespace is exactly as we found it: still Bound, still through standard, its Deployment still running. We changed which class is default without editing a single existing object.

$ kubectl apply -f scratch-writer.yaml
pod/scratch-writer created

$ kubectl -n scratch get pvc,pods
NAME                                  STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
persistentvolumeclaim/scratch-claim   Bound    pvc-2bcf522a-4a2e-4eba-9e5d-2887b123165b   64Mi       RWO            fast-local     <unset>                 10s

NAME                 READY   STATUS    RESTARTS   AGE
pod/scratch-writer   1/1     Running   0          5s

$ kubectl get pvc,pods -n ledger
NAME                                STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
persistentvolumeclaim/ledger-data   Bound    pvc-8bf9e803-20a7-4015-aa77-5d3879dcf33d   128Mi      RWO            standard       <unset>                 15s

NAME                              READY   STATUS    RESTARTS   AGE
pod/ledger-api-6bff9497c6-jw27j   1/1     Running   0          15s

Exam tips

Things worth carrying into the exam. Copy the StorageClass example from the docs and delete every field the task did not mention, because extra fields are extra chances to be wrong. Reuse the provisioner string exactly as it appears in the existing class; do not type it from memory. Remember the annotation value is a quoted string, true, not a boolean. Always unset the old default before setting the new one, and confirm with kubectl get storageclass that exactly one class shows the marker. Do not try to move an existing claim onto your new class: a bound PVC's storage class cannot be changed anyway, and here you would be modifying an object the task explicitly protects. And if a new claim sits Pending on a WaitForFirstConsumer class, that is the design, not a bug.

Strip the docs example to what was asked: name, provisioner, volumeBindingMode
Copy the provisioner string from the existing class, do not retype it
The annotation value is the STRING "true" / "false"
Unset the old default FIRST, then confirm exactly one (default) marker
Never re-point an existing PVC: immutable once bound, and off limits here
Pending on a WaitForFirstConsumer class is expected, not a failure

Recap

fast-local created: existing provisioner + volumeBindingMode WaitForFirstConsumer
Default annotation moved off 'standard', onto 'fast-local' (exactly one marker)
Unqualified claim landed on fast-local, Pending -> Bound on first consumer
ns 'ledger' untouched, still Bound through 'standard'; subscribe + dev.to writeup

Reproduce this yourself

The entire scenario is scripted on a throwaway kind cluster: https://github.com/The-Cyber-Sidekick/TCS_CKA_2026_Exam_Scenarios

git clone https://github.com/The-Cyber-Sidekick/TCS_CKA_2026_Exam_Scenarios.git
cd TCS_CKA_2026_Exam_Scenarios/learning/scenarios/scenario13-storageclass-default
./setup.sh        # creates the cluster AND arms the scenario
# solve it by hand, or:
./solution.sh     # apply the answer key and verify

If this helped, subscribe to The Cyber SideKick on YouTube for more CKA drills, and grab the newsletter at https://thecybersidekick.beehiiv.com.

Name a container port, then expose it with a NodePort Service (CKA)

The Cyber Sidekick — Fri, 31 Jul 2026 16:39:54 +0000

Name a container port, then expose it with a NodePort Service (CKA)

This is a two minute question. In the exam you should recognize it, answer it, and move on to something hard. Today's CKA task: give an existing webserver container a named port, then expose it with a Service that is reachable both inside the cluster and on the nodes themselves. It is three deliverables, and one of them is a single command. Let's run it on a live cluster.

🎥 Watch the video: https://www.youtube.com/watch?v=K8BCAu9Fnvw

This is a CKA Services & Networking walkthrough. Every command below is real output from a live cluster, and you can reproduce the whole thing yourself (scripts at the end).

The scenario

Here is the setup. A Deployment called catalog is already running in the storefront namespace, and its container, webserver, declares no ports at all. Three things are asked of you. Give that container a port entry: name it web, port 80, protocol TCP. Put a Service called catalog-svc in front of the Deployment, and have it target that named port rather than a raw number. And make the Service answer on each node's own address as well as inside the cluster, which is a decision about the Service type. One edit, one Service, and the right type.

Container 'webserver' in Deployment 'catalog' (ns 'storefront') declares no ports
1. Give it a port entry: name 'web', port 80, protocol TCP
2. Service 'catalog-svc' in front of it, targeting that NAMED port
3. It must answer on each node's address too, so pick the right Service type
Budget: two to three minutes, then move on

How the ports chain together

Before we touch anything, understand what a containerPort is, because that is what this task is really testing. Declaring a containerPort does not open a port. nginx already listens on 80 whether or not the pod spec mentions it, and nothing blocks traffic to an undeclared port. What the entry gives you is documentation and, more importantly, a NAME. A Service's targetPort can then say web instead of 80, so the backend port can move without editing the Service. That is the chain: the Service has its own port, its targetPort points at the container's named port, and if the Service type is NodePort, the API server also allocates a port between 30000 and 32767 that is opened on every node in the cluster.

What you are handed

Start by reading what you were handed. In the storefront namespace there is one Deployment, catalog, with three pods Running, and no Service at all. Now read the pod template. The container is named webserver, not nginx, which is a good habit to build: read the container name out of the template instead of assuming it matches the image. And there is no ports block anywhere in it. The pods serve traffic fine right now, because the container listens on 80 regardless. What is missing is the declaration, and specifically the name the Service will need.

$ kubectl get deployments,pods,svc -n storefront
NAME                      READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/catalog   3/3     3            3           2s

NAME                          READY   STATUS    RESTARTS   AGE
pod/catalog-f9fb568c4-c4dqm   1/1     Running   0          2s
pod/catalog-f9fb568c4-hfksk   1/1     Running   0          2s
pod/catalog-f9fb568c4-skpjt   1/1     Running   0          2s

$ kubectl -n storefront get deployment catalog -o yaml | grep -A5 'containers:'
      containers:
      - image: nginx:1.27-alpine
        imagePullPolicy: IfNotPresent
        name: webserver
        resources: {}
        terminationMessagePath: /dev/termination-log

Add the named port

Part one. Edit the Deployment in place with kubectl edit deployment catalog in the storefront namespace. Find the container named webserver and add a ports block underneath it: name web, containerPort 80, protocol TCP. Copy the shape from the Kubernetes docs if you want, the Service page has it, but watch the indentation. Ports belongs to the container, at the same level as name and image, not to the pod spec. Write and quit, and kubectl pushes the change straight back to the API server.

$ kubectl -n storefront edit deployment catalog
      containers:
      - image: nginx:1.27-alpine
        imagePullPolicy: IfNotPresent
        name: webserver
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File

$ (after editing)
      containers:
      - image: nginx:1.27-alpine
        imagePullPolicy: IfNotPresent
        name: webserver
        ports:
        - containerPort: 80
          name: web
          protocol: TCP
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File

The rollout

Changing the pod template triggers a rolling update, so the Deployment replaces all three pods. Wait for rollout status to say successfully rolled out, because a half-finished rollout is how you end up verifying against a pod that does not have your change. Then read the field back. The container now carries containerPort 80, name web, protocol TCP. Part one is graded on exactly that.

$ kubectl -n storefront rollout status deployment/catalog
Waiting for deployment "catalog" rollout to finish: 1 out of 3 new replicas have been updated...
Waiting for deployment "catalog" rollout to finish: 1 out of 3 new replicas have been updated...
Waiting for deployment "catalog" rollout to finish: 1 out of 3 new replicas have been updated...
Waiting for deployment "catalog" rollout to finish: 2 out of 3 new replicas have been updated...
Waiting for deployment "catalog" rollout to finish: 2 out of 3 new replicas have been updated...
Waiting for deployment "catalog" rollout to finish: 2 out of 3 new replicas have been updated...
Waiting for deployment "catalog" rollout to finish: 2 out of 3 new replicas have been updated...
Waiting for deployment "catalog" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "catalog" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "catalog" rollout to finish: 1 old replicas are pending termination...
deployment "catalog" successfully rolled out

$ kubectl -n storefront get deployment catalog -o jsonpath='{.spec.template.spec.containers[0].ports}'
[{"containerPort":80,"name":"web","protocol":"TCP"}]

Expose it as NodePort

Parts two and three are one command. kubectl expose deployment catalog, name it catalog-svc, port 80, target port web, protocol TCP, type NodePort. Every part of the task maps onto a flag. Target port web is the payoff for part one, since we can now reference the port by name. And expose copies the selector from the Deployment for you, which matters more than it sounds: a hand-written Service with a selector that does not match the pod labels is the classic way to end up with a Service that has no endpoints and answers nothing. Look at the result: type NodePort, port 80, and the API server allocated a node port from the 30000 range.

$ kubectl -n storefront expose deployment catalog --name=catalog-svc --port=80 --target-port=web --protocol=TCP --type=NodePort
service/catalog-svc exposed

$ kubectl get svc -n storefront
NAME          TYPE       CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE
catalog-svc   NodePort   10.96.83.91   <none>        80:31307/TCP   0s

TargetPort + endpoints

Describe the Service and check the three lines a grader would check. TargetPort reads web, not a number, which proves the Service is following the name we added. NodePort shows the allocated port, and it is open on every node in the cluster, not just the one the pods happen to run on. And Endpoints lists all three pod IPs on port 80, which means the selector matched. An empty Endpoints line is the failure mode to look for: the Service exists, it answers nothing, and the task is marked wrong.

$ kubectl -n storefront describe svc catalog-svc
...
Type:                     NodePort
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.96.83.91
IPs:                      10.96.83.91
Port:                     <unset>  80/TCP
TargetPort:               web/TCP
NodePort:                 <unset>  31307/TCP
Endpoints:                10.244.0.23:80,10.244.0.21:80,10.244.0.22:80
Session Affinity:         None
External Traffic Policy:  Cluster
Internal Traffic Policy:  Cluster
Events:                   <none>

Curl both paths

Now prove it carries traffic, both ways. First through the ClusterIP, by name: catalog-svc.storefront resolves through cluster DNS and nginx answers. Then through the node port, using the node's own IP and the allocated port, which is the part deliverable three was really about. Same welcome page, a completely different path through the stack. One note on the lab: these curls run from a small pod inside the cluster, because on kind the node addresses are not routable from the laptop. On a real exam node you would curl the node IP and port directly from your terminal.

$ kubectl -n probe exec deploy/tester -- sh -c "curl -s --max-time 10 http://catalog-svc.storefront | grep -i '<title>'"
<title>Welcome to nginx!</title>

$ kubectl -n probe exec deploy/tester -- sh -c "curl -s --max-time 10 http://172.18.0.17:31307 | grep -i '<title>'"
<title>Welcome to nginx!</title>

Exam tips

A few things to carry into the exam. Ports goes inside the container, not the pod spec, and the wrong indentation is the number one way to lose this one. The port NAME matters: if the task names the port web, then targetPort has to say web. Use kubectl expose instead of writing YAML, it is faster and it gets the selector right by construction. If you do write YAML, the selector must match the pod labels, and remember it is the pod template labels, not the Deployment's own name. NodePort does not replace the ClusterIP, it adds to it, so nothing else has to change. And do not set a nodePort value yourself unless a specific one is asked for, since letting the API server allocate is both faster and less likely to collide.

'ports' belongs to the CONTAINER (same level as name/image), not the pod spec
targetPort: web references the NAME you added; that is why part one exists
kubectl expose --port/--target-port/--protocol/--type does parts 2 and 3 in one line
Hand-written YAML? the selector must match the POD TEMPLATE labels or Endpoints is empty
NodePort is a superset of ClusterIP; let the API server allocate the 30000-32767 port

Recap

Container 'webserver' now declares: name web, containerPort 80, protocol TCP
kubectl expose ... --target-port=web --type=NodePort created catalog-svc
describe svc: TargetPort web/TCP, NodePort allocated, 3 endpoints
Verified through the ClusterIP and through :; subscribe + dev.to writeup

Reproduce this yourself

The entire scenario is scripted on a throwaway kind cluster: https://github.com/The-Cyber-Sidekick/TCS_CKA_2026_Exam_Scenarios

git clone https://github.com/The-Cyber-Sidekick/TCS_CKA_2026_Exam_Scenarios.git
cd TCS_CKA_2026_Exam_Scenarios/learning/scenarios/scenario12-nodeport-service-expose
./setup.sh        # creates the cluster AND arms the scenario
# solve it by hand, or:
./solution.sh     # apply the answer key and verify

If this helped, subscribe to The Cyber SideKick on YouTube for more CKA drills, and grab the newsletter at https://thecybersidekick.beehiiv.com.

Create a PriorityClass, patch the Deployment, let the scheduler evict the neighbors (CKA)

The Cyber Sidekick — Thu, 30 Jul 2026 17:56:13 +0000

Patch a Deployment with a new PriorityClass (and watch preemption evict the neighbors)

Kubernetes is about to evict a perfectly healthy, running pod, and that is the correct answer to this exam question. Today's CKA task: create a new PriorityClass, patch a Deployment to use it, and roll it out on a node that has no room left. The question even warns you that pods from other deployments will be evicted. Most people read that sentence twice. By the end of this video you will know exactly why it happens, and why you should leave the victims alone. Let's run it.

This is a CKA Workloads & Scheduling walkthrough. Every command below is real output from a live cluster, and you can reproduce the whole thing yourself (scripts at the end).

The scenario

Here is the question. Create a new PriorityClass named high-priority for user workloads, with a value that is one less than the highest existing user-defined priority class. Then patch the busybox-logger Deployment, running in the priority namespace, to use that new class, and ensure the Deployment rolls out successfully. Two warnings come with it. First: it is expected that pods from other Deployments in the priority namespace get evicted. Second: do not modify those other Deployments, or you lose points. So the deliverables are one PriorityClass, one patch, one healthy rollout, and the discipline to not touch anything else.

Create PriorityClass 'high-priority': value = highest USER-DEFINED class minus one
Patch Deployment 'busybox-logger' in ns 'priority' to use it, then roll out
EXPECTED: pods from other Deployments in the namespace get evicted
Do NOT modify the other Deployments (you lose points if you do)

How priority and preemption work

Two mechanisms drive this question. First, priority. A PriorityClass is a tiny cluster-scoped object: a name and an integer. When a pod's spec names it, the pod is admitted with that integer as its priority; bigger means more important, and the default is zero. Second, preemption. When the scheduler cannot find a node with room for a pending pod, it looks for nodes where evicting pods of LOWER priority would make room, evicts them, and schedules the pending pod in the freed space. Now connect that to the task: patching the Deployment's pod template triggers a rolling update, the replacement pods carry the new high priority, and on a full node the scheduler has to preempt someone to place them. That is why the question can promise evictions in advance.

The existing classes

Step one: what already exists? kubectl get priorityclass. Read this table the way the question wants you to. The two classes starting with system, node critical and cluster critical, sit around two billion; they belong to Kubernetes itself and you never touch them or count them. The USER-DEFINED classes are the other two: low-priority at minus ten, yes, negative values are legal, and user-high-priority at one million. The highest user-defined value is therefore 1000000, and one less than that is 999999. That single subtraction is the entire trick of part one.

$ kubectl get priorityclass
NAME                      VALUE        GLOBAL-DEFAULT   AGE     PREEMPTIONPOLICY
low-priority              -10          false            6m31s   PreemptLowerPriority
system-cluster-critical   2000000000   false            6m37s   PreemptLowerPriority
system-node-critical      2000001000   false            6m37s   PreemptLowerPriority
user-high-priority        1000000      false            6m31s   PreemptLowerPriority

Create high-priority

You do not need YAML for this. kubectl create priorityclass high-priority, value 999999, and a description, because the question says it is for user workloads. One line, done. If you prefer the declarative route, the docs page for Pod Priority and Preemption has a copy-paste example, but in an exam the imperative command is thirty seconds faster. Confirm it: high-priority now exists with exactly the value we derived, one less than the highest user-defined class.

$ kubectl create priorityclass high-priority --value=999999 --description="high priority for user workloads"
priorityclass.scheduling.k8s.io/high-priority created

$ kubectl get priorityclass high-priority
NAME            VALUE    GLOBAL-DEFAULT   AGE   PREEMPTIONPOLICY
high-priority   999999   false            0s    PreemptLowerPriority

The full node

Before the patch, look at the namespace we are about to disturb. Three Deployments: busybox-logger, the one we are told to patch, plus queue-worker and metrics-agent, the other deployments the question warns about. Everything is Running. Now print each pod's actual priority. The busybox-logger pods run at zero, the default, because their template names no priority class. The neighbors run at minus ten, from the low-priority class. Keep this picture in mind: the node is nearly out of CPU, and every replacement pod we are about to create outranks everything else here by roughly a million.

$ kubectl get deployments -n priority
NAME             READY   UP-TO-DATE   AVAILABLE   AGE
busybox-logger   2/2     2            2           3s
metrics-agent    1/1     1            1           2s
queue-worker     2/2     2            2           2s

$ kubectl get pods -n priority -o custom-columns='NAME:.metadata.name,STATUS:.status.phase,PRIORITY:.spec.priority'
NAME                              STATUS    PRIORITY
busybox-logger-594f746ddf-dvfnj   Running   0
busybox-logger-594f746ddf-mnqkv   Running   0
metrics-agent-8c89d9895-g2ktn     Running   -10
queue-worker-76df598f64-vbrrj     Running   -10
queue-worker-76df598f64-w4mtl     Running   -10

Patch the deployment

Part two: patch the Deployment. The field lives in the POD TEMPLATE: spec.template.spec.priorityClassName``, set to high-priority. kubectl patch with a small JSON snippet does it in one command; kubectl edit gets you to the same place if you prefer an editor, the docs show the exact field under the Pod spec. The moment this lands, the Deployment controller starts a rolling update: it must bring up new pods that carry priority 999999 on a node that has no space for them. Watch what the scheduler does about that.

`console $ kubectl -n priority patch deployment busybox-logger -p '{"spec":{"template":{"spec":{"priorityClassName":"high-priority"}}}}' deployment.apps/busybox-logger patched `

The eviction

Here is the moment the question promised. The fresh busybox-logger pod went Pending, the scheduler compared priorities, and a low-priority neighbor is being terminated to make room; its replacement sits Pending because, at priority minus ten, it cannot evict anyone. Kubernetes writes this decision down: the events show Preempted, naming the victim, evicted by the scheduler on behalf of the higher-priority pod. This is not a failure and there is nothing to fix. The question told you it was expected; your only job is to let it happen.

`console
$ kubectl get pods -n priority
NAME READY STATUS RESTARTS AGE
busybox-logger-594f746ddf-dvfnj 1/1 Running 0 4s
busybox-logger-594f746ddf-mnqkv 1/1 Running 0 4s
busybox-logger-645b98678b-kfwmw 0/1 Pending 0 0s
metrics-agent-8c89d9895-68282 0/1 Pending 0 0s
metrics-agent-8c89d9895-g2ktn 1/1 Terminating 0 3s
queue-worker-76df598f64-vbrrj 1/1 Running 0 3s
queue-worker-76df598f64-w4mtl 1/1 Running 0 3s

$ kubectl get events -n priority | grep -i preempt
0s Warning FailedScheduling pod/metrics-agent-8c89d9895-68282 0/1 nodes are available: 1 Insufficient cpu. no new claims to deallocate, preemption: 0/1 nodes are available: 1 No preemption victims found for incoming pod.
0s Normal Preempted pod/metrics-agent-8c89d9895-g2ktn Preempted by pod 6a344267-de23-4005-b466-00fe7699cf20 on node cka-scenario11-control-plane
`

Rollout + verify

Part three: prove the rollout finished. rollout status reports the deployment successfully rolled out. Now verify like a grader: print the priorities again. Both busybox-logger pods now show priority 999999 with class high-priority, and the neighbors still hold their original spec, untouched. Notice the namespace healed itself: once the rolling update retired the old zero-priority pods, their CPU freed up and the evicted neighbor's replacement scheduled on its own. In the exam it may just as well stay Pending, and that is fine too; the question graded the PriorityClass, the patch, and the rollout, not the neighbors' comfort.

`console
$ kubectl -n priority rollout status deployment/busybox-logger
Waiting for deployment "busybox-logger" rollout to finish: 1 out of 2 new replicas have been updated...
Waiting for deployment "busybox-logger" rollout to finish: 1 out of 2 new replicas have been updated...
Waiting for deployment "busybox-logger" rollout to finish: 1 out of 2 new replicas have been updated...
Waiting for deployment "busybox-logger" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "busybox-logger" rollout to finish: 1 old replicas are pending termination...
deployment "busybox-logger" successfully rolled out

$ kubectl get pods -n priority -o custom-columns='NAME:.metadata.name,STATUS:.status.phase,PRIORITY:.spec.priority,CLASS:.spec.priorityClassName'
NAME STATUS PRIORITY CLASS
busybox-logger-645b98678b-bf5t5 Running 999999 high-priority
busybox-logger-645b98678b-kfwmw Running 999999 high-priority
metrics-agent-8c89d9895-68282 Running -10 low-priority
queue-worker-76df598f64-vbrrj Running -10 low-priority
queue-worker-76df598f64-w4mtl Running -10 low-priority

$ kubectl get pods -n priority
NAME READY STATUS RESTARTS AGE
busybox-logger-645b98678b-bf5t5 1/1 Running 0 7s
busybox-logger-645b98678b-kfwmw 1/1 Running 0 13s
metrics-agent-8c89d9895-68282 1/1 Running 0 13s
queue-worker-76df598f64-vbrrj 1/1 Running 0 16s
queue-worker-76df598f64-w4mtl 1/1 Running 0 16s
`

Exam tips

The traps in this question are all reading comprehension. Highest USER-DEFINED class means you skip everything that starts with system; counting the two billion system classes gives a wildly wrong value. One less means minus one, so from one million you write 999999; do not invent a round number. kubectl create priorityclass is the fastest correct answer, and the priorityClassName field goes in the pod template, not on the Deployment's own spec. When the neighbors get evicted, leave them alone: the question explicitly says modifying other deployments costs points, and preemption is the intended behavior, not an incident. And verify with the pods, not the deployment: custom-columns on .spec.priority shows exactly what the grader checks.

'Highest user-defined' = ignore system-node-critical / system-cluster-critical
'One less' = 1000000 - 1 = 999999; kubectl create priorityclass does it in one line
priorityClassName goes in spec.template.spec (the POD template)
Evicted neighbors are EXPECTED: do not 'fix' them, do not modify their Deployments
Verify like the grader: pods' .spec.priority + rollout status

Recap

Highest user-defined class 1000000 -> high-priority created at 999999
One patch: spec.template.spec.priorityClassName = high-priority
Full node -> scheduler PREEMPTS a low-priority pod (Preempted event), rollout completes
Neighbors evicted as promised, never modified; subscribe + dev.to writeup

Reproduce this yourself

The entire scenario is scripted on a throwaway kind cluster: https://github.com/The-Cyber-Sidekick/TCS_CKA_2026_Exam_Scenarios

`bash
git clone https://github.com/The-Cyber-Sidekick/TCS_CKA_2026_Exam_Scenarios.git
cd TCS_CKA_2026_Exam_Scenarios/learning/scenarios/scenario11-priorityclass-preemption
./setup.sh # creates the cluster AND arms the scenario

solve it by hand, or:

./solution.sh # apply the answer key and verify
`

If this helped, subscribe to The Cyber SideKick on YouTube for more CKA drills, and grab the newsletter at https://thecybersidekick.beehiiv.com.

Install Argo CD with Helm, and survive the missing-CRD trap (CKA)

The Cyber Sidekick — Wed, 22 Jul 2026 16:38:38 +0000

Install Argo CD with Helm (and survive the missing-CRD trap)

This exam question looks like three Helm commands: add a repository, render a template, install a release. And it hands you a comforting line: the Argo CD CRDs have already been pre-installed in the cluster. In this lab that line is false, on purpose, and what happens next teaches you exactly what crds.install=false really means. Let's run it.

🎥 Watch the video: https://www.youtube.com/watch?v=kYcHmIh_eHw

This is a CKA Cluster Architecture, Installation & Configuration walkthrough. Every command below is real output from a live cluster, and you can reproduce the whole thing yourself (scripts at the end).

The scenario

Here is the question. Add the official Argo CD Helm repository to the cluster under the name argo; the URL is given in the question, you never have to memorize repository links. The Argo CD CRDs, it says, have already been pre-installed. Generate a Helm template for release argocd, chart version 7.7.3, in the argocd namespace, save it to argo-helm.yaml, and configure the chart to not install the CRDs. Then install the release with the same name, version and namespace, again without CRDs. And one thing you do not have to do: configure access to the Argo CD server UI. Three commands, one premise. Remember the premise.

Add the official Argo CD Helm repo as 'argo' (URL is given in the question)
Premise: the Argo CD CRDs 'have already been pre-installed in the cluster'
helm template: release argocd, chart version 7.7.3, ns argocd -> argo-helm.yaml, NO CRDs
helm install: same release/version/namespace, NO CRDs; the server UI is out of scope

How the Helm flow works

Three Helm verbs do the work. helm repo add registers where charts come from, under a name you choose. helm template renders the chart into plain YAML on your machine; nothing touches the cluster, which is why the question can ask you to save the output to a file. helm install submits that same render to the cluster as a managed release. The flag that ties them together is set crds.install=false. The argo-cd chart bundles the Argo CD custom resource definitions, and when a question says the CRDs are managed separately, your job is to keep the chart's hands off them, in the template and in the install. Keep the two commands identical apart from the verb; if they drift, the file you saved describes a different system than the one you deployed.

Add the argo repo

Step one. helm repo add argo, then the URL from the question. Helm fetches the repository index immediately, so a mistyped URL fails right here instead of at install time. helm repo list confirms the repository is registered under exactly the name the question asked for: argo.

$ helm repo add argo https://argoproj.github.io/argo-helm
"argo" has been added to your repositories

$ helm repo list
NAME        URL                                     
argo        https://argoproj.github.io/argo-helm

Render the template

Step two, the template. Release name argocd, chart argo/argo-cd, version 7.7.3 pinned exactly, namespace argocd, crds.install set to false, and the whole thing redirected into argo-helm.yaml. The command prints nothing because the entire render went into the file, and it is substantial: over three thousand lines of YAML. Here is the check that proves the flag worked: grep the file for kind CustomResourceDefinition. Zero. This chart normally ships three CRDs; with the flag set, the render contains none of them.

$ helm template argocd argo/argo-cd --version 7.7.3 --namespace argocd --set crds.install=false > argo-helm.yaml

$ wc -l argo-helm.yaml
3057 argo-helm.yaml

$ grep -c 'kind: CustomResourceDefinition' argo-helm.yaml
0

helm install

Step three is the same command with the verb swapped: helm install, same release name, same chart, same version, same namespace, same crds.install false. Helm prints the release notes, and helm ls in the argocd namespace shows release argocd, revision one, status deployed, chart argo-cd 7.7.3, app version 2.13.0. As far as Helm is concerned, the job is done. The cluster is about to disagree.

$ helm install argocd argo/argo-cd --version 7.7.3 --namespace argocd --set crds.install=false
NAME: argocd
LAST DEPLOYED: Mon Jul 13 09:49:46 2026
NAMESPACE: argocd
STATUS: deployed
REVISION: 1
DESCRIPTION: Install complete
TEST SUITE: None

$ helm ls -n argocd
NAME    NAMESPACE   REVISION    UPDATED                                 STATUS      CHART           APP VERSION
argocd  argocd      1           2026-07-13 09:49:46.583903829 -0400 EDT deployed    argo-cd-7.7.3   v2.13.0

The crashloop

Give the pods a moment and look again. Redis and dex are up, but the server and the applicationset controller are already in a crash loop, and the application controller never goes ready; everything that watches Argo CD resources is failing. Pull the logs from the server and the last line says exactly why: level fatal, the server could not find the requested resource. That is a component asking the API server for a resource type that does not exist. We told the chart not to install the CRDs because the question said they were pre-installed. They are not. The premise was false, and Helm had no way to know.

$ kubectl get pods -n argocd
NAME                                               READY   STATUS             RESTARTS      AGE
argocd-application-controller-0                    0/1     Running            0             84s
argocd-applicationset-controller-bcdc99fcf-jdfj9   0/1     CrashLoopBackOff   3 (41s ago)   84s
argocd-dex-server-77f8fcf9d9-plbt8                 1/1     Running            0             84s
argocd-notifications-controller-7769fd5fd-rdqzh    1/1     Running            0             84s
argocd-redis-768545f6f-rjsgq                       1/1     Running            0             84s
argocd-repo-server-fd74968b8-dr772                 1/1     Running            0             84s
argocd-server-74b5fbf858-cn5t4                     0/1     CrashLoopBackOff   4 (4s ago)    84s

$ kubectl logs deploy/argocd-server -n argocd --tail=3
time="2026-07-13T13:51:10Z" level=info msg="Starting configmap/secret informers"
time="2026-07-13T13:51:11Z" level=info msg="Configmap/secret informer synced"
time="2026-07-13T13:51:11Z" level=fatal msg="the server could not find the requested resource (post appprojects.argoproj.io)"

The missing CRDs

Confirm the diagnosis in two seconds: kubectl get crd, grep argoproj, nothing. So we install the CRDs ourselves, but pinned to the right version, because CRDs and controllers drift apart across releases. The chart's appVersion tells you which Argo CD this chart deploys: helm show chart says 7.7.3 ships version 2.13.0. Apply the three CRD manifests from the Argo CD repository at exactly that tag: applications, applicationsets, and appprojects. Now the same grep finds all three.

$ kubectl get crd | grep argoproj
No resources found

$ helm show chart argo/argo-cd --version 7.7.3 | grep appVersion
appVersion: v2.13.0
version: 7.7.3

$ kubectl apply -f crds/
customresourcedefinition.apiextensions.k8s.io/applications.argoproj.io created
customresourcedefinition.apiextensions.k8s.io/applicationsets.argoproj.io created
customresourcedefinition.apiextensions.k8s.io/appprojects.argoproj.io created

$ kubectl get crd | grep argoproj
applications.argoproj.io      2026-07-13T13:51:16Z
applicationsets.argoproj.io   2026-07-13T13:51:16Z
appprojects.argoproj.io       2026-07-13T13:51:16Z

Restart + verify

The crashlooped pods would eventually recover on their own as the backoff retries, but do not sit and wait in an exam. One rollout restart of the workloads in the namespace brings fresh pods up immediately, and this time every informer finds its CRDs. A minute later the whole namespace is Running and Ready. That is the full question: repository added as argo, template saved with zero CRDs in it, release installed at 7.7.3, and a healthy Argo CD. And notice what the fix was not: we never touched the Helm release. The release was correct; the environment broke the promise.

$ kubectl -n argocd rollout restart deployment
deployment.apps/argocd-applicationset-controller restarted
deployment.apps/argocd-dex-server restarted
deployment.apps/argocd-notifications-controller restarted
deployment.apps/argocd-redis restarted
deployment.apps/argocd-repo-server restarted
deployment.apps/argocd-server restarted

$ kubectl get pods -n argocd
NAME                                                READY   STATUS    RESTARTS   AGE
argocd-application-controller-0                     1/1     Running   0          11s
argocd-applicationset-controller-547b7778c8-psvtd   1/1     Running   0          58s
argocd-dex-server-65cd9bfbb8-ptbnj                  1/1     Running   0          58s
argocd-notifications-controller-76fcfb8864-6s7sb    1/1     Running   0          58s
argocd-redis-6d9cb5f875-6z8xw                       1/1     Running   0          58s
argocd-repo-server-87cf9c697-dcqzx                  1/1     Running   0          58s
argocd-server-6f7b69c655-wgrhn                      1/1     Running   0          58s

Exam tips

A few traps. The repository URL and the chart version are printed in the question; copy them, pin --version exactly, and never guess. Template and install must carry identical values: same namespace, same set crds.install=false, only the verb changes. When a question states a premise like CRDs are pre-installed, verify it, it costs two seconds: kubectl get crd, grep argoproj. And learn the signature: a controller crashlooping with the server could not find the requested resource means a missing CRD, and the chart's appVersion tells you exactly which CRD version to apply.

The repo URL + chart version are IN the question: copy them, pin --version exactly
template and install take identical values; only the verb changes
Premises are verifiable: 'CRDs pre-installed' -> kubectl get crd | grep argoproj (2 seconds)
'could not find the requested resource' = missing CRD; match it to the chart's appVersion

Recap

helm repo add argo -> helm template -> helm install, pinned to 7.7.3, crds.install=false on both
helm ls said 'deployed'; the pods crashlooped: Helm status is not health
Missing argoproj CRDs -> 'could not find the requested resource'; apply CRDs at appVersion v2.13.0
rollout restart, all pods Running; subscribe + dev.to writeup

Reproduce this yourself

The entire scenario is scripted on a throwaway kind cluster: https://github.com/The-Cyber-Sidekick/TCS_CKA_2026_Exam_Scenarios

git clone https://github.com/The-Cyber-Sidekick/TCS_CKA_2026_Exam_Scenarios.git
cd TCS_CKA_2026_Exam_Scenarios/learning/scenarios/scenario10-argocd-helm-install
./setup.sh        # creates the cluster AND arms the scenario
# solve it by hand, or:
./solution.sh     # apply the answer key and verify

If this helped, subscribe to The Cyber SideKick on YouTube for more CKA drills, and grab the newsletter at https://thecybersidekick.beehiiv.com.

Install Calico and prove NetworkPolicy enforcement (CKA Services & Networking)

The Cyber Sidekick — Wed, 22 Jul 2026 16:09:06 +0000

Install Calico and prove NetworkPolicy enforcement

The exam says install and configure a CNI of your choice, flannel or Calico, and then adds one line that makes the choice for you: it must support NetworkPolicy enforcement. flannel does not enforce policies, so this is a Calico question. Let's install it from the manifests, verify it, and prove a policy actually blocks traffic.

🎥 Watch the video: https://www.youtube.com/watch?v=7Eaf7Wogsa4

This is a CKA Services & Networking walkthrough. Every command below is real output from a live cluster, and you can reproduce the whole thing yourself (scripts at the end).

The scenario

Here is the setup. A fresh cluster has no CNI, so its nodes are NotReady and CoreDNS is Pending. The question offers flannel or Calico, says install from manifest files, do not use Helm, and lists three requirements: the CNI is properly installed and configured, pods can communicate with each other, and it supports NetworkPolicy enforcement. That last requirement is the trap. flannel gives you pod networking but silently ignores NetworkPolicy objects. Only Calico satisfies all three.

A fresh cluster with NO CNI: nodes NotReady, CoreDNS Pending
Install a CNI of your choice (flannel or Calico) from manifests, no Helm
Pods must communicate with each other, and the CNI must be properly installed
'Must support NetworkPolicy enforcement' => flannel is out, Calico is the answer

How the Calico operator install works

Calico's manifest install is operator based and comes in three parts. First the operator CRDs, then the Tigera operator itself, a deployment that knows how to install and manage Calico. Then you hand the operator an Installation resource describing the Calico you want, and it deploys everything into the calico-system namespace to match. Two things to know before you touch the keyboard. The Installation's IP pool cidr must agree with the cluster's pod CIDR, which is 192.168.0.0/16 here. And these manifests need kubectl create, not apply: the CRD file is bigger than the annotation kubectl apply attaches, so apply fails on it.

No CNI: nodes NotReady

Start by confirming the starting state. Every node is NotReady, the classic symptom of a missing CNI. And listing pods across all namespaces shows two things: CoreDNS is Pending because there is no pod network for it to join, and there are no calico or flannel pods anywhere, so no CNI is installed yet. This is a fresh cluster, and the network plugin is ours to install.

$ kubectl get nodes
NAME                          STATUS     ROLES           AGE   VERSION
cka-scenario9-control-plane   NotReady   control-plane   8s    v1.36.1
cka-scenario9-worker          NotReady   <none>          0s    v1.36.1

$ kubectl get pods -A
NAMESPACE            NAME                                                  READY   STATUS    RESTARTS   AGE
kube-system          coredns-589f44dc88-522x8                              0/1     Pending   0          1s
kube-system          coredns-589f44dc88-zsxh2                              0/1     Pending   0          1s
kube-system          etcd-cka-scenario9-control-plane                      0/1     Running   0          8s
kube-system          kube-apiserver-cka-scenario9-control-plane            1/1     Running   0          8s
kube-system          kube-controller-manager-cka-scenario9-control-plane   1/1     Running   0          8s
kube-system          kube-proxy-ds9pp                                      1/1     Running   0          1s
kube-system          kube-proxy-lwx6m                                      0/1     Pending   0          3s
kube-system          kube-scheduler-cka-scenario9-control-plane            0/1     Running   0          8s
local-path-storage   local-path-provisioner-855c7b7774-75cbk               0/1     Pending   0          1s

Create the CRDs + operator

Install the operator pieces with kubectl create. The first file registers the custom resource definitions, thirty two of them. The second deploys the Tigera operator itself. Note the verb: create, not apply. Some of these CRDs are larger than the last-applied annotation kubectl apply wants to attach, so apply fails on this file, and the Calico docs say create for exactly that reason. A few seconds later the operator pod is Running in the tigera-operator namespace, waiting for us to tell it what to install.

$ kubectl create -f operator-crds.yaml
...
customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/stagedglobalnetworkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/stagedkubernetesnetworkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/stagednetworkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/tiers.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/adminnetworkpolicies.policy.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/baselineadminnetworkpolicies.policy.networking.k8s.io created

$ kubectl create -f tigera-operator.yaml
namespace/tigera-operator created
serviceaccount/tigera-operator created
clusterrole.rbac.authorization.k8s.io/tigera-operator-secrets created
clusterrole.rbac.authorization.k8s.io/tigera-operator created
clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created
rolebinding.rbac.authorization.k8s.io/tigera-operator-secrets created
deployment.apps/tigera-operator created

$ kubectl get pods -n tigera-operator
NAME                               READY   STATUS    RESTARTS   AGE
tigera-operator-696d7c8fc4-59bf5   1/1     Running   0          4s

The Installation resource

Now describe the Calico you want. This Installation resource is the heart of the stock custom-resources file, and the field that matters is the IP pool cidr: 192.168.0.0/16, matching this cluster's pod CIDR. If those two disagree you get the same class of pain as a flannel CIDR mismatch, so always check before you create it. Create it and the operator takes over, pulling Calico up piece by piece. The tigerastatus resource is your progress bar: when every row reports Available True, the install is done.

$ cat custom-resources.yaml
# The Installation resource: the Tigera operator watches for this and deploys Calico
# to match it. The ipPool cidr MUST agree with the cluster's pod CIDR (192.168.0.0/16).
apiVersion: operator.tigera.io/v1
kind: Installation
metadata:
  name: default
spec:
  calicoNetwork:
    ipPools:
      - cidr: 192.168.0.0/16
        encapsulation: VXLANCrossSubnet

$ kubectl create -f custom-resources.yaml
installation.operator.tigera.io/default created

$ kubectl get tigerastatus
NAME      AVAILABLE   PROGRESSING   DEGRADED   SINCE
calico    True        False         False      0s
ippools   True        False         False      55s

Calico up, nodes Ready

Trust but verify. In calico-system there is a calico-node pod on every node, that is the dataplane that wires up pods and enforces policy, plus typha and the kube-controllers, all Running. And the payoff: both nodes have flipped to Ready, and CoreDNS is finally scheduled. The CNI is properly installed and configured, which was requirement one.

$ kubectl get pods -n calico-system -o wide
NAME                                       READY   STATUS    RESTARTS   AGE   IP                NODE                          NOMINATED NODE   READINESS GATES
calico-kube-controllers-6c8496f5c8-flpxp   1/1     Running   0          63s   192.168.227.197   cka-scenario9-control-plane   <none>           <none>
calico-node-dpqqj                          1/1     Running   0          63s   172.18.0.15       cka-scenario9-worker          <none>           <none>
calico-node-pdgjt                          1/1     Running   0          63s   172.18.0.16       cka-scenario9-control-plane   <none>           <none>
calico-typha-6f8c54fdf6-mnvnn              1/1     Running   0          63s   172.18.0.15       cka-scenario9-worker          <none>           <none>
csi-node-driver-6mlr5                      2/2     Running   0          63s   192.168.227.194   cka-scenario9-control-plane   <none>           <none>
csi-node-driver-ntcxl                      2/2     Running   0          63s   192.168.178.65    cka-scenario9-worker          <none>           <none>

$ kubectl get nodes
NAME                          STATUS   ROLES           AGE   VERSION
cka-scenario9-control-plane   Ready    control-plane   82s   v1.36.1
cka-scenario9-worker          Ready    <none>          73s   v1.36.1

Prove pods can talk

Requirement two: pods must communicate with each other. Create two test pods pinned to different nodes and list them wide, each gets an IP from 192.168.0.0/16 on a different host. Then exec into test one and ping test two's pod IP: replies come back across the nodes, so pod-to-pod networking works end to end. One practical note: ping the IP, not the pod name. Bare pods do not get DNS records, so a name lookup fails with bad address even on a perfectly healthy network.

$ kubectl apply -f connectivity-test.yaml
pod/test1 created
pod/test2 created

$ kubectl get pods -o wide -l app=conn-test
NAME    READY   STATUS    RESTARTS   AGE   IP                NODE                          NOMINATED NODE   READINESS GATES
test1   1/1     Running   0          3s    192.168.178.66    cka-scenario9-worker          <none>           <none>
test2   1/1     Running   0          3s    192.168.227.198   cka-scenario9-control-plane   <none>           <none>

$ kubectl exec test1 -- ping -c 3 <test2-ip>
PING 192.168.227.198 (192.168.227.198): 56 data bytes
64 bytes from 192.168.227.198: seq=0 ttl=62 time=0.351 ms
64 bytes from 192.168.227.198: seq=1 ttl=62 time=0.091 ms
64 bytes from 192.168.227.198: seq=2 ttl=62 time=0.100 ms

--- 192.168.227.198 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.091/0.180/0.351 ms

Prove policy enforcement

Requirement three is the one flannel cannot do: NetworkPolicy enforcement. This default-deny policy comes straight from the Kubernetes docs. The empty podSelector selects every pod in the default namespace, and listing both Ingress and Egress with no rules denies all traffic in both directions. Apply it, confirm it exists, and run the exact same ping again. Three packets, zero replies, one hundred percent loss. The policy is not just stored in the API, Calico is enforcing it on the wire. That failing ping is the proof the question asked for.

$ cat deny-all.yaml
# Default-deny-all, straight from the Kubernetes docs: the empty podSelector selects
# EVERY pod in the namespace; both policyTypes with no rules = deny all traffic.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: default
spec:
  podSelector: {}
  policyTypes:
    - Ingress
    - Egress

$ kubectl apply -f deny-all.yaml
networkpolicy.networking.k8s.io/default-deny-all created

$ kubectl get networkpolicy
NAME               POD-SELECTOR   AGE
default-deny-all   <none>         0s

$ kubectl exec test1 -- ping -c 3 -w 5 <test2-ip>
PING 192.168.227.198 (192.168.227.198): 56 data bytes

--- 192.168.227.198 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss
command terminated with exit code 1

Exam tips

A few traps. When a question says the CNI must support NetworkPolicy enforcement, that one phrase eliminates flannel; pick Calico and move on. Use kubectl create for the Calico operator manifests, because apply chokes on the big CRD file. Check that the Installation's IP pool cidr matches the cluster pod CIDR before you create it, then watch kubectl get tigerastatus until everything is Available. And verification means behavior, not status: prove the ping works, then prove a default-deny breaks the same ping.

'Must support NetworkPolicy enforcement' => flannel is out, use Calico
kubectl create -f (not apply) for the Calico manifests: the CRD file is too big for apply
Installation ipPool cidr must match the cluster pod CIDR; watch 'kubectl get tigerastatus'
Verify behavior: ping works, then a default-deny makes the same ping fail

Recap

NetworkPolicy enforcement required => Calico, installed with kubectl create (no Helm)
operator CRDs -> Tigera operator -> Installation (ipPool = cluster pod CIDR)
tigerastatus Available, nodes Ready, cross-node ping works
default-deny => the same ping fails: enforcement proven; subscribe + dev.to writeup

Reproduce this yourself

The entire scenario is scripted on a throwaway kind cluster: https://github.com/The-Cyber-Sidekick/TCS_CKA_2026_Exam_Scenarios

git clone https://github.com/The-Cyber-Sidekick/TCS_CKA_2026_Exam_Scenarios.git
cd TCS_CKA_2026_Exam_Scenarios/learning/scenarios/scenario9-calico-network-policy
./setup.sh        # creates the cluster AND arms the scenario
# solve it by hand, or:
./solution.sh     # apply the answer key and verify

If this helped, subscribe to The Cyber SideKick on YouTube for more CKA drills, and grab the newsletter at https://thecybersidekick.beehiiv.com.

Self-Hosting LLMs on Kubernetes: When vLLM Beats Managed APIs on Cost

The Cyber Sidekick — Sat, 18 Jul 2026 19:24:13 +0000

A practitioner's cost-benefit analysis of vLLM on Kubernetes versus OpenAI and other managed inference APIs for high-volume LLM workloads.

Organizations running high-volume LLM inference can reduce per-token costs by 60-80% by self-hosting with vLLM on Kubernetes, but the economics only work after solving GPU scheduling, model serving, and operational complexity. This article walks platform engineers through the breakeven analysis, infrastructure architecture, and operational tooling required to make self-hosted inference viable in production.

The Economics: Where Self-Hosted Inference Wins

The LLM inference market is splitting into two camps: managed API providers like OpenAI, Anthropic, and Google Vertex AI charging per-token premiums, and self-hosted inference stacks on Kubernetes that amortize GPU costs across high request volumes. The crossover point sits at roughly 10 to 20 million tokens per day per model, where reserved A100 or H100 instances typically break even against OpenAI API pricing within 3 to 6 months. That window shrinks further when you factor in quantization: AWQ and GPTQ reduce model memory footprint by 2 to 4x, letting you serve more concurrent requests from the same GPU, which directly compresses the breakeven timeline. The economic case is also being accelerated by inference-time compute scaling, where o1-style chain-of-thought reasoning dramatically inflates output token volumes, making per-token API billing increasingly untenable for high-throughput production applications. Open-weight models like Llama 3.1, Mistral, Qwen2, and Gemma 2 have meanwhile closed the capability gap with proprietary APIs for many enterprise use cases, removing the last justification for paying the managed API premium when volume is high enough.

vLLM and Kubernetes: The Infrastructure Stack

vLLM, developed by LMSys, implements PagedAttention, a memory management algorithm inspired by OS virtual memory paging that achieves near-zero KV cache memory waste and delivers up to 24x higher throughput than naive HuggingFace Transformers serving. Its continuous batching keeps GPU utilization above 80% under sustained load, compared to static batching which frequently yields sub-40% utilization, and vLLM 0.4 added production-grade features including OpenAI-compatible REST endpoints, speculative decoding, chunked prefill, and multi-LoRA serving that make it a viable drop-in API replacement. On the Kubernetes side, the NVIDIA GPU Operator automates GPU driver installation, device plugin deployment, and MIG partitioning, with H100 MIG allowing a single 80GB GPU to be sliced into up to 7 isolated instances so smaller 7B parameter models can be scheduled on fractional GPU resources alongside larger workloads. For multi-node tensor parallelism, platform teams are choosing between KubeRay with RayServe and native vLLM Kubernetes deployments, with KubeRay offering richer pipeline parallelism and autoscaling primitives while native vLLM deployments reduce operational surface area. Karpenter handles GPU node autoscaling, and spot instance availability for H100 NVLink, AMD MI300X, and Google TPU v5 hardware is continuing to lower amortized cost per token on major cloud providers.

Operational Complexity: What You Are Actually Signing Up For

The 60-80% cost reduction is real, but it comes with an operational contract that managed APIs abstract away entirely, and platform teams need to account for that engineering investment before committing. Production vLLM deployments require Prometheus and OpenTelemetry instrumentation at the token level to surface queue depth, time-to-first-token, and inter-token latency metrics, and autoscaling policies need to be built around queue depth rather than the CPU and memory signals that Kubernetes HPA uses by default. Model version management requires Argo Rollouts or equivalent canary tooling to safely promote new model weights or LoRA adapters without dropping traffic, and multi-LoRA hot-swapping for fine-tuned model variants adds another layer of complexity around adapter lifecycle management. Multi-tenant vLLM deployments in internal LLM platforms need namespace isolation, per-team rate limiting, and chargeback instrumentation so that cost savings are actually attributed and not just absorbed into platform overhead. Teams that underestimate this operational complexity often find that the first three months of self-hosting are net-negative on engineering productivity, which is why the 3 to 6 month breakeven estimate should be treated as a floor, not a guarantee, for teams without prior GPU infrastructure experience.

Conclusion

Self-hosted vLLM on Kubernetes is genuinely the right choice for organizations running sustained high-volume LLM inference, but the decision should be driven by a clear-eyed token volume analysis rather than enthusiasm for infrastructure ownership. At 10 to 20 million tokens per day per model, the economics are compelling and the tooling ecosystem around GPU Operator, KubeRay, Karpenter, and Prometheus has matured enough to make production deployments tractable for experienced platform teams. Looking ahead, inference-time compute scaling will continue to inflate token volumes across the industry, which will push more organizations past the breakeven threshold faster than they expect. Hardware commoditization through spot H100 and AMD MI300X availability will further compress managed API margins, making the self-hosted case stronger over the next 12 to 18 months. Platform teams that invest now in building internal LLM inference platforms on Kubernetes, with solid observability, autoscaling, and model lifecycle tooling, will be positioned to serve inference demand at a cost structure that managed APIs structurally cannot match at scale.

Technologies covered: vLLM, Kubernetes, GPU resource management, container orchestration, inference optimization

Sources aggregated from: CNCF Blog, Kubernetes.io, DevOps Weekly

📬 Stay current with cloud-native

Get the latest Kubernetes, DevOps, and platform engineering insights delivered to your inbox.

Subscribe to The Cyber SideKick Newsletter — free, no spam, unsubscribe anytime.

Platform Engineering for AI-Native Workloads: Managing Cognitive Load at Scale

The Cyber Sidekick — Mon, 13 Jul 2026 13:17:53 +0000

How platform teams can architect internal developer platforms optimized for GPU scheduling, model serving, and experiment tracking without overwhelming ML engineers.

AI workloads are multiplying exponentially, yet fewer than 30% of organizations have extended their internal developer platforms to natively support GPU workloads and ML pipelines, according to Puppet's 2024 State of Platform Engineering report. Platform teams that close this gap by layering AI-native abstractions atop Kubernetes can dramatically reduce cognitive load for ML engineers while containing runaway GPU costs through intelligent resource orchestration.

The Abstraction Gap Undermining AI Platform Maturity

Kubernetes was architected for stateless microservices, and stretching it to accommodate high-memory-bandwidth training jobs, long-running batch workloads, and sub-100ms inference SLAs exposes serious abstraction gaps that raw kubectl access cannot paper over. The result is a tax on ML engineers who must simultaneously master Kubernetes primitives, GPU driver nuances, and distributed training frameworks before writing a single line of model code. Purpose-built control planes including Run:ai, Volcano, and Loft's vCluster are gaining traction precisely because they sit atop Kubernetes and expose ML-specific primitives, such as experiment tracking dashboards, model registries, and GPU quota views, shielding practitioners from infrastructure complexity. The 87% of organizations with mature platform engineering practices that report measurably reduced developer cognitive load have one thing in common: they treat the platform as a product with well-defined, opinionated abstractions rather than as a collection of loosely integrated open-source tools.

GPU Orchestration and Multi-Tenancy Through MIG and KubeRay

Granular GPU resource isolation is now achievable without whole-GPU allocation, and platform teams that ignore this capability are leaving significant efficiency gains on the table. NVIDIA's MIG Manager within the GPU Operator allows a single A100 to be partitioned into up to seven isolated instances, enabling Kubernetes resource quotas as specific as 1g.10gb, which translates to up to 40% less GPU idle time in multi-tenant inference clusters compared to whole-GPU scheduling. For distributed training and online inference, KubeRay has emerged as the most compelling unified compute layer, with adoption growing over 300% year-over-year in 2023 and 2024 based on GitHub stars and Helm chart downloads. Organizations deploying RayService for model inference via KubeRay's operator-based CRDs report sub-100ms p99 latency at scales exceeding 10,000 requests per second through Kubernetes-native horizontal autoscaling, making it a credible alternative to purpose-built inference servers for teams already invested in the Ray ecosystem.

GitOps, Observability, and FinOps as First-Class Platform Concerns

Bringing software engineering discipline to the ML lifecycle requires treating model weights, feature stores, and evaluation datasets with the same versioning rigor applied to application code, and GitOps-driven workflows through ArgoCD and Kubeflow Pipelines v2 with an Argo Workflows backend are making this operationally tractable at scale. Service mesh capabilities via Istio extend this discipline into inference traffic management, enabling weighted routing for shadow deployments and header-based routing for model version targeting, which gives platform teams a safe mechanism for canary model promotions without custom networking code. Observability remains a critical and underinvested area, with leading teams instrumenting ML pipelines through OpenTelemetry, Prometheus custom metrics, and distributed tracing via Tempo to correlate model performance degradation with infrastructure-level anomalies in a single unified trace. On the cost side, GPU spend now dominates cloud bills for AI-heavy organizations, making spot-instance-aware schedulers, idle GPU detection via Prometheus alerting, and per-team chargeback dashboards in Grafana not optional enhancements but core platform features that directly influence engineering budget conversations.

Conclusion

The platform engineering teams that will define the next generation of AI infrastructure are those treating AI-native workloads not as an edge case bolted onto an existing IDP but as the primary design constraint for every abstraction they build. The convergence of MLOps tooling with traditional platform engineering practices is accelerating, and the organizations moving fastest are the ones investing simultaneously in GPU resource isolation through MIG partitioning, unified compute layers like KubeRay, GitOps-native model promotion pipelines, and FinOps visibility that holds teams accountable for GPU utilization. As foundation model sizes grow and inference latency budgets tighten, the pressure on platform teams to deliver self-service ML infrastructure without cognitive overload will only intensify, making purpose-built AI platform abstractions one of the highest-leverage bets an engineering organization can make in the next 18 months.

Technologies covered: Kubernetes GPU scheduling and resource quotas, Ray and Kubeflow for distributed ML, Service mesh (Istio) for model inference routing, ArgoCD for MLOps GitOps, Observability stacks (Prometheus, Grafana, Tempo) for ML pipeline tracing, Containerization and OCI standards

Sources aggregated from: CNCF Blog, Kubernetes.io, DevOps Weekly

📬 Stay current with cloud-native

Get the latest Kubernetes, DevOps, and platform engineering insights delivered to your inbox.

Subscribe to The Cyber SideKick Newsletter — free, no spam, unsubscribe anytime.

GhostLock (CVE-2026-43499): How a Linux Kernel Privilege Escalation Exposes Kubernetes Multi-Tenant Security Gaps

The Cyber Sidekick — Mon, 13 Jul 2026 12:44:15 +0000

A newly disclosed kernel vulnerability forces a hard reassessment of the container isolation assumptions underpinning multi-tenant Kubernetes clusters.

CVE-2026-43499, dubbed GhostLock, is a Linux kernel privilege escalation vulnerability that allows unprivileged processes to gain root-level access by exploiting flaws in kernel subsystems, effectively collapsing the security boundary between a container and its host node. Kubernetes operators running multi-tenant clusters are acutely exposed because Kubernetes-native controls like Pod Security Admission, seccomp profiles, and AppArmor all operate above the kernel layer and cannot compensate for an unpatched host kernel.

Why Kernel Privilege Escalation Hits Kubernetes Differently

Kubernetes delegates container isolation to Linux kernel primitives: namespaces partition process visibility, cgroups enforce resource boundaries, and capabilities constrain privileged operations. When a vulnerability like GhostLock allows an attacker inside a container to escalate to root on the host kernel, every one of those primitives becomes irrelevant. In a multi-tenant cluster where a single node may run hundreds of pods across different trust boundaries, a single exploitable container can become a foothold into the entire node and, from there, into cluster control plane credentials mounted via service account tokens. The Linux kernel averaged between 1,800 and 2,000 CVEs per year from 2020 to 2023 according to NVD data, with privilege escalation categories consistently representing the highest-severity subset, yet enterprise cluster upgrade cycles routinely lag behind kernel patch cadences, leaving nodes exposed for weeks or months at a time.

Where Kubernetes Hardening Falls Short Against Kernel CVEs

The deprecation of PodSecurityPolicy in Kubernetes 1.25 and its replacement with Pod Security Admission has left many teams with coarser enforcement granularity, particularly around syscall restrictions. According to the 2023 CNCF Security Report, over 60 percent of production Kubernetes clusters surveyed did not enforce seccomp profiles by default, meaning the syscall-level attack surface that GhostLock targets is fully exposed in the majority of real-world deployments. Aqua Security research further identified that container escape techniques leveraging host kernel vulnerabilities represent a significant portion of realistic attack paths, with privileged pods and hostPath mounts acting as common amplifying misconfigurations that lower the bar for exploitation. Seccomp, surfaced through Kubernetes securityContext, can reduce the exploitable syscall surface, but it requires deliberate, per-workload policy authoring that most teams have not yet operationalized at scale.

Detection, Mitigation, and Stronger Isolation Primitives

Practitioners responding to GhostLock have three complementary mitigation layers available today. First, prioritize emergency node patching or, better, replace nodes entirely using immutable image-based operating systems like Bottlerocket or Flatcar Linux, where full node replacement is faster and more automated than in-place patching, directly closing the kernel patch lag window. Second, deploy eBPF-based runtime security tooling such as the CNCF Falco project, which instruments the kernel to detect anomalous syscall patterns consistent with privilege escalation attempts, providing detection coverage while patching cycles complete. Third, for workloads with the highest trust sensitivity, adopt syscall interposition sandboxes like gVisor, which interposes on syscalls through a user-space kernel, dramatically reducing the exposed host kernel attack surface and making kernel CVEs like GhostLock largely irrelevant to sandboxed workloads. Confidential computing approaches using AMD SEV or Intel TDX provide hardware-enforced memory isolation that can further constrain what a kernel-level attacker can observe or modify across tenant boundaries.

Conclusion

GhostLock is not an anomaly; it is a predictable entry in a long series of kernel privilege escalation vulnerabilities that will continue to challenge the container isolation model Kubernetes depends on. The fundamental tension is that Kubernetes security controls are policy abstractions layered on top of a shared kernel, and no amount of policy sophistication fully compensates for an unpatched vulnerability in that shared kernel. The industry trajectory toward immutable node infrastructure, eBPF-based runtime observability, and hardware-enforced isolation through confidential computing represents the correct long-term response, moving isolation guarantees progressively closer to hardware and further from software policies that can be bypassed. In the near term, operators should treat kernel CVE patching with the same urgency as control plane upgrades, enforce seccomp profiles broadly using the RuntimeDefault baseline as a starting point, and audit clusters for privileged pods and hostPath mounts that would amplify any successful kernel exploit into a full cluster compromise.

Technologies covered: Linux kernel, Kubernetes, container security, privilege escalation, pod security policies, seccomp

Sources aggregated from: CNCF Blog, Kubernetes.io, DevOps Weekly, Hacker News, InfoQ

📬 Stay current with cloud-native

Get the latest Kubernetes, DevOps, and platform engineering insights delivered to your inbox.

Subscribe to The Cyber SideKick Newsletter — free, no spam, unsubscribe anytime.

2026 CKA Exam - Scenario 8 Install a CNI and fix the flannel pod-CIDR mismatch (CKA Services & Networking)

The Cyber Sidekick — Fri, 10 Jul 2026 13:02:27 +0000

Install a CNI and fix the pod-CIDR mismatch

The cluster is running, but every node is NotReady, because no network plugin is installed. The exam gives you a flannel manifest and asks you to install and configure a CNI. Let's apply it, watch it fail, find out why, and fix it.

🎥 Watch the video: https://www.youtube.com/watch?v=wzFE66kfRl4

This is a CKA Services & Networking walkthrough. Every command below is real output from a live cluster, and you can reproduce the whole thing yourself (scripts at the end).

The scenario

Here is the setup. A fresh cluster has no CNI, so its nodes are NotReady and CoreDNS is stuck Pending. You are handed a flannel manifest and told to install a network plugin of your choice. The catch, which you only discover after applying, is that the manifest's network does not match the cluster's pod CIDR.

A fresh cluster with NO CNI: nodes NotReady, CoreDNS Pending
You're handed a flannel manifest and told to install a CNI
The manifest's Network (10.244.0.0/16) != the cluster pod CIDR (192.168.0.0/16)
Install it, fix the mismatch, and prove pods can talk

How flannel and CNIs work

A CNI plugin is what gives pods IP addresses and wires up pod-to-pod routing; without one the kubelet reports the node NotReady. flannel runs as a DaemonSet, one pod per node, and reads its settings from a ConfigMap called kube-flannel-cfg. The important field is net-conf.json Network, the address space flannel hands out. Because flannel runs with kube-subnet-mgr, it also reads each node's assigned podCIDR, and it refuses to start if that podCIDR is not inside its configured Network. Match those two and it works.

No CNI: nodes NotReady

Start by confirming the starting state. Every node is NotReady, the classic symptom of a missing CNI. And in kube-system, CoreDNS is Pending because it has no pod network to join yet. Everything else the control plane needs is up, so this really is just the network plugin that's missing.

$ kubectl get nodes
NAME                          STATUS     ROLES           AGE   VERSION
cka-scenario8-control-plane   NotReady   control-plane   8d    v1.36.1
cka-scenario8-worker          NotReady   <none>          8d    v1.36.1

$ kubectl get pods -n kube-system -o wide
NAME                                                  READY   STATUS    RESTARTS   AGE   IP            NODE                          NOMINATED NODE   READINESS GATES
coredns-589f44dc88-8m6kd                              1/1     Running   0          8d    192.168.1.4   cka-scenario8-worker          <none>           <none>
coredns-589f44dc88-shgqb                              1/1     Running   0          8d    192.168.1.3   cka-scenario8-worker          <none>           <none>
etcd-cka-scenario8-control-plane                      1/1     Running   0          8d    172.18.0.13   cka-scenario8-control-plane   <none>           <none>
kube-apiserver-cka-scenario8-control-plane            1/1     Running   0          8d    172.18.0.13   cka-scenario8-control-plane   <none>           <none>
kube-controller-manager-cka-scenario8-control-plane   1/1     Running   0          8d    172.18.0.13   cka-scenario8-control-plane   <none>           <none>
kube-proxy-gkw9p                                      1/1     Running   0          8d    172.18.0.14   cka-scenario8-worker          <none>           <none>
kube-proxy-nxbwr                                      1/1     Running   0          8d    172.18.0.13   cka-scenario8-control-plane   <none>           <none>
kube-scheduler-cka-scenario8-control-plane            1/1     Running   0          8d    172.18.0.13   cka-scenario8-control-plane   <none>           <none>

Apply flannel (it CrashLoops)

Now install flannel by applying the manifest. It creates the kube-flannel namespace, the ConfigMap, and the DaemonSet. But when you look at the pods a few seconds later, they are not Running: they're in CrashLoopBackOff or Error. Applying the manifest was necessary, but on this cluster it is not sufficient.

$ kubectl apply -f kube-flannel.yml
namespace/kube-flannel created
serviceaccount/flannel created
clusterrole.rbac.authorization.k8s.io/flannel unchanged
clusterrolebinding.rbac.authorization.k8s.io/flannel unchanged
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds created

$ kubectl -n kube-flannel get pods -o wide
NAME                    READY   STATUS   RESTARTS     AGE   IP            NODE                          NOMINATED NODE   READINESS GATES
kube-flannel-ds-6mhjm   0/1     Error    1 (9s ago)   12s   172.18.0.13   cka-scenario8-control-plane   <none>           <none>
kube-flannel-ds-jq4n7   0/1     Error    1 (9s ago)   12s   172.18.0.14   cka-scenario8-worker          <none>           <none>

Read the lease error

Don't guess, read the logs. The flannel pod says it failed to acquire a lease, because the node's pod subnet, a slice of 192.168.0.0/16, is not inside the flannel net configuration of 10.244.0.0/16. That's the whole problem in one line: the manifest ships a default Network that does not match how this cluster was built. The cluster's pod CIDR is authoritative, so flannel is the thing that has to change.

$ kubectl -n kube-flannel logs <flannel-pod>
...
I0710 11:54:49.724901       1 kube.go:163] Node controller sync successful
I0710 11:54:49.724926       1 main.go:252] Created subnet manager: Kubernetes Subnet Manager - cka-scenario8-control-plane
I0710 11:54:49.724929       1 main.go:255] Installing signal handlers
I0710 11:54:49.725064       1 main.go:534] Found network config - Backend type: vxlan
I0710 11:54:49.726846       1 kube.go:737] List of node(cka-scenario8-control-plane) annotations: map[string]string{"flannel.alpha.coreos.com/backend-data":"{\"VNI\":1,\"VtepMAC\":\"76:f3:d3:22:ef:55\"}", "flannel.alpha.coreos.com/backend-type":"vxlan", "flannel.alpha.coreos.com/kube-subnet-manager":"true", "flannel.alpha.coreos.com/public-ip":"172.18.0.13", "node.alpha.kubernetes.io/ttl":"0", "volumes.kubernetes.io/controller-managed-attach-detach":"true"}
I0710 11:54:49.726881       1 match.go:211] Determining IP address of default interface
I0710 11:54:49.727126       1 match.go:269] Using interface with name eth0 and address 172.18.0.13
I0710 11:54:49.727151       1 match.go:291] Defaulting external address to interface address (172.18.0.13)
I0710 11:54:49.727200       1 vxlan.go:128] VXLAN config: VNI=1 Port=0 GBP=false Learning=false DirectRouting=false
I0710 11:54:49.728578       1 kube.go:704] List of node(cka-scenario8-control-plane) annotations: map[string]string{"flannel.alpha.coreos.com/backend-data":"{\"VNI\":1,\"VtepMAC\":\"76:f3:d3:22:ef:55\"}", "flannel.alpha.coreos.com/backend-type":"vxlan", "flannel.alpha.coreos.com/kube-subnet-manager":"true", "flannel.alpha.coreos.com/public-ip":"172.18.0.13", "node.alpha.kubernetes.io/ttl":"0", "volumes.kubernetes.io/controller-managed-attach-detach":"true"}
I0710 11:54:49.728602       1 vxlan.go:199] Interface flannel.1 mac address set to: 76:f3:d3:22:ef:55
E0710 11:54:49.729096       1 main.go:381] Error registering network: failed to acquire lease: subnet "10.244.0.0/16" specified in the flannel net config doesn't contain "192.168.0.0/24" PodCIDR of the "cka-scenario8-control-plane" node
I0710 11:54:49.729142       1 main.go:514] Stopping shutdownHandler...

Fix the CIDR in the ConfigMap

Fix it in the ConfigMap. kubectl edit opens the live kube-flannel-cfg object in vi. Inside net-conf.json, Network is set to 10.244.0.0/16, the manifest's default. Change it to 192.168.0.0/16 so it matches the cluster's pod CIDR, then save and quit. kubectl pushes the edit back to the API server the moment you write the file. Nothing else in the ConfigMap changes.

$ kubectl -n kube-flannel edit configmap kube-flannel-cfg

- "Network": "10.244.0.0/16",
+ "Network": "192.168.0.0/16",

Restart, then nodes go Ready

A ConfigMap change does not restart pods on its own, so the flannel pods keep crashing on the old config until you cycle them. Delete the flannel pods and the DaemonSet recreates them, this time reading the corrected Network. Now they come up Running, and within a few seconds the nodes flip to Ready and CoreDNS schedules. The CNI is installed and configured.

$ kubectl -n kube-flannel delete pod -l app=flannel
pod "kube-flannel-ds-6mhjm" deleted from kube-flannel namespace
pod "kube-flannel-ds-jq4n7" deleted from kube-flannel namespace

$ kubectl -n kube-flannel get pods -o wide
NAME                    READY   STATUS    RESTARTS   AGE   IP            NODE                          NOMINATED NODE   READINESS GATES
kube-flannel-ds-5c2pb   1/1     Running   0          3s    172.18.0.13   cka-scenario8-control-plane   <none>           <none>
kube-flannel-ds-fntnk   1/1     Running   0          3s    172.18.0.14   cka-scenario8-worker          <none>           <none>

$ kubectl get nodes
NAME                          STATUS   ROLES           AGE   VERSION
cka-scenario8-control-plane   Ready    control-plane   8d    v1.36.1
cka-scenario8-worker          Ready    <none>          8d    v1.36.1

Prove pod-to-pod connectivity

Finish by proving it actually networks, not just that the pods are green. Create two pods on different nodes and list them wide: each has an IP from 192.168.0.0/16, on a different host. Ping one from the other and the packets ride the vxlan overlay between nodes. Replies come back, so the CNI is doing its job end to end.

$ kubectl apply -f connectivity-test.yaml
pod/test1 created
pod/test2 created

$ kubectl get pods -o wide -l app=conn-test
NAME    READY   STATUS    RESTARTS   AGE   IP            NODE                          NOMINATED NODE   READINESS GATES
test1   1/1     Running   0          1s    192.168.1.8   cka-scenario8-worker          <none>           <none>
test2   1/1     Running   0          1s    192.168.0.5   cka-scenario8-control-plane   <none>           <none>

$ kubectl exec test1 -- ping -c 3 <test2-ip>
PING 192.168.0.5 (192.168.0.5): 56 data bytes
64 bytes from 192.168.0.5: seq=0 ttl=62 time=0.507 ms
64 bytes from 192.168.0.5: seq=1 ttl=62 time=0.096 ms
64 bytes from 192.168.0.5: seq=2 ttl=62 time=0.151 ms

--- 192.168.0.5 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.096/0.251/0.507 ms

Exam tips

A few traps. NotReady nodes plus Pending CoreDNS almost always means no CNI, so install one first. After you apply a CNI, always check its pods are actually Running, don't assume the apply worked. When flannel logs failed to acquire a lease, that is a pod CIDR versus Network mismatch: make flannel's Network match the cluster, since the cluster CIDR is fixed. And remember a ConfigMap edit needs a pod restart to take effect.

NotReady nodes + Pending CoreDNS => no CNI installed
After applying a CNI, verify its pods are Running (don't assume)
'failed to acquire lease' = flannel Network must match the cluster pod CIDR
A ConfigMap edit needs a pod restart to take effect

Recap

No CNI => NotReady; install flannel from the manifest
CrashLoop => logs => flannel Network must match the cluster pod CIDR
Fix net-conf.json, restart the pods, nodes go Ready
Prove it with a cross-node ping; subscribe + dev.to writeup

Reproduce this yourself

The entire scenario is scripted on a throwaway kind cluster: https://github.com/The-Cyber-Sidekick/TCS_CKA_2026_Exam_Scenarios

git clone https://github.com/The-Cyber-Sidekick/TCS_CKA_2026_Exam_Scenarios.git
cd TCS_CKA_2026_Exam_Scenarios/learning/scenarios/scenario8-cni-flannel-install
./setup.sh        # creates the cluster AND arms the scenario
# solve it by hand, or:
./solution.sh     # apply the answer key and verify

If this helped, subscribe to The Cyber SideKick on YouTube for more CKA drills, and grab the newsletter at https://thecybersidekick.beehiiv.com.

AI Agents in DevOps: Why Traditional CI/CD Pipelines Break at 1000 Deployments Per Month

The Cyber Sidekick — Wed, 08 Jul 2026 19:03:00 +0000

How LLM-driven orchestration agents are replacing rule-based pipeline automation to sustain hyperscale deployment velocities that static DSLs were never designed to handle.

At hyperscale deployment velocities, traditional CI/CD pipelines built on sequential, rule-based automation collapse under the cognitive load of thousands of concurrent deployment decisions that require real-time reasoning across telemetry signals, failure modes, and risk tolerances. A new class of autonomous deployment agents, combining LLM-based orchestration, GitOps declarative state management, and eBPF-powered observability, is emerging as the only viable architecture for platforms that must ship reliably at this scale.

The Scaling Wall That Rules-Based Pipelines Cannot Climb

Traditional CI/CD pipelines were architected for deployment velocities measured in dozens of releases per day, using sequential stage gates, hardcoded approval thresholds, and static rollback conditions encoded in Jenkinsfiles or GitHub Actions YAML. At 1000 deployments per month across heterogeneous Kubernetes clusters, these pipelines do not simply slow down; they produce compounding decision debt, where a misconfigured canary threshold written six months ago now governs a microservice that serves ten times the original traffic volume. Google's internal Borg-derived systems already handle over 4 billion container launches per week, a scale that makes the limitations of rule-based scheduling immediately visible, since no human-authored ruleset can evaluate scheduling and deployment constraints within the sub-second latency budgets those systems require. The fundamental architectural mismatch is not one of tooling performance but of decision architecture: static pipelines can execute instructions, but they cannot reason about novel failure combinations, predict cascading degradations across service meshes, or rewrite their own deployment strategies in response to real-time SLO signals.

How Agentic Orchestration Layers Replace Static Pipeline DSLs

The ecosystem is actively transitioning from imperative pipeline scripting toward agentic orchestration layers where LLMs serve as meta-controllers, dynamically composing deployment strategies by consuming Prometheus metrics, distributed traces, and changelog semantics simultaneously. Projects like Argo Rollouts are embedding AI-augmented analysis templates that ingest Datadog and Prometheus metric providers to make autonomous canary promotion decisions, eliminating the manual threshold tuning that becomes untenable across hundreds of services. Fluxcd paired with OpenAI function-calling agents enables intelligent drift detection and self-correcting GitOps reconciliation loops, where the agent can distinguish between an intentional declarative state change and an unauthorized configuration drift without requiring a human to inspect the diff. Keptn v2 Lifecycle Toolkit extends this further by providing OpenTelemetry-native evaluation hooks that AI agents consume for SLO-driven deployment gating, meaning a deployment can be autonomously promoted, paused, or rolled back based on a structured conversation between the orchestration agent and a unified observability substrate rather than a brittle shell script comparing integer thresholds. Platforms like Dagger are enabling portable, composable pipeline primitives that LLM agents can assemble on-demand, shifting engineering teams from maintaining pipeline code to expressing desired deployment outcomes and acceptable risk tolerances as declarative intent.

The Observability and Infrastructure Substrate That Makes Agents Viable

Autonomous deployment agents require a standardized signal vocabulary to reason reliably, and the maturation of OpenTelemetry as a universal observability substrate across traces, metrics, and logs is providing exactly that foundation at a moment when it is most needed. Without a consistent schema for telemetry signals, an LLM-based agent cannot reliably distinguish a latency spike caused by a flawed deployment from one caused by an upstream dependency degrading independently, making autonomous rollback decisions dangerous rather than helpful. The infrastructure layer is also evolving to meet agents where they need to operate: Kubernetes Gateway API and WASM-based extensibility now allow AI agents to manipulate traffic routing at a granularity that previously required manual SRE intervention, enabling progressive delivery patterns like weighted traffic splits and header-based routing to be adjusted dynamically as canary analysis proceeds. Kubernetes-native admission webhooks and CEL-based policy surfaces give agents a programmable enforcement plane they can update at runtime without requiring cluster restarts or human-authored policy changes. Datadog's 2024 Container Report quantifies what happens when this infrastructure is absent, finding that organizations running more than 500 Kubernetes nodes experience incident rates 3.2 times higher during deployment windows, with the average cost per major outage reaching approximately $2.3 million, a figure that makes the ROI case for AI-driven progressive delivery and automated rollback straightforward to calculate.

Conclusion

The 2023 DORA State of DevOps Report found that elite performers deploy 182 times more frequently than low performers, and analysts project that AI-assisted pipelines will push that multiplier beyond 500 times by 2026 as autonomous deployment agents eliminate manual approval bottlenecks and replace them with SLO-aware, telemetry-driven decision loops. The path forward is not incrementally smarter pipeline scripts but a wholesale architectural shift toward declarative intent expression, where engineering teams define outcomes and risk tolerances while agents handle tactical execution across multi-cluster federation topologies, availability zone-aware scheduling, and real-time traffic shaping. Organizations that begin this transition now, starting with AI-augmented canary analysis on top of existing Argo Rollouts or Flux installations, will build the operational muscle memory and telemetry hygiene needed to run fully autonomous deployment systems before the next generation of deployment velocity expectations arrives. Those that wait for the tools to mature further may find that the velocity gap between elite and average performers has grown too wide to close through iteration alone.

Technologies covered: AI agents (LLM-based orchestration), GitOps with intelligent rollback, Kubernetes native auto-scaling, Observability platforms (Datadog, Prometheus), Self-healing infrastructure

Sources aggregated from: DevOps Weekly, GitHub Trending, Hacker News, InfoQ

📬 Stay current with cloud-native

Get the latest Kubernetes, DevOps, and platform engineering insights delivered to your inbox.

Subscribe to The Cyber SideKick Newsletter — free, no spam, unsubscribe anytime.

GitOps at Scale: How Event-Driven and AI-Assisted Deployments Are Replacing Manual Environment Promotion

The Cyber Sidekick — Thu, 02 Jul 2026 19:51:44 +0000

Event-driven architectures and AI-powered validation are automating the entire GitOps promotion pipeline, eliminating the manual bottlenecks that throttle release velocity in large-scale Kubernetes environments.

Organizations managing hundreds of microservices are discovering that traditional GitOps promotion workflows, built around manual approval gates and human intervention, cannot scale to meet the demands of modern cloud-native delivery. Event-driven automation combined with ML-based quality gates is now enabling fully autonomous promotion decisions driven by real-time observability signals, policy-as-code enforcement, and historical deployment telemetry.

The Scaling Ceiling of Manual GitOps Promotion

Traditional GitOps pipelines treat environment promotion as a human-coordinated handoff: an engineer reviews test results, eyeballs dashboards, and clicks an approval button to advance a workload from staging to production. This model collapses under the weight of scale. When a platform team is responsible for hundreds of microservices across dozens of clusters, manual gates become the rate-limiting step in every release cycle. According to the 2023 DORA State of DevOps Report, elite performers deploy 127 times more frequently than low performers, and automated promotion pipelines are consistently cited as a key differentiator that keeps change failure rates below five percent. The problem is not that engineers make poor decisions; it is that the volume of decisions required in a large-scale Kubernetes environment exceeds what any team can handle reliably and quickly without automation.

Event-Driven Promotion: Wiring Observability Signals Into the GitOps Control Loop

The practical solution emerging across the CNCF ecosystem is to replace human approval gates with event-driven promotion logic that consumes signals from the observability stack in real time. Progressive delivery controllers like Argo Rollouts and Flagger connect directly to Prometheus, Datadog, and OpenTelemetry data sources, using metric-driven analysis templates to make canary and blue-green promotion decisions without waiting for a human to read a dashboard. Platform teams are routing Kubernetes events through NATS, Kafka, and CloudEvents-compliant brokers into GitOps reconcilers, so that SLO breaches, security scan failures, and load test outcomes can automatically trigger or block ArgoCD ApplicationSet promotions the moment the signal is available. Argo Rollouts alone has accumulated more than 5,800 GitHub stars and is running in production environments managing thousands of workloads, with documented case studies reporting a 60 to 70 percent reduction in deployment incidents attributable to analysis-based automated promotion. The CNCF's convergence on CloudEvents as a universal eventing substrate is accelerating interoperability between Tekton, Argo Events, Keptn, and external vendors, making it increasingly practical to compose these signals into a single, coherent promotion control plane.

AI-Augmented Quality Gates and Policy-as-Code Guardrails

Event-driven promotion handles the mechanics of signal routing, but AI and ML layers are adding a higher-order capability: deployment risk scoring based on patterns in historical telemetry that no human analyst would have the bandwidth to synthesize in real time. Tools like Keptn are integrating ML models trained on past deployment outcomes to score incoming releases and automate rollback decisions before a bad deployment can propagate to production. OpenFeature and custom admission webhooks are emerging as integration points for embedding these models directly into the Kubernetes API surface, while the Flux CD Notification Controller extends GitOps reconciliation triggers to respond to external quality signals via CloudEvents. Alongside AI scoring, policy-as-code frameworks are shifting compliance enforcement left: OPA Gatekeeper and Kyverno are now validating promotion eligibility before a Git commit is even merged, not just at deployment time, creating a continuous compliance loop across the entire software development lifecycle. Gartner projects that by 2026, more than 60 percent of organizations with mature DevOps practices will implement AI-augmented continuous delivery pipelines, up from fewer than 10 percent in 2023, driven by the economics of reducing mean time to recovery in cloud-native environments.

Conclusion

The convergence of event-driven architecture, progressive delivery controllers, and AI-augmented quality gates is fundamentally reshaping what a GitOps promotion pipeline looks like at scale. Platform engineering teams are already standardizing on Internal Developer Platforms that abstract promotion complexity behind golden paths, embedding these capabilities directly into Backstage templates and Crossplane compositions so that individual service teams inherit automated, policy-compliant promotion by default rather than by custom effort. The trajectory is clear: the approval button is being replaced by a scoring model, the Slack notification is being replaced by a CloudEvent, and the manual rollback is being replaced by an analysis-driven controller acting within seconds of a signal breach. Organizations that invest now in the observability instrumentation, eventing infrastructure, and policy-as-code discipline required to feed these systems will be positioned to treat safe, frequent, autonomous deployment not as an aspirational benchmark but as a daily operational baseline.

Technologies covered: GitOps, Event-Driven Architecture, Kubernetes, CI/CD Pipelines, Machine Learning Operations, ArgoCD, Flux CD, Policy as Code

Sources aggregated from: CNCF Blog, Kubernetes.io, DevOps Weekly

📬 Stay current with cloud-native

Get the latest Kubernetes, DevOps, and platform engineering insights delivered to your inbox.

Subscribe to The Cyber SideKick Newsletter — free, no spam, unsubscribe anytime.