DEV Community

The Cyber Sidekick
The Cyber Sidekick

Posted on

Add a streaming sidecar so kubectl logs can see a legacy app (CKA)

#kubernetes #cka #logging #devops

Add a streaming sidecar so kubectl logs can see a legacy app (CKA)

Here is a pod that is perfectly healthy, doing real work, and completely silent as far as Kubernetes is concerned. kubectl logs returns nothing at all. Today's CKA task is to fix that without rewriting the application: add a second container alongside it that streams the app's log file to standard output, and share a volume between the two so it can actually read that file. This is the streaming sidecar pattern, and it is one of the most quietly useful things on the exam. Let's run it on a live cluster.

🎥 Watch the video: https://www.youtube.com/watch?v=JDQgrcbmFH0

This is a CKA Workloads & Scheduling walkthrough. Every command below is real output from a live cluster, and you can reproduce the whole thing yourself (scripts at the end).

The scenario

Here is the setup. A legacy application has been packaged into a Deployment called report-engine, in the legacy namespace. It writes everything it does to a log file inside its container and prints nothing to standard output, so it is invisible to the cluster's logging architecture. Three things are asked of you. Add a second, co-located container to the same pod, named log-stream, using the busybox image, whose command streams that log file to standard output. Give both containers a shared volume so the file is visible to the new container. And change nothing else about the container that is already there.

  • Deployment 'report-engine' (ns 'legacy') logs to a FILE, not stdout
  • 1. Add a co-located container 'log-stream' (busybox) that streams that file
  • 2. Share a volume between both containers so the file is visible
  • 3. Change nothing else about the existing 'app' container

How pod logging actually works

One fact explains this whole question. Kubernetes has a single logging path: the kubelet captures what a container writes to standard output and standard error, and kubectl logs replays that. It does not know about files, it does not look inside your container's filesystem, and there is no configuration to point it at one. So an app that appends to a log file is not badly configured, it is simply outside the pipeline. The documented fix is a streaming sidecar: put a second container in the same pod whose only job is to tail that file and echo it to its own standard output, where the pipeline picks it up. That works because containers in a pod can share volumes. Mount one volume, an emptyDir is plenty, into both containers at the directory holding the log file, and both processes see the same file.

The empty kubectl logs

Let's see the problem before fixing it. The Deployment is running, one pod, READY one of one, no restarts. Nothing is broken. Now ask for its logs, and look carefully: there is no output. Not an error, not a warning, just nothing, which is the most confusing failure mode there is. The app is definitely producing output though. Exec into the container and tail the file it writes, and there are the log lines, timestamped, several of them, sitting in the container's own filesystem where kubectl cannot reach them.

$ kubectl get deployments,pods -n legacy
NAME                            READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/report-engine   1/1     1            1           8s

NAME                                 READY   STATUS    RESTARTS   AGE
pod/report-engine-7d75b577d9-cs5nz   1/1     Running   0          8s

$ kubectl -n legacy logs deploy/report-engine
(no output at all)

$ kubectl -n legacy exec deploy/report-engine -- tail -n 3 /var/log/report-engine.log
2026-07-31 10:58:03 report-engine: batch complete
2026-07-31 10:58:08 report-engine: batch complete
Enter fullscreen mode Exit fullscreen mode

Add the sidecar

Now the edit. In the exam you would run kubectl edit on the Deployment, or dump it to a file, change it, and re-apply; both are fine. Three things go in. A volumeMount on the existing container, pointing at the directory that holds the log file. A second container, named log-stream, image busybox, whose command tails that same file, plus the identical volumeMount. And the volume itself, an emptyDir, which sits at the pod spec level next to containers, not inside them. Watch the indentation there, because that is where this question is usually lost. Notice what did not change: the app container keeps its image and its command exactly as they were.

$ vi report-engine.yaml
...
        app: report-engine
    spec:
      containers:
        - name: app
          image: busybox:1.36
          command:
            - sh
            - -c
            - >-
              mkdir -p /var/log;
              while true; do
              echo "$(date '+%F %T') report-engine: batch complete" >> /var/log/report-engine.log;
              sleep 5; done

$ (after editing)
...
              sleep 5; done
          volumeMounts:
            - name: logs
              mountPath: /var/log
        - name: log-stream
          image: busybox:1.36
          command: ["sh", "-c", "tail -n+1 -F /var/log/report-engine.log"]
          volumeMounts:
            - name: logs
              mountPath: /var/log
      volumes:
        - name: logs
          emptyDir: {}
Enter fullscreen mode Exit fullscreen mode

Rollout to 2/2

Apply it. Changing the pod template starts a rolling update, so the Deployment builds a replacement pod with both containers and retires the old one. Wait for rollout status to report success, and then look at the pod list. The READY column now says two of two: two containers in a single pod, both running, sharing a network namespace and, more importantly for us, sharing a volume. One pod, not two, and that distinction is the whole point of a sidecar.

$ kubectl apply -f report-engine.yaml
deployment.apps/report-engine configured

$ kubectl -n legacy rollout status deployment/report-engine
Waiting for deployment "report-engine" rollout to finish: 0 out of 1 new replicas have been updated...
Waiting for deployment "report-engine" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "report-engine" rollout to finish: 1 old replicas are pending termination...
deployment "report-engine" successfully rolled out

$ kubectl get pods -n legacy
NAME                             READY   STATUS    RESTARTS   AGE
report-engine-764cb77fd9-bgmqs   2/2     Running   0          33s
Enter fullscreen mode Exit fullscreen mode

The payoff

And here is the payoff. Ask for the logs again, this time naming the container with -c log-stream, and the app's log file comes streaming out of kubectl. Same lines, same timestamps we saw hiding in that file a minute ago, except now they are flowing through the cluster's normal logging path, which means anything that collects logs from this cluster picks them up too. The application was never touched. Note the -c: once a pod has more than one container, kubectl needs to know which one you mean.

$ kubectl -n legacy logs deploy/report-engine -c log-stream --tail=4
2026-07-31 10:58:31 report-engine: batch complete
2026-07-31 10:58:36 report-engine: batch complete
2026-07-31 10:58:41 report-engine: batch complete
2026-07-31 10:58:46 report-engine: batch complete
Enter fullscreen mode Exit fullscreen mode

One volume, two containers

It is worth proving that this really is one shared volume and not a coincidence. Exec into the sidecar, the second container, and list the log file: it is right there, owned by root, growing, even though a completely different container is the one writing to it. Then read the mounts back from the Deployment. Both containers mount the same volume, named logs, at the same path. That is the mechanism in two lines of output: same volume, same mount path, two processes, one file.

$ kubectl -n legacy exec deploy/report-engine -c log-stream -- ls -l /var/log/report-engine.log
-rw-r--r--    1 root     root           400 Jul 31 10:58 /var/log/report-engine.log

$ kubectl -n legacy get deployment report-engine -o jsonpath='{range .spec.template.spec.containers[*]}{.name}{" mounts "}{.volumeMounts[0].name}{" at "}{.volumeMounts[0].mountPath}{"\n"}{end}'
app mounts logs at /var/log
log-stream mounts logs at /var/log
Enter fullscreen mode Exit fullscreen mode

Nothing else moved

Last check, and it is the one part three grades. Read the original container back: same name, same image, same command it started with. The only thing that changed about it is the volume mount it needed to share the file, which the task explicitly allows. And the volume itself is a plain emptyDir, created with the pod and thrown away with it, which is exactly right for a log relay. No PersistentVolumeClaim, no storage class, nothing to provision.

$ kubectl -n legacy get deployment report-engine -o jsonpath='{.spec.template.spec.containers[0].name}{" / "}{.spec.template.spec.containers[0].image}{"\n"}{.spec.template.spec.containers[0].command[2]}'
app / busybox:1.36
mkdir -p /var/log; while true; do echo "$(date '+%F %T') report-engine: batch complete" >> /var/log/report-engine.log; sleep 5; done

$ kubectl -n legacy get deployment report-engine -o jsonpath='{.spec.template.spec.volumes}'
[{"emptyDir":{},"name":"logs"}]
Enter fullscreen mode Exit fullscreen mode

Exam tips

Things to carry into the exam. The volume mount goes in every container that needs the file, and there are two of them; forgetting the app container is the classic mistake, because then the sidecar tails a file that will never appear. The volume itself lives at the pod spec level, beside containers, not inside one. An empty emptyDir, written as `emptyDir: {}, is all you need; search the docs page foremptyDirand copy the shape. Mount at the directory, not the file. Once the pod has two containers,kubectl logsand kubectl exec both need-cto say which one. And one modern footnote: Kubernetes does have native sidecars now, an init container with arestartPolicy: Always`, but a question that says co-located container in the same pod means a plain second entry under containers, so do the simple thing.

  • volumeMounts go in BOTH containers; the app one is the one people forget
  • The volume itself sits at pod-spec level, beside 'containers', not inside it
  • 'emptyDir: {}' is enough; it is born and dies with the pod
  • Mount the DIRECTORY that holds the log file, not the file
  • Two containers means kubectl logs / exec need -c
  • Native sidecars (initContainer + restartPolicy: Always) exist, but this asks for a plain container

Recap

  • kubectl logs reads stdout only, so a file-logging app is invisible
  • Added 'log-stream' (busybox) tailing the file to stdout, in the SAME pod
  • emptyDir 'logs' mounted at /var/log in both containers: one file, two processes
  • READY 2/2, logs streaming, original container untouched; subscribe + dev.to writeup

Reproduce this yourself

The entire scenario is scripted on a throwaway kind cluster: https://github.com/The-Cyber-Sidekick/TCS_CKA_2026_Exam_Scenarios

`bash
git clone https://github.com/The-Cyber-Sidekick/TCS_CKA_2026_Exam_Scenarios.git
cd TCS_CKA_2026_Exam_Scenarios/learning/scenarios/scenario14-sidecar-logging
./setup.sh # creates the cluster AND arms the scenario

solve it by hand, or:

./solution.sh # apply the answer key and verify
`

If this helped, subscribe to The Cyber SideKick on YouTube for more CKA drills, and grab the newsletter at https://thecybersidekick.beehiiv.com.

Top comments (0)