I once watched a candidate with "3 years Kubernetes" on his resume describe a Deployment and a ReplicaSet as basically interchangeable terms for "the thing that keeps my pods running." Technically not wrong, exactly — a Deployment does manage a ReplicaSet under the hood — but the follow-up question is the one that actually separates people: "if you edit a ReplicaSet directly instead of the Deployment, what happens on the next rollout?" He didn't know, because he'd never had a reason to touch anything below the Deployment layer, which is fine until the cluster does something unexpected and you're the one debugging it at 11pm. Here are nine questions that come up constantly, and what they're actually testing for.
1. Deployment, ReplicaSet, Pod — walk me through the hierarchy, and what breaks if you edit a ReplicaSet directly?
A Deployment owns one or more ReplicaSets (one active, plus old ones kept around for rollback), and each ReplicaSet owns a set of Pods matching its selector. Edit a ReplicaSet's spec directly and the change works — until the Deployment controller reconciles state on its next pass and overwrites your edit back to whatever the Deployment's spec says, because the Deployment is the source of truth, not the ReplicaSet. I ask this because it's the fastest way to tell if someone's ever actually debugged a "why did my manual fix disappear" incident, which is a rite of passage for anyone who's spent real time in a cluster.
2. Liveness, readiness, and startup probes — what failure mode does each one actually protect against?
Liveness probes answer "is this container stuck and needs a restart" — fail it enough times and kubelet kills and restarts the container. Readiness probes answer "should this pod currently receive traffic" — fail it and the pod gets pulled from the Service's endpoints without being restarted, which matters enormously during startup or a slow dependency check. Startup probes exist specifically for slow-booting apps, disabling liveness/readiness checks until the app reports it's actually initialized, so a legitimately slow Spring Boot app doesn't get killed in a restart loop before it ever finishes starting. The mistake I see constantly: teams configure only a liveness probe, pointed at the same endpoint as readiness, and then wonder why a pod that's merely slow to reconnect to its database gets killed and restarted instead of just quietly pulled from rotation until it recovers.
3. What actually happens to a request when you kubectl scale a Deployment down, mid-traffic?
The Deployment controller deletes the excess pods, but there's a window most people don't account for: the pod gets a SIGTERM, and unless your app handles graceful shutdown, in-flight requests get dropped mid-response while the pod is still technically in the Service's endpoint list for a few hundred milliseconds to a couple seconds, depending on kube-proxy's sync interval and how fast your app stops accepting new connections. preStop hooks and a reasonable terminationGracePeriodSeconds exist for exactly this — sleep for a couple seconds in preStop so kube-proxy has time to remove the pod from rotation before the app actually stops accepting connections. I've seen production incidents traced entirely to a missing preStop hook during routine autoscaling, not even a deploy — just normal scale-down dropping a percentage of requests nobody had budgeted for.
4. Resource requests vs. limits — what's the actual failure mode when you get each one wrong?
Requests are what the scheduler uses to decide which node has room for your pod — set them too low and you get overcommitted nodes where pods fight for actual CPU/memory at runtime. Limits are hard ceilings enforced by the kubelet — hit a memory limit and the container gets OOMKilled outright; hit a CPU limit and the container gets throttled (CFS throttling), which doesn't kill anything but silently makes your app slower in a way that's brutal to diagnose because nothing crashes, nothing logs an error, it just... gets slow under load. The wrong answer treats requests and limits as roughly the same knob turned to different values. The real signal is knowing that a memory limit without a request just means your pod schedules wherever, potentially crowds out other workloads, and then gets killed anyway once it actually hits that ceiling.
5. How does traffic from a Service actually reach a Pod? Trace it end to end.
A Service gets a stable virtual IP and DNS name; kube-proxy watches the Service and its matching Endpoints (or EndpointSlices on newer clusters) and programs iptables or IPVS rules on every node to load-balance traffic across the healthy pod IPs behind it. There's no single "load balancer process" routing packets in the way people picture it — it's distributed rule-programming on every node, which is also why a stale kube-proxy or a slow endpoint sync can mean traffic keeps hitting a pod that was just marked not-ready, for a brief but real window. A candidate who says "the Service load-balances to the pods" isn't wrong, but someone who can explain that it's iptables/IPVS rules being reconciled from EndpointSlices, not a live proxy process making per-request decisions, has clearly had to debug a networking issue that "should have just worked."
6. ConfigMap vs. Secret — what's the actual security difference, and where does the common assumption go wrong?
Both store key-value data and both get mounted the same way, as environment variables or files — that similarity is exactly what causes the mistake. Secrets are base64-encoded, not encrypted, by default; base64 is trivially reversible, and without encryption-at-rest enabled on etcd (which is a separate, explicit cluster configuration, not automatic), a Secret sitting in etcd is functionally as exposed as a ConfigMap to anyone with etcd access. I've interviewed people who confidently said "Secrets are encrypted, ConfigMaps aren't" as if that's a default guarantee, and it just isn't unless someone configured encryption providers on the API server. RBAC restricting who can read Secret objects is doing more real protection than the base64 encoding is.
7. Rolling update strategy — what do maxSurge and maxUnavailable actually control, and what breaks if you get them wrong?
maxUnavailable caps how many pods from the old ReplicaSet can be down at once during the rollout; maxSurge caps how many extra pods above the desired count can exist temporarily while new ones spin up alongside old ones. Set maxUnavailable too high (or maxSurge to 0 with maxUnavailable high) on a service without enough replica headroom, and a rolling update can genuinely drop capacity below what your traffic needs mid-rollout — not a bug, just math nobody checked. The follow-up I actually care about: what happens if your new pods pass readiness checks but are still functionally broken — bad config, wrong image tag pointed at a broken build. The rollout completes successfully by Kubernetes' definition of success, because readiness passed, while your actual service is degraded. That gap between "rollout succeeded" and "service is healthy" is why real teams pair rollouts with actual traffic-based monitoring, not just kubectl rollout status.
8. Horizontal Pod Autoscaler — what's the actual mechanism, and why does it sometimes thrash?
HPA polls the metrics API (metrics-server by default, or a custom/external metrics adapter for anything beyond CPU/memory) on a fixed interval, compares current usage against the target you set, and computes a desired replica count from that ratio — it's not instant, and it's not smart about trends, just a periodic proportional calculation. Thrashing happens when a workload's actual load oscillates faster than the HPA's stabilization window accounts for — scale up, load drops because you just added capacity, scale back down, load spikes again from the reduced capacity, repeat. The fix most people don't know by name: behavior.scaleDown.stabilizationWindowSeconds, which makes the HPA wait and look at the max recommendation over a rolling window before actually scaling down, specifically to dampen this exact oscillation.
9. StatefulSet vs. Deployment — when do you genuinely need one, and what's the tell that someone's using it out of habit rather than necessity?
StatefulSets give you stable, predictable pod names and persistent volume claims that survive pod rescheduling — pod-0 is always pod-0, with the same storage, even after a restart. That matters for genuinely stateful workloads: databases, anything doing leader election by ordinal identity, Kafka brokers. The tell I look for: someone reaching for a StatefulSet for a stateless API "just to be safe," which buys you nothing but slower, ordered rollouts (StatefulSets update pods one at a time by default, not in parallel) and stable network identity you don't need. If your pods don't care about their own name or need their own dedicated persistent volume, a Deployment is simpler and rolls out faster, full stop.
None of these are hard once you've actually run a cluster past the tutorial stage and had something quietly break in a way kubectl get pods didn't explain. They're hard live, reasoning through it out loud while someone's watching for whether you actually understand the mechanism or just memorized the vocabulary. I've been running through scenario questions like these with LastRound AI's AI Mock Interview before Kubernetes-heavy interviews — saying "the rollout succeeded doesn't mean the service is healthy, those are different signals" out loud, under a bit of pressure, is a different skill than nodding along when someone else says it: https://lastroundai.com/products/mock-interviews
Top comments (0)