DEV Community: Silpa Nallagopula

I read the Kubernetes changelogs from 1.32 to 1.37 properly

Silpa Nallagopula — Tue, 15 Sep 2026 13:04:57 +0000

I've been building an upgrade-readiness tool for about a year, and for most of that time I worked from the changelogs the way everyone does: find the version, skim "Urgent Upgrade Notes", move on. Then a cluster I care about hit something on 1.33 that I hadn't seen coming, and I decided to stop skimming.

So I cloned kubernetes/kubernetes and wrote a script that pulls out, per release, the sections that carry obligations: Urgent Upgrade Notes, Action Required, Deprecations, API Changes, Known Issues. Patch releases included, because they sneak things in. That's 36 files going back to 1.2. Then I sat down with 1.32 through 1.37 and read them. All of it. It took most of a weekend and I don't recommend it as a weekend.

The numbers
152 bullets that require someone to do something. Of those, roughly 73 I can verify by looking at the cluster: objects, CRDs, ConfigMaps, node info. The other 79 or so I can't, because they're about kubelet flags, node-level config, scrape configs, or client-side kubeconfigs. For the 1.33 → 1.34 hop specifically, 7 of the 10 urgent items are in that second bucket. I'll come back to why that matters.

Things I didn't know
The changelog is wrong about locked feature gates, at least twice. When a gate gets locked to its new default, any cluster that had it set the other way changes behaviour on upgrade. The notes mention some of these. The actual source of truth is the LockToDefault field in kube_features.go, so I diffed that between tags instead of trusting the prose. 73 gates locked across 1.32–1.35. And two dates are wrong: MaxUnavailableStatefulSet is listed under 1.35 but locks in 1.37, and StatefulSetAutoDeletePVC is listed under 1.32 but locks in 1.33. I only caught them because the source diff and the notes didn't agree. If your rules come from the release notes, you have the same two errors I had.

1.30 has no Urgent Upgrade Notes. None. I re-checked because I assumed my extractor had broken. It hadn't. There just aren't any. A tool that reports something for every hop is making it up for this one.

The scariest bullets in 1.34 sit under the least scary heading. They're under a line that literally says "(No, really, you MUST read this)": apiserver metric-label renames, the kubelet --cloud-config flag removed, static-pod admission changed. But in the rendered file, the heading above that block is just "v1.34.0". I scrolled past it the first time. So did the LLM I was testing that week, which is a story for another post.

Old changelogs are formatted differently in ways that break naive parsing. The 1.7–1.15 era puts everything under a narrative H1, and 1.7 bold-wraps its headings. Nobody cares about this unless you write the extractor, and then you care a lot.

The add-ons move when Kubernetes moves. A 1.36 target drags CoreDNS and kube-proxy along, and Istio, Karpenter, cert-manager if you run them, each with their own notes. Istio's upgradeNotes cover maybe half of what actually breaks; the rest is in the Removed/Deprecated lists and in flags whose defaults flip. Istio also drops compatibility profiles without a release note. The 1.21 profile vanished at 1.25. I only noticed because I was diffing the registration code.

What I did with it
Rules. Hand-written, one per bullet, per version, with the upstream sentence quoted verbatim and a kubectl command to verify it. About 600 across Kubernetes and seven add-ons now, plus 146 locked-gate rules generated from the source diff rather than the changelog. They run in a small Go CLI that needs a kubeconfig context and never writes anything:

kube-upgrade-check --context my-cluster --target 1.36 -o json
Every rule comes back as APPLIES with the objects named, CLEARED, or NOT DETECTABLE with the manual check spelled out. That last outcome exists because of the 79. I'd rather the tool say "I can't see your kubelet flags, go run this" than pretend it looked.

Repo: github.com/runtimez-com/kube-upgrade-check
My first pass missed 31 items I only found on the second read, so I'm fairly sure there are more. If your last upgrade broke on something that isn't in there, the changelog line in an issue is enough.

From Kubernetes PR to Deployment Verdict: Approve, Warn, or Block

Silpa Nallagopula — Mon, 03 Aug 2026 04:00:00 +0000

A small change in a Helm values file can look completely harmless during review.

The YAML is valid.

The Helm chart renders successfully.

The CI pipeline is green.

The pull request gets approved.

But after deployment, the application starts failing, pods are restarted, or availability is reduced.

The problem is that most CI checks focus on whether the Kubernetes configuration is technically valid. They do not always explain what the change will do to the workloads already running in the cluster.

Why normal CI checks are not enough

Schema validation is useful. Policy checks are useful. Linting is useful.

But passing those checks does not automatically mean that a change is safe to deploy.

A pull request may contain changes such as:

Reducing replicas from three to one
Removing a memory limit
Changing a readiness probe
Removing a PodDisruptionBudget
Using a floating image tag
Modifying a NetworkPolicy
Changing scheduling or topology constraints

Every one of these changes may be valid Kubernetes configuration.

The operational risk depends on the application, the current cluster state, the traffic pattern, and the workloads affected by the change.

A simple example

Consider this change:

replicaCount: 3

changed to:

replicaCount: 1

There is nothing syntactically wrong with the new value.

The Helm chart will render, and Kubernetes will accept the deployment.

But the operational impact could be significant.

With only one replica:

A restart may temporarily make the service unavailable
A node drain could interrupt the application
A rolling deployment may have less capacity
A single pod failure could affect all users of the service

A reviewer should not have to discover all of this by manually reading the YAML and trying to understand the possible impact.

The pull request should clearly explain that availability is being reduced.

From validation to a deployment verdict

Instead of returning only a list of configuration changes, a PR-time check should provide a clear deployment verdict.

Approve

The change does not introduce a meaningful reliability, security, cost, or availability risk.

For example, updating a harmless label or increasing a safe resource limit may be approved automatically.

Approve with warning

The change is likely safe, but the reviewer should be aware of a possible impact.

For example, increasing memory limits may increase cost, even if it does not create an immediate reliability problem.

Require approval

The change introduces meaningful operational risk and should be reviewed by the appropriate owner.

Examples may include:

Reducing replicas
Modifying resource limits
Changing probes
Removing a PodDisruptionBudget
Changing network access rules

Block

The change violates a critical reliability, security, or organizational policy.

Examples may include:

Removing required resource limits
Introducing a known critical vulnerability
Using a prohibited image tag
Breaking a mandatory availability requirement
Blocking an active service connection

What the reviewer should see

A useful PR-time verdict should explain more than whether a rule passed or failed.

The reviewer should be able to understand:

What materially changed
Why the change matters
Which workloads or services are affected
What the possible blast radius is
Whether the change requires additional approval
Whether the change can be rolled back safely
What action should be taken before deployment

The goal is not to replace engineering judgment.

The goal is to give reviewers enough context to make a better decision before the change reaches production.

The larger idea

Kubernetes deployment safety is not only a YAML-validation problem.

It is a decision problem.

Teams need to know whether a change should be approved, approved with a warning, reviewed by an owner, or blocked before it reaches the cluster.

That decision should be based on the operational effect of the change, not only whether the configuration is valid.

Originally published on the Runtimez blog:

https://runtimez.io/blog/pr-time-kubernetes-verdict