I've been building an upgrade-readiness tool for about a year, and for most of that time I worked from the changelogs the way everyone does: find the version, skim "Urgent Upgrade Notes", move on. Then a cluster I care about hit something on 1.33 that I hadn't seen coming, and I decided to stop skimming.
So I cloned kubernetes/kubernetes and wrote a script that pulls out, per release, the sections that carry obligations: Urgent Upgrade Notes, Action Required, Deprecations, API Changes, Known Issues. Patch releases included, because they sneak things in. That's 36 files going back to 1.2. Then I sat down with 1.32 through 1.37 and read them. All of it. It took most of a weekend and I don't recommend it as a weekend.
The numbers
152 bullets that require someone to do something. Of those, roughly 73 I can verify by looking at the cluster: objects, CRDs, ConfigMaps, node info. The other 79 or so I can't, because they're about kubelet flags, node-level config, scrape configs, or client-side kubeconfigs. For the 1.33 → 1.34 hop specifically, 7 of the 10 urgent items are in that second bucket. I'll come back to why that matters.
Things I didn't know
The changelog is wrong about locked feature gates, at least twice. When a gate gets locked to its new default, any cluster that had it set the other way changes behaviour on upgrade. The notes mention some of these. The actual source of truth is the LockToDefault field in kube_features.go, so I diffed that between tags instead of trusting the prose. 73 gates locked across 1.32–1.35. And two dates are wrong: MaxUnavailableStatefulSet is listed under 1.35 but locks in 1.37, and StatefulSetAutoDeletePVC is listed under 1.32 but locks in 1.33. I only caught them because the source diff and the notes didn't agree. If your rules come from the release notes, you have the same two errors I had.
1.30 has no Urgent Upgrade Notes. None. I re-checked because I assumed my extractor had broken. It hadn't. There just aren't any. A tool that reports something for every hop is making it up for this one.
The scariest bullets in 1.34 sit under the least scary heading. They're under a line that literally says "(No, really, you MUST read this)": apiserver metric-label renames, the kubelet --cloud-config flag removed, static-pod admission changed. But in the rendered file, the heading above that block is just "v1.34.0". I scrolled past it the first time. So did the LLM I was testing that week, which is a story for another post.
Old changelogs are formatted differently in ways that break naive parsing. The 1.7–1.15 era puts everything under a narrative H1, and 1.7 bold-wraps its headings. Nobody cares about this unless you write the extractor, and then you care a lot.
The add-ons move when Kubernetes moves. A 1.36 target drags CoreDNS and kube-proxy along, and Istio, Karpenter, cert-manager if you run them, each with their own notes. Istio's upgradeNotes cover maybe half of what actually breaks; the rest is in the Removed/Deprecated lists and in flags whose defaults flip. Istio also drops compatibility profiles without a release note. The 1.21 profile vanished at 1.25. I only noticed because I was diffing the registration code.
What I did with it
Rules. Hand-written, one per bullet, per version, with the upstream sentence quoted verbatim and a kubectl command to verify it. About 600 across Kubernetes and seven add-ons now, plus 146 locked-gate rules generated from the source diff rather than the changelog. They run in a small Go CLI that needs a kubeconfig context and never writes anything:
kube-upgrade-check --context my-cluster --target 1.36 -o json
Every rule comes back as APPLIES with the objects named, CLEARED, or NOT DETECTABLE with the manual check spelled out. That last outcome exists because of the 79. I'd rather the tool say "I can't see your kubelet flags, go run this" than pretend it looked.
Repo: github.com/runtimez-com/kube-upgrade-check
My first pass missed 31 items I only found on the second read, so I'm fairly sure there are more. If your last upgrade broke on something that isn't in there, the changelog line in an issue is enough.
Top comments (0)