I encountered one issue after performing the above on a cluster; The below resolved it for me, hope it comes in handy for others!
Environment (context):
Host nodes: Ubuntu 21.04 (amd64 arch) - but imagine much the same for any cluster
Moving from: k8s v1.21.5
Moving to: k8s v1.22.3
Cluster installed with: kubeadm
Problem:
Upgrading the k8s version with kubeadm failed as follows:
k8s-cp-node:~# kubeadm upgrade plan
k8s-cp-node:~# kubeadm upgrade apply vX.X.XX Error:
"docker is required for container runtime: exec: "docker": executable file not found in $PATH"
Cause:
As docker runtime was uninstalled, it was no longer present on nodes of course. This combined with a lingering ANNOTATION (applied by kubeadm on initial cluster install) was still pointing to dockershim's unix socket on each node, which was blocking the k8s version upgrade:
Fix:
I moved over to using containerd as my container runtime.
Check the location of your new container runtime unix socket (runtime endpoint) before changing anything. Mine was as exactly as per Laurent's main article above.
Update your "cri-socket" node annotations (for ALL your nodes) before you upgrade k8s version. Run for each of your nodes, replacing with the actual name of your node. This worked for me:
$ kubectl annotate node --overwrite kubeadm.alpha.kubernetes.io/cri-socket=/var/run/containerd/containerd.sock
You can check the annotation(s) after changing them:
$ kubectl describe node | grep Annotations -A5
Proceed with your k8s version cluster upgrade as per normal. You should no longer get complaints of missing docker problems...
Other Notes
I also had a few static control plane pods (api-server etc) getting stuck "Pending" during the upgrade and had to nuke them as follows, after which all was good...
$ kubectl -n kube-system delete pod kube-apiserver- --force --grace-period 0
(replace with your actual pod name in the command of course)
Missing static control plane pods will automatically be re-created by the node when it sees they are missing.
Happy k8sing all!
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Very useful and concise thank you!
I encountered one issue after performing the above on a cluster; The below resolved it for me, hope it comes in handy for others!
Environment (context):
Host nodes: Ubuntu 21.04 (amd64 arch) - but imagine much the same for any cluster
Moving from: k8s v1.21.5
Moving to: k8s v1.22.3
Cluster installed with: kubeadm
Problem:
Upgrading the k8s version with kubeadm failed as follows:
k8s-cp-node:~# kubeadm upgrade plan
k8s-cp-node:~# kubeadm upgrade apply vX.X.XX
Error:
"docker is required for container runtime: exec: "docker": executable file not found in $PATH"
Cause:
As docker runtime was uninstalled, it was no longer present on nodes of course. This combined with a lingering ANNOTATION (applied by kubeadm on initial cluster install) was still pointing to dockershim's unix socket on each node, which was blocking the k8s version upgrade:
Check your nodes cri-socket annotation with:
$ kubectl describe node | grep Annotations -A5
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
...
Fix:
I moved over to using containerd as my container runtime.
Check the location of your new container runtime unix socket (runtime endpoint) before changing anything. Mine was as exactly as per Laurent's main article above.
Update your "cri-socket" node annotations (for ALL your nodes) before you upgrade k8s version. Run for each of your nodes, replacing with the actual name of your node. This worked for me:
$ kubectl annotate node --overwrite kubeadm.alpha.kubernetes.io/cri-socket=/var/run/containerd/containerd.sock
You can check the annotation(s) after changing them:
$ kubectl describe node | grep Annotations -A5
Proceed with your k8s version cluster upgrade as per normal. You should no longer get complaints of missing docker problems...
Other Notes
I also had a few static control plane pods (api-server etc) getting stuck "Pending" during the upgrade and had to nuke them as follows, after which all was good...
$ kubectl -n kube-system delete pod kube-apiserver- --force --grace-period 0
(replace with your actual pod name in the command of course)
Missing static control plane pods will automatically be re-created by the node when it sees they are missing.
Happy k8sing all!