Kubernetes Disk Expansion
So your Workload's Persistent Volume is almost full. How can you expand it? With Kubernetes 1.11 and above, this can now easily be done by just updating the Persistent Volume Claim storage specification.
My example in this post is tested in Google's Kubernetes Engine.
Let's get started!
Enabling Volume Expansion in Storage Class
To enable the disk resizing feature, ensure that the Storage Class that your Workload's Persistent Volume Claim uses is configured with allowVolumeExpansion: true.
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: standard
parameters:
type: pd-standard
provisioner: kubernetes.io/gce-pd
allowVolumeExpansion: true # ensure that this is true
A Side Note for StatefulSet Workloads
If you want to resize a disk for a StatefulSet with a configured volumeClaimTemplates like the following,
volumeClaimTemplates:
- metadata:
name: postgresql-storage
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
storageClassName: standard
you will only have to modify the Persistent Volume Claim that it has provisioned.
Resizing a Persistent Volume Claim
Find the Persistent Volume Claim to resize.
kubectl get pvc -l app=postgresql
Let's say you want to update postgresql-storage-postgresql-02-1 from 1Gi to 10Gi
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
postgresql-storage-postgresql-01-0 Bound pvc-613c74c3-66b5-11e9-a602-42010a8400c0 10Gi RWO standard 6m12s
postgresql-storage-postgresql-01-1 Bound pvc-88f897b1-60d4-11e9-a602-42010a8400c0 5Gi RWO standard 7d11h
postgresql-storage-postgresql-02-0 Bound pvc-727c167c-6089-11e9-a602-42010a8400c0 10Gi RWO standard 7d20h
postgresql-storage-postgresql-02-1 Bound pvc-80421dab-6089-11e9-a602-42010a8400c0 1Gi RWO standard 7d20h
kubectl edit pvc postgresql-storage-postgresql-02-1
From your text editor, change 1Gi to 10Gi
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
annotations:
pv.kubernetes.io/bind-completed: "yes"
pv.kubernetes.io/bound-by-controller: "yes"
volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/gce-pd
creationTimestamp: "2019-04-16T20:52:09Z"
finalizers:
- kubernetes.io/pvc-protection
labels:
app: postgresql
name: postgresql-storage-postgresql-02-1
namespace: default
resourceVersion: "29892621"
selfLink: /api/v1/namespaces/default/persistentvolumeclaims/postgresql-storage-postgresql-02-1
uid: 80421dab-6089-11e9-a602-42010a8400c0
spec:
accessModes:
- ReadWriteOnce
dataSource: null
resources:
requests:
storage: 1Gi # change this to 10Gi
storageClassName: standard
volumeName: pvc-80421dab-6089-11e9-a602-42010a8400c0
status:
accessModes:
- ReadWriteOnce
capacity:
storage: 1Gi # change this to 10Gi
phase: Bound
After successfully modifying the Persistent Volume Claim, observe its status.
kubectl get pvc postgresql-storage-postgresql-02-1 -o yaml
This should be the example output. Observe the status at the very end of the YAML.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
annotations:
pv.kubernetes.io/bind-completed: "yes"
pv.kubernetes.io/bound-by-controller: "yes"
volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/gce-pd
creationTimestamp: "2019-04-16T20:52:09Z"
finalizers:
- kubernetes.io/pvc-protection
labels:
app: postgresql
name: postgresql-storage-postgresql-02-1
namespace: default
resourceVersion: "33540657"
selfLink: /api/v1/namespaces/default/persistentvolumeclaims/postgresql-storage-postgresql-02-1
uid: 80421dab-6089-11e9-a602-42010a8400c0
spec:
accessModes:
- ReadWriteOnce
dataSource: null
resources:
requests:
storage: 10Gi
storageClassName: standard
volumeName: pvc-80421dab-6089-11e9-a602-42010a8400c0
status:
accessModes:
- ReadWriteOnce
capacity:
storage: 1Gi
conditions:
- lastProbeTime: null
lastTransitionTime: "2019-04-24T17:30:51Z"
message: Waiting for user to (re-)start a pod to finish file system resize of
volume on node.
status: "True"
type: FileSystemResizePending
phase: Bound
Restart the Pod
The next step is to restart the Pod as recommended by Kubernetes.
kubectl delete pod postgresql-02-1
Then wait until the Pod starts running again.
Observe the Changes
After a successful Pod restart, check the status of the Persistent Volume Claim again and you should see that the storage has been updated.
kubectl get pvc postgresql-storage-postgresql-02-1 -o yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
annotations:
pv.kubernetes.io/bind-completed: "yes"
pv.kubernetes.io/bound-by-controller: "yes"
volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/gce-pd
creationTimestamp: "2019-04-16T20:52:09Z"
finalizers:
- kubernetes.io/pvc-protection
labels:
app: postgresql
name: postgresql-storage-postgresql-02-1
namespace: default
resourceVersion: "33541444"
selfLink: /api/v1/namespaces/default/persistentvolumeclaims/postgresql-storage-postgresql-02-1
uid: 80421dab-6089-11e9-a602-42010a8400c0
spec:
accessModes:
- ReadWriteOnce
dataSource: null
resources:
requests:
storage: 10Gi
storageClassName: standard
volumeName: pvc-80421dab-6089-11e9-a602-42010a8400c0
status:
accessModes:
- ReadWriteOnce
capacity:
storage: 10Gi
phase: Bound
Do kubectl exec to the Pod and verify that the disk has been resized!
Run df -h.
# df -h
Filesystem Size Used Avail Use% Mounted on
overlay 5.7G 4.7G 1.1G 83% /
tmpfs 1.9G 0 1.9G 0% /dev
tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
/dev/sda1 5.7G 4.7G 1.1G 83% /etc/hosts
shm 64M 0 64M 0% /dev/shm
/dev/sdc 9.9G 1.3G 8.6G 13% /var/lib/postgresql
tmpfs 1.9G 12K 1.9G 1% /run/secrets/kubernetes.io/serviceaccount
tmpfs 1.9G 0 1.9G 0% /sys/firmware
Magic!
Top comments (0)