Kubernetes Delete Stuck Deployment

Use case is a Kubernetes deployment in a namespace, but you want to keep part of the deployment such as Persistent Volume Claims (PVC) and Secrets, but you want to remove everything else.

In this example, my namespace is named "ns".

Normally, you can do this with:

terraform destroy -auto-approve

However, there are cases where the deployment gets stuck often when you are in a CrashBackoffLoop.

In these cases, deleting your namespace is the shortest path.

But this will not work if you have a situation like mine with PVC and so on as they live in our namespace.

DO NOT DO THIS IF YOU WANT TO KEEP PVC AND SECRETS:

kubectl delete namespace ns

Instead, I wrote a script which will delete all the things I want to remove but keep pvc and secrets. Feel free from the script other things you want to keep:

#!/usr/bin/bash

# This script is intended to take down everything in otel except
# the Persistent Storage

NS=$1

if [[ -z "$1" ]]; then
    echo "Warning this will take down everything in the given namespace"
    echo ""
    echo "PLEASE ONLY USES THIS IF YOU KNOW WHAT YOU ARE DOING"
    echo ""
    echo "Usage: $0 namespace_to_delete"
    exit 1
fi


kubectl scale deployment --all -n $NS --replicas=0
kubectl scale statefulset --all -n $NS --replicas=0
kubectl delete configmap --all -n $NS
kubectl delete daemonset --all -n $NS
kubectl delete statefulset --all -n $NS
kubectl delete serviceaccount --all -n $NS
kubectl delete ingress --all -n $NS
kubectl delete pod --all -n $NS
kubectl delete service --all -n $NS
# helm uninstall helm_service -n $NS

Save this script as manual-delete-deployment.sh and run it:

bash manual-deplete-deployment.sh ns

Be sure to replace the helm_service with any helm services you have as they will often stay stuck as well.

You can find them with

helm list -n ns

You can check to ensure that your deployment is gone:

kubectl get all -n ns

Happy coding!