DEV Community

Constantine Ukah
Constantine Ukah

Posted on

Kubernetes Secret Store CSI Driver with Azure Vault.

You have your pod in the Azure Kubernetes Cluster (AKS) that should retrieve a secret in the Key Vault (NB: You must have created an instance within and uploaded the necessary secrets).

To allow the pod communication to an external secret managed solution, you must create -

  1. Secret Store CSI Driver
  2. Installation of the relevant provider that the driver has to talk to such as Hashicorp, Azure Key Vault etc.
  3. Integrate the AKS to the managed identity to allow connection to the azure key vault.
  4. To allow the specific pod to reach the key vault, a Service account for the pod has to be created and integrated with the managed identity.

Steps to setup the cluster and all needed requirements using CLI.

  • Create Azure Resource Group 
az group create --name keyvault-demo --location eastus
Enter fullscreen mode Exit fullscreen mode
  • Create an AKS cluster with Azure Key Vault provider for Secrets Store CSI Driver support 
az aks create --name keyvault-demo-cluster -g keyvault-demo --node-count 1 --enable-addons azure-keyvault-secrets-provider --enable-oidc-issuer --enable-workload-identity
Enter fullscreen mode Exit fullscreen mode
  • Get the Kubernetes cluster credentials (Update kubeconfig) 
az aks get-credentials --resource-group keyvault-demo --name keyvault-demo-cluster
Enter fullscreen mode Exit fullscreen mode
  • Verify that each node in your cluster's node pool has a Secrets Store CSI Driver pod and a Secrets Store Provider Azure pod running 
kubectl get pods -n kube-system -l 'app in (secrets-store-csi-driver,secrets-store-provider-azure)' -o wide
Enter fullscreen mode Exit fullscreen mode

Key vault creation and configuration

  • Create a key vault with Azure role-based access control (Azure RBAC). 

az keyvault create -n aks-demo-<_name of your choice_> -g keyvault-demo -l eastus --enable-rbac-authorization
Enter fullscreen mode Exit fullscreen mode

NB: You may run into the below issue when using the above command for the first time.

.

Simply, check the registration state of your provide using the command below -

az provider show --namespace Microsoft.KeyVault --query registrationState
Enter fullscreen mode Exit fullscreen mode

If it returns anything other than Registered, register it using the command -

az provider register --namespace Microsoft.KeyVault
Enter fullscreen mode Exit fullscreen mode

Monitor the progress via

az provider show --namespace Microsoft.KeyVault --query registrationState
Enter fullscreen mode Exit fullscreen mode

Wait until it shows: Registered

Then re-run the command-

az keyvault create -n aks-demo-<_name of your choice_> -g keyvault-demo -l eastus --enable-rbac-authorization
Enter fullscreen mode Exit fullscreen mode

Connect your Azure ID to the Azure Key Vault Secrets Store CSI Driver.

  • Configure workload identity: 
export SUBSCRIPTION_ID=<your own subscription ID>
export RESOURCE_GROUP=keyvault-demo
export UAMI=<the name of your managed identity>
export KEYVAULT_NAME=<your keyvault name>
export CLUSTER_NAME=keyvault-demo-cluster

az account set --subscription $SUBSCRIPTION_ID
Enter fullscreen mode Exit fullscreen mode

  • Create a managed identity: 
az identity create --name $UAMI --resource-group $RESOURCE_GROUP

export USER_ASSIGNED_CLIENT_ID="$(az identity show -g $RESOURCE_GROUP --name $UAMI --query 'clientId' -o tsv)"
export IDENTITY_TENANT=$(az aks show --name $CLUSTER_NAME --resource-group $RESOURCE_GROUP --query identity.tenantId -o tsv)
Enter fullscreen mode Exit fullscreen mode


  • Create a role assignment that grants the workload ID access the key vault. 
export KEYVAULT_SCOPE=$(az keyvault show --name $KEYVAULT_NAME --query id -o tsv)

az role assignment create --role "Key Vault Administrator" --assignee $USER_ASSIGNED_CLIENT_ID --scope $KEYVAULT_SCOPE
Enter fullscreen mode Exit fullscreen mode


  • Get the AKS cluster OIDC Issuer URL This is necessary because to connect your service account of your kubernetes pod to the managed identity. 
export AKS_OIDC_ISSUER="$(az aks show --resource-group $RESOURCE_GROUP --name $CLUSTER_NAME --query "oidcIssuerProfile.issuerUrl" -o tsv)"
echo $AKS_OIDC_ISSUER
Enter fullscreen mode Exit fullscreen mode

  • Create the service account for the pod 
export SERVICE_ACCOUNT_NAME="workload-identity-sa"
export SERVICE_ACCOUNT_NAMESPACE="default"
Enter fullscreen mode Exit fullscreen mode 
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    azure.workload.identity/client-id: ${USER_ASSIGNED_CLIENT_ID}
  name: ${SERVICE_ACCOUNT_NAME}
  namespace: ${SERVICE_ACCOUNT_NAMESPACE}
EOF
Enter fullscreen mode Exit fullscreen mode

  • Setup Federation 
export FEDERATED_IDENTITY_NAME="aksfederatedidentity" 

az identity federated-credential create --name $FEDERATED_IDENTITY_NAME --identity-name $UAMI --resource-group $RESOURCE_GROUP --issuer ${AKS_OIDC_ISSUER} --subject system:serviceaccount:${SERVICE_ACCOUNT_NAMESPACE}:${SERVICE_ACCOUNT_NAME}
Enter fullscreen mode Exit fullscreen mode

  • Create the Secret Provider Class 
cat <<EOF | kubectl apply -f -
# This is a SecretProviderClass example using workload identity to access your key vault
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: azure-kvname-wi # needs to be unique per namespace
spec:
  provider: azure
  parameters:
    usePodIdentity: "false"
    clientID: "${USER_ASSIGNED_CLIENT_ID}" # Setting this to use workload identity
    keyvaultName: ${KEYVAULT_NAME}       # Set to the name of your key vault
    cloudName: ""                         # [OPTIONAL for Azure] if not provided, the Azure environment defaults to AzurePublicCloud
    objects:  |
      array:
        - |
          objectName: secret1             # Set to the name of your secret
          objectType: secret              # object types: secret, key, or cert
          objectVersion: ""               # [OPTIONAL] object versions, default to latest if empty
        - |
          objectName: key1                # Set to the name of your key
          objectType: key
          objectVersion: ""
    tenantId: "${IDENTITY_TENANT}"        # The tenant ID of the key vault
EOF
Enter fullscreen mode Exit fullscreen mode

Verify Keyvault AKS Integration

  • Create a sample pod to mount the secrets 
cat <<EOF | kubectl apply -f -
# This is a sample pod definition for using SecretProviderClass and workload identity to access your key vault
kind: Pod
apiVersion: v1
metadata:
  name: busybox-secrets-store-inline-wi
  labels:
    azure.workload.identity/use: "true"
spec:
  serviceAccountName: "workload-identity-sa"
  containers:
    - name: busybox
      image: registry.k8s.io/e2e-test-images/busybox:1.29-4
      command:
        - "/bin/sleep"
        - "10000"
      volumeMounts:
      - name: secrets-store01-inline
        mountPath: "/mnt/secrets-store"
        readOnly: true
  volumes:
    - name: secrets-store01-inline
      csi:
        driver: secrets-store.csi.k8s.io
        readOnly: true
        volumeAttributes:
          secretProviderClass: "azure-kvname-wi"
EOF
Enter fullscreen mode Exit fullscreen mode

  • List the contents of the volume 
kubectl exec busybox-secrets-store-inline-wi -- ls /mnt/secrets-store/
Enter fullscreen mode Exit fullscreen mode
  • Verify the contents in the file 
kubectl exec busybox-secrets-store-inline-wi -- cat /mnt/secrets-store/secret1
Enter fullscreen mode Exit fullscreen mode

Delete your setup 

az group delete --name keyvault-demo
Enter fullscreen mode Exit fullscreen mode

Top comments (0)