DEV Community

Cover image for Sealed secrets - Bring your own keys and multi-cluster scenario
Ashok Nagaraj
Ashok Nagaraj

Posted on

10 3

Sealed secrets - Bring your own keys and multi-cluster scenario

Bitnami sealed secrets have a key rotation done every 30 days by default and every sealed-secret-controller creates it's own private key. This makes it hard to adopt for multi-cluster scenarios. It can be mitigated by using one's own key pair for encryption/decryption across multiple clusters.

Using own key pair
  1. Preparation
❯ kubectl create ns sealed-secrets
❯ export PRIVATEKEY="acmetls.key"export PUBLICKEY="acmetls.crt"export NAMESPACE="sealed-secrets"export SECRETNAME="acme-keys"

# Create key pair
❯ openssl req -x509 -nodes -newkey rsa:4096 -keyout "$PRIVATEKEY" -out "$PUBLICKEY" -subj "/CN=sealed-secret/O=sealed-secret"
Generating a 4096 bit RSA private key
.....++
......++
writing new private key to 'acmetls.key'
-----ls acme*
acmetls.crt acmetls.key
Enter fullscreen mode Exit fullscreen mode
  1. Prepare and update new keys into controller
❯ kubectl create ns sealed-secrets
namespace/sealed-secrets created

# Create secret and label it as "active"
❯ kubectl -n "$NAMESPACE" create secret tls "$SECRETNAME" --cert="$PUBLICKEY" --key="$PRIVATEKEY"
kubectl -n "$NAMESPACE" label secret "$SECRETNAME" sealedsecrets.bitnami.com/sealed-secrets-key=active
secret/acme-keys created
secret/acme-keys labeled

# Install the controller
❯ helm upgrade --install ss-app sealed-secrets/sealed-secrets --namespace=sealed-secrets

# delete existing key
❯ kubectl get pods -n sealed-secrets
NAME                                     READY   STATUS    RESTARTS   AGE
ss-app-sealed-secrets-556c68c858-qq5k6   1/1     Running   0          3m47s
❯ kubectl delete pod ss-app-sealed-secrets-556c68c858-qq5k6 -n sealed-secrets
pod "ss-app-sealed-secrets-556c68c858-qq5k6" deleted

# another instance of the pod will come up with the new "acme-keys"
❯ kubectl logs ss-app-sealed-secrets-556c68c858-gqpwf -n sealed-secrets
2022/04/25 13:02:01 Starting sealed-secrets controller version: 0.17.4
controller version: 0.17.4
2022/04/25 13:02:01 Searching for existing private keys
2022/04/25 13:02:02 ----- sealed-secrets-key5nhjv
2022/04/25 13:02:02 ----- sealed-secrets-key98m56
2022/04/25 13:02:02 ----- sealed-secrets-keyztj4v
2022/04/25 13:02:02 ----- acme-keys
2022/04/25 13:02:02 HTTP server serving on :8080
Enter fullscreen mode Exit fullscreen mode
  1. Encrypt with new keys
❯ kubectl create secret generic secret-name --dry-run=client --from-literal=foo=bar -o yaml | kubeseal --controller-name=ss-app-sealed-secrets --controller-namespace=sealed-secrets -o yaml --cert=$PUBLICKEY | kubectl apply -f -
sealedsecret.bitnami.com/secret-name created
Enter fullscreen mode Exit fullscreen mode
  1. Verify decryption
❯ kubectl logs ss-app-sealed-secrets-556c68c858-gqpwf -n sealed-secrets | tail -2
2022/04/25 13:08:07 Updating default/secret-name
2022/04/25 13:08:07 Event(v1.ObjectReference{Kind:"SealedSecret", Namespace:"default", Name:"secret-name", UID:"c9ee00ac-c974-43f7-9b7d-16cda1b516ff", APIVersion:"bitnami.com/v1alpha1", ResourceVersion:"57973", FieldPath:""}): type: 'Normal' reason: 'Unsealed' SealedSecret unsealed successfully

❯ kubectl get sealedsecrets.bitnami.com
NAME          AGE
secret-name   47s

❯ kubectl get secrets secret-name
NAME          TYPE     DATA   AGE
secret-name   Opaque   1      57s

Enter fullscreen mode Exit fullscreen mode
Toying with the secret rotation interval during creation of the sealed-secrets controller
  • Edit --key-renew-period=<value> in the command line args for the controller (in-case of deployment)
  • Use --set=keyrenewperiod=<value> while installing through Helm

Note: keyrenewperiod=0 stops keys from being rotated; though seemingly easy solution, not recommended in production use-cases

Billboard image

Deploy and scale your apps on AWS and GCP with a world class developer experience

Coherence makes it easy to set up and maintain cloud infrastructure. Harness the extensibility, compliance and cost efficiency of the cloud.

Learn more

Top comments (0)

Image of Docusign

🛠️ Bring your solution into Docusign. Reach over 1.6M customers.

Docusign is now extensible. Overcome challenges with disconnected products and inaccessible data by bringing your solutions into Docusign and publishing to 1.6M customers in the App Center.

Learn more