DEV Community

Cover image for Vault With Kubernetes 🔐
Omar Ahmed
Omar Ahmed

Posted on • Edited on

Vault With Kubernetes 🔐

What is HashiCorp Vault?

HashiCorp Vault is a secrets & encryption platform for securely storing, generating, and controlling access to sensitive data (API keys, DB creds, TLS certs, tokens, etc.). Teams use it to centralize secrets, issue short-lived credentials on demand, and enforce least-privilege access across apps, CI/CD, and infrastructure

Install Vault Without a DataStorage (Dev Mode)

helm repo add hashicorp https://helm.releases.hashicorp.com
helm repo update
helm repo list
helm install vault hashicorp/vault \
  --namespace vault --create-namespace \
  --set 'server.dev.enabled=true' \
  --set 'server.dataStorage.enabled=false' \
  --set 'ui.enabled=true'
kubectl config set-context --current --namespace=vault
Enter fullscreen mode Exit fullscreen mode

Install Vault With a DataStorage (Raft storage)

cat > values.yaml <<'YAML'
server:
  dev:
    enabled: false
  standalone:
    enabled: true
    config: |
      ui = true
      listener "tcp" {
        address         = "[::]:8200"
        cluster_address = "[::]:8201"
        tls_disable     = 1
      }
      storage "raft" {
        path = "/vault/data"
      }
  dataStorage:
    enabled: true
    size: 1Gi

ui:
  enabled: true
  serviceType: ClusterIP   # use port-forward; change to NodePort/Ingress if you prefer
YAML
Enter fullscreen mode Exit fullscreen mode
helm repo add hashicorp https://helm.releases.hashicorp.com
helm repo update
helm -n vault install vault hashicorp/vault -f values.yaml --create-namespace --version 0.16.1
Enter fullscreen mode Exit fullscreen mode

Seal/Unseal

When you start a Vault server, it starts in a sealed state. In this state, Vault can access the physical storage, but it cannot decrypt any of the data on it.
Vault encrypts the data using an encryption key (in the keyring) and stores them in its storage backend. To protect this encryption key, Vault encrypts it using another encryption key known as the root key and stores it with the data.
To decrypt the data, Vault needs the root key so that it can decrypt the encryption key. Unsealing is the process of getting access to this root key. Vault encrypts the root key using the unseal key, and stores it alongside all other Vault data.
To summarize, Vault encrypts most data using the encryption key in the keyring. To get the keyring, Vault uses the root key to decrypt it. The root key itself requires the unseal key to decrypt it.

Shamir seals

Key Shares / Unseal Keys                Root Key               Encryption Key
                                                              (protected by root key)
┌─────────────────────┐
│ Jon      🔑         │
├─────────────────────┤
│ Jane     🔑         │-------------->>>🔑 ---------------------->>🔑
├─────────────────────┤                Root Key                  Encryption Key
│ James    🔑         │
├─────────────────────┤
│ Alison   🔑         │
├─────────────────────┤
│ Pam      🔑         │
└─────────────────────┘
Enter fullscreen mode Exit fullscreen mode

Instead of distributing the unseal key to an operator as a single key, the default Vault configuration uses an algorithm known as Shamir's Secret Sharing to split the key into shares.
Vault requires a certain threshold of shares to reconstruct the unseal key. Vault operators add shares one at a time in any order until Vault has enough shares to reconstruct the key. Then, Vault uses the unseal key to decrypt the root key. This is the Vault unseal process.
To Summarize:

  • Vault never writes the master key to disk.
  • Only encrypted data is stored (plus an encrypted data-encryption key).
  • On startup Vault can’t decrypt anything by itself, so it starts sealed.
  • Unseal = provide a quorum of unseal key shares (Shamir) to reconstruct the master key in memory → that master key decrypts the data-encryption key → Vault becomes unsealed and can serve requests.
❯❯❯ k get po                                                                                ⎈ (kind-my-cluster/vault)
NAME                                    READY   STATUS    RESTARTS   AGE
vault-0                                 0/1     Running   0          55s
vault-agent-injector-6d97459765-4mc6b   1/1     Running   0          55s
Enter fullscreen mode Exit fullscreen mode

vault-0 0/1 Ready (not 1/1) >> because vault is sealed

kubectl -n vault exec -it vault-0 -- vault status # sealed=true as shown below
Key                Value
---                -----
Seal Type          shamir
Initialized        true
Sealed             true
Total Shares       5
Threshold          3
Unseal Progress    0/3
Unseal Nonce       n/a
Version            1.8.3
Storage Type       raft
HA Enabled         true
Enter fullscreen mode Exit fullscreen mode
❯❯❯ k exec -it vault-0 -- /bin/sh                                    ⎈ (kind-my-cluster/vault)
/ $ vault operator init 
Unseal Key 1: Q3Awl3Qw+ItIWFyLFVGFiE3OOJ1qHrHC+kQnFD2kwLbE
Unseal Key 2: OON236vYc+bf3K45J75lVHy3FmwIRdUT+mGPU3Iq4SX9
Unseal Key 3: N53iBqGPiL9RRkNNVg5DmrlD5dn6R3Vt703y6NhVUB+0
Unseal Key 4: mWPyNPitk8JTlxZByNkyYJefDo+MTpfIEcDn/lkaKuP3
Unseal Key 5: kRHm81+cYIbjO12eFJp0iJP8y3u7CR4NkOAb/4nHS5Kl

Initial Root Token: s.bQN8VBeKBga76dAUVmaGyTjJ

Vault initialized with 5 key shares and a key threshold of 3. Please securely
distribute the key shares printed above. When the Vault is re-sealed,
restarted, or stopped, you must supply at least 3 of these keys to unseal it
before it can start servicing requests.

Vault does not store the generated master key. Without at least 3 keys to
reconstruct the master key, Vault will remain permanently sealed!

It is possible to generate new unseal keys, provided you have a quorum of
existing unseal keys shares. See "vault operator rekey" for more information.
/ $ 
Enter fullscreen mode Exit fullscreen mode
/ $ vault operator unseal Q3Awl3Qw+ItIWFyLFVGFiE3OOJ1qHrHC+kQnFD2kwLbE
Key                Value
---                -----
Seal Type          shamir
Initialized        true
Sealed             true ###### it is still sealed
Total Shares       5
Threshold          3
Unseal Progress    1/3
Unseal Nonce       8932e24e-a03d-59e7-a694-9c9d9d6005f3
Version            1.8.3
Storage Type       raft
HA Enabled         true
/ $ vault operator unseal OON236vYc+bf3K45J75lVHy3FmwIRdUT+mGPU3Iq4SX9
Key                Value
---                -----
Seal Type          shamir
Initialized        true
Sealed             true
Total Shares       5
Threshold          3
Unseal Progress    2/3
Unseal Nonce       8932e24e-a03d-59e7-a694-9c9d9d6005f3
Version            1.8.3
Storage Type       raft
HA Enabled         true
/ $ vault operator unseal N53iBqGPiL9RRkNNVg5DmrlD5dn6R3Vt703y6NhVUB+0
Key                     Value
---                     -----
Seal Type               shamir
Initialized             true
Sealed                  false ######## it is unsealed
Total Shares            5
Threshold               3
Version                 1.8.3
Storage Type            raft
Cluster Name            vault-cluster-62a5fa0a
Cluster ID              97d206e3-d4a2-2c1f-f898-6a95c034b87e
HA Enabled              true
HA Cluster              n/a
HA Mode                 standby
Active Node Address     <none>
Raft Committed Index    24
Raft Applied Index      24
/ $ vault status
Key                     Value
---                     -----
Seal Type               shamir
Initialized             true
Sealed                  false #<<<<<<<<<<<#########
Total Shares            5
Threshold               3
Version                 1.8.3
Storage Type            raft
Cluster Name            vault-cluster-62a5fa0a
Cluster ID              97d206e3-d4a2-2c1f-f898-6a95c034b87e
HA Enabled              true
HA Cluster              https://vault-0.vault-internal:8201
HA Mode                 active
Active Since            2025-09-03T20:26:58.922335892Z
Raft Committed Index    29
Raft Applied Index      29
Enter fullscreen mode Exit fullscreen mode

Vault-0 becomes 1/1 Ready state (now vault is unsealed):

❯❯❯ k get po                                                        ⎈ (kind-my-cluster/vault)
NAME                                    READY   STATUS    RESTARTS   AGE
vault-0                                 1/1     Running   0          27m
vault-agent-injector-6d97459765-4mc6b   1/1     Running   0          27m
Enter fullscreen mode Exit fullscreen mode
/ $ vault operator init 
Unseal Key 1: Q3Awl3Qw+ItIWFyLFVGFiE3OOJ1qHrHC+kQnFD2kwLbE
Unseal Key 2: OON236vYc+bf3K45J75lVHy3FmwIRdUT+mGPU3Iq4SX9
Unseal Key 3: N53iBqGPiL9RRkNNVg5DmrlD5dn6R3Vt703y6NhVUB+0
Unseal Key 4: mWPyNPitk8JTlxZByNkyYJefDo+MTpfIEcDn/lkaKuP3
Unseal Key 5: kRHm81+cYIbjO12eFJp0iJP8y3u7CR4NkOAb/4nHS5Kl

Initial Root Token: s.bQN8VBeKBga76dAUVmaGyTjJ

/ $ vault login s.bQN8VBeKBga76dAUVmaGyTjJ
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

Key                  Value
---                  -----
token                s.bQN8VBeKBga76dAUVmaGyTjJ
token_accessor       mkdn3cQnsFcO4hmklMrV8Chq
token_duration       ∞
token_renewable      false
token_policies       ["root"]
identity_policies    []
policies             ["root"]
/ $ 
Enter fullscreen mode Exit fullscreen mode
# Enable Kubernetes Authentication
kubectl exec -n vault -it vault-0 -- vault auth enable kubernetes
# Create Service Account for App Pods
kubectl create namespace webapps
kubectl create serviceaccount vault-auth -n webapps

# Configure Kubernetes Auth in Vault
# Extract required info:
SERVICE_ACCOUNT_NAME=vault-auth
NAMESPACE=webapps

# JWT Token
TOKEN_REVIEW_JWT=$(kubectl get secret $(kubectl get serviceaccount $SERVICE_ACCOUNT_NAME -n $NAMESPACE -o jsonpath="{.secrets[0].name}") -n $NAMESPACE -o jsonpath="{.data.token}" | base64 --decode)

# Kubernetes API Host
KUBE_HOST=$(kubectl config view --raw -o=jsonpath='{.clusters[0].cluster.server}')

# Kubernetes CA Cert
KUBE_CA_CERT=$(kubectl get secret $(kubectl get serviceaccount $SERVICE_ACCOUNT_NAME -n $NAMESPACE -o jsonpath="{.secrets[0].name}") -n $NAMESPACE -o jsonpath="{.data['ca.crt']}" | base64 --decode)

# Configure in Vault:
kubectl exec -n vault -it vault-0 -- vault write auth/kubernetes/config \
    token_reviewer_jwt="$TOKEN_REVIEW_JWT" \
    kubernetes_host="$KUBE_HOST" \
    kubernetes_ca_cert="$KUBE_CA_CERT"
Enter fullscreen mode Exit fullscreen mode

Purpose of Kubernetes Auth Configuration:
This command configures Vault to trust and communicate with the Kubernetes API server so that:

  • Vault can validate that service account tokens are legitimate
  • Pods can authenticate to Vault using their Kubernetes service account tokens
  • Vault can verify which namespace and service account a request is coming from Breaking Down Each Parameter: token_reviewer_jwt
"$(kubectl get secret vault-token-reviewer -o jsonpath='{.data.token}' | base64 -d)"
Enter fullscreen mode Exit fullscreen mode
  • What it is: A service account token that Vault uses to authenticate with the Kubernetes API
  • Why needed: Vault needs to call the Kubernetes TokenReview API to validate incoming service account tokens
  • What it does: Acts as Vault's "identity" when talking to Kubernetes

kubernetes_host

"https://kubernetes.default.svc:443"
Enter fullscreen mode Exit fullscreen mode
  • What it is: The URL of the Kubernetes API server from inside the cluster
  • Why needed: Vault needs to know where to send TokenReview requests
  • kubernetes.default.svc: This is the internal DNS name for the Kubernetes API server

kubernetes_ca_cert

"$(kubectl get secret vault-token-reviewer -o jsonpath='{.data.ca\.crt}' | base64 -d)"
Enter fullscreen mode Exit fullscreen mode
  • What it is: The Certificate Authority certificate for the Kubernetes cluster
  • Why needed: Vault uses this to verify the SSL certificate when connecting to the Kubernetes API
  • Security: Prevents man-in-the-middle attacks

The Authentication Flow:
Here's what happens when a pod tries to authenticate to Vault:

1. Pod → Vault: "Here's my service account token"
2. Vault → Kubernetes API: "Is this token valid?" (using token_reviewer_jwt)
3. Kubernetes API → Vault: "Yes, it belongs to service account X in namespace Y"
4. Vault → Pod: "Authentication successful, here's your Vault token"
Enter fullscreen mode Exit fullscreen mode

Without this configuration:

  • Vault can't verify that service account tokens are real
  • Any pod could potentially forge authentication requests
  • You lose the security boundary that Kubernetes provides

With this configuration:

  • Vault cryptographically verifies each authentication request
  • Only legitimate pods with valid service account tokens can authenticate
  • You get fine-grained access control based on Kubernetes RBAC

This step essentially creates a trust relationship between Vault and your Kubernetes cluster, enabling secure, automated authentication for your workloads.

Create Vault Policy: Create a file myapp-policy.hcl:
A policy in Vault is a set of rules (paths + capabilities) that defines what a token/identity is allowed to do.
Vault is deny-by-default—nothing is allowed unless a policy explicitly grants it.

# Access to read/write secret data
path "secret/data/mysql" {  
  capabilities = ["create", "update", "read", "delete", "list"]
}

path "secret/data/frontend" {
  capabilities = ["create", "update", "read", "delete", "list"]
}

# Access to list secrets under the path
path "secret/metadata/mysql" {
  capabilities = ["list"]
}

path "secret/metadata/frontend" {
  capabilities = ["list"]
}
Enter fullscreen mode Exit fullscreen mode

Apply Vault Policy:

kubectl cp myapp-policy.hcl vault/vault-0:/tmp/myapp-policy.hcl
kubectl exec -n vault -it vault-0 -- vault policy write myapp-policy /tmp/myapp-policy.hcl
vault policy read <policy name>                   # show a policy
vault policy list
# to use the app policy token 
export VAULT_TOKEN="$(vault token create -field token -policy=app)"
Enter fullscreen mode Exit fullscreen mode

Create Role in Vault to Map Pod to Policy:

kubectl exec -n vault -it vault-0 -- vault write auth/kubernetes/role/vault-role \
    bound_service_account_names=vault-auth \
    bound_service_account_namespaces="webapps" \
    policies=myapp-policy \
    ttl=24h

kubectl describe clusterrolebindings vault-server-binding
Enter fullscreen mode Exit fullscreen mode

Store Secrets in Vault:

# Enable KV V2 Engine at the specific path (secret)
kubectl exec -n vault -it vault-0 -- vault secrets enable -path=secret -version=2 kv
# Store Secrets in Vault
kubectl exec -n vault -it vault-0 -- vault kv put secret/mysql MYSQL_DATABASE=bankappdb MYSQL_ROOT_PASSWORD=Test@123
kubectl exec -n vault -it vault-0 -- vault kv put secret/frontend MYSQL_ROOT_PASSWORD=Test@123
kubectl exec -n vault -it vault-0 -- vault kv get secret/frontend
Enter fullscreen mode Exit fullscreen mode

What makes KV v2 different from v1:

  • Versioning: every write creates a new version of the secret.
  • Soft delete & undelete: you can delete specific versions and later restore them.

Example:

/ $ vault secrets enable -path=crds kv-v2
Success! Enabled the kv-v2 secrets engine at: crds/
/ $ vault kv put crds/mysql username=root
Key              Value
---              -----
created_time     2025-09-04T12:00:14.567764837Z
deletion_time    n/a
destroyed        false
version          1
/ $ vault kv put crds/mysql username=root password=12345
Key              Value
---              -----
created_time     2025-09-04T12:00:22.119818763Z
deletion_time    n/a
destroyed        false
version          2
/ $ vault kv get crds/mysql
====== Metadata ======
Key              Value
---              -----
created_time     2025-09-04T12:00:22.119818763Z
deletion_time    n/a
destroyed        false
version          2

====== Data ======
Key         Value
---         -----
password    12345
username    root
/ $ vault kv get crds/mysql
====== Metadata ======
Key              Value
---              -----
created_time     2025-09-04T12:00:22.119818763Z
deletion_time    n/a
destroyed        false
version          2

====== Data ======
Key         Value
---         -----
password    12345
username    root
/ $ vault kv metadata get crds/mysql
========== Metadata ==========
Key                     Value
---                     -----
cas_required            false
created_time            2025-09-04T12:00:14.567764837Z
current_version         2
delete_version_after    0s
max_versions            0
oldest_version          0
updated_time            2025-09-04T12:00:22.119818763Z

====== Version 1 ======
Key              Value
---              -----
created_time     2025-09-04T12:00:14.567764837Z
deletion_time    n/a
destroyed        false

====== Version 2 ======
Key              Value
---              -----
created_time     2025-09-04T12:00:22.119818763Z
deletion_time    n/a
destroyed        false
/ $ vault kv delete crds/mysql
Success! Data deleted (if it existed) at: crds/mysql
/ $ vault kv get crds/mysql
====== Metadata ======
Key              Value
---              -----
created_time     2025-09-04T12:00:22.119818763Z
deletion_time    2025-09-04T12:02:04.56772668Z
destroyed        false
version          2

/ $ 
Enter fullscreen mode Exit fullscreen mode

Create YAML Manifest File (With Below Configurations):
Example annotation block for a pod:

annotations:
  vault.hashicorp.com/agent-inject: "true" # this should be set to a true or false, default to false
  vault.hashicorp.com/role: "vault-role" # role name
  vault.hashicorp.com/agent-inject-secret-MYSQL_ROOT_PASSWORD: "secret/mysql" # agent-inject-secret-<the name of the secret> , this will retrieve the secret from Vault
  # agent-inject-template-<the name of the secret>: used for rendering a secret
  vault.hashicorp.com/agent-inject-template-MYSQL_ROOT_PASSWORD: |
    {{- with secret "secret/mysql" -}}
    export MYSQL_ROOT_PASSWORD="{{ .Data.data.MYSQL_ROOT_PASSWORD }}"
    {{- end }}
Enter fullscreen mode Exit fullscreen mode

what to read and how to write.
vault.hashicorp.com/agent-inject-secret-MYSQL_ROOT_PASSWORD: "<vault path>"
Tells the injector which secret to fetch from Vault and that it should write a file named MYSQL_ROOT_PASSWORD under /vault/secrets/ (unless you override the filename/path).
vault.hashicorp.com/agent-inject-template-MYSQL_ROOT_PASSWORD:
Provides a custom template for what to write into that file. Without this, the injector writes a default rendering (key=value lines). With it, you control the exact content.
Template: It renders a shell line into the injected file, like: export MYSQL_ROOT_PASSWORD="supersecret123"

    {{- with secret "secret/mysql" -}}
    export MYSQL_ROOT_PASSWORD="{{ .Data.data.MYSQL_ROOT_PASSWORD }}"
    {{- end }}
Enter fullscreen mode Exit fullscreen mode

The sidecar Vault Agent will:
• Auth using the service account token
• Fetch secrets from Vault
• Write them to /vault/secrets/... >>> /vault/secrets/MYSQL_ROOT_PASSWORD inside the pod

UI access

kubectl -n vault get svc vault
kubectl -n vault port-forward --address=0.0.0.0 svc/vault 8200:8200
Enter fullscreen mode Exit fullscreen mode

Clean-up steps

helm uninstall vault
kubectl -n vault delete pod -l app.kubernetes.io/name=vault --force --grace-period=0 || true
kubectl -n vault delete pvc -l app.kubernetes.io/name=vault || true
Enter fullscreen mode Exit fullscreen mode

Top comments (0)