DEV Community

giveitatry
giveitatry

Posted on

Complete Guide: Let's Encrypt SSL with cert-manager and Hetzner DNS on Kubernetes

#kubernetes #devops #security #tutorial

Introduction

This guide walks you through setting up automatic SSL certificates from Let's Encrypt on a Kubernetes cluster using cert-manager with Hetzner DNS for DNS-01 challenge validation. This is the most reliable method for issuing and renewing SSL certificates automatically, especially if you have multiple domains or subdomains.

By the end of this guide, you'll have:

  • Automatic SSL certificate issuance via Let's Encrypt
  • DNS-based validation (no need to expose HTTP validation endpoints)
  • Automatic certificate renewal before expiration
  • HTTPS redirect middleware in Traefik

Prerequisites

  • A Kubernetes cluster (k3s, kubeadm, EKS, etc.)
  • kubectl configured to access your cluster
  • A domain with DNS hosted at Hetzner DNS
  • Helm installed on your local machine
  • Traefik ingress controller (this guide uses Traefik)

Architecture Overview

Here's how the process works:

  1. You create a Certificate resource in Kubernetes
  2. cert-manager reads this and contacts Let's Encrypt
  3. Let's Encrypt returns a DNS challenge token
  4. cert-manager calls the Hetzner webhook with this token
  5. The Hetzner webhook uses the Hetzner DNS API to create a TXT record
  6. Let's Encrypt queries the DNS and validates the TXT record
  7. Let's Encrypt issues the certificate
  8. cert-manager stores it in a Kubernetes Secret
  9. Your Ingress uses this Secret for HTTPS

Step 1: Install cert-manager

cert-manager is the core component that manages certificate lifecycle.

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.19.2/cert-manager.yaml
Enter fullscreen mode Exit fullscreen mode

Wait for it to be ready:

kubectl wait --for=condition=ready pod -l app.kubernetes.io/instance=cert-manager -n cert-manager --timeout=300s
Enter fullscreen mode Exit fullscreen mode

Verify installation:

kubectl get pods -n cert-manager
Enter fullscreen mode Exit fullscreen mode

You should see three pods running:

  • cert-manager
  • cert-manager-cainjector
  • cert-manager-webhook

Step 2: Create Hetzner DNS API Token

  1. Log in to Hetzner DNS Control Panel
  2. Navigate to "API Tokens" (usually under account settings)
  3. Click "Create new token"
  4. Give it a name like "cert-manager"
  5. Copy the token immediately (you won't see it again)

Important: Keep this token secret. Anyone with this token can modify your DNS records.

Step 3: Install the Hetzner Webhook for cert-manager

The Hetzner webhook is a cert-manager extension that knows how to talk to the Hetzner DNS API.

Add the Helm repository:

helm repo add cert-manager-webhook-hetzner https://vadimkim.github.io/cert-manager-webhook-hetzner
helm repo update
Enter fullscreen mode Exit fullscreen mode

Install the webhook:

helm install cert-manager-webhook-hetzner \
  --namespace cert-manager \
  cert-manager-webhook-hetzner/cert-manager-webhook-hetzner
Enter fullscreen mode Exit fullscreen mode

Verify the webhook pod is running:

kubectl get pods -n cert-manager | grep hetzner
Enter fullscreen mode Exit fullscreen mode

Step 4: Create the Hetzner API Secret

Create a file called hetzner-secret.yaml:

apiVersion: v1
kind: Secret
metadata:
  name: hcloud-api-token-secret
  namespace: cert-manager
type: Opaque
stringData:
  api-key: "your-hetzner-api-token-here"
Enter fullscreen mode Exit fullscreen mode

Important: The key must be named api-key (not token). This is what the webhook expects.

Apply it:

kubectl apply -f hetzner-secret.yaml
Enter fullscreen mode Exit fullscreen mode

Verify the secret was created:

kubectl get secret hcloud-api-token-secret -n cert-manager
Enter fullscreen mode Exit fullscreen mode

Step 5: Verify the Webhook's API Group Registration

Before creating the ClusterIssuer, you need to know the exact API group name that the webhook registered. This is crucial and often a source of problems.

Run this command:

kubectl api-resources | grep hetzner
Enter fullscreen mode Exit fullscreen mode

You should see output like:

hetzner    hetzner.cert-mananger-webhook.noshoes.xyz/v1alpha1    false    ChallengePayload
Enter fullscreen mode Exit fullscreen mode

Note the exact API group name - in this example it's hetzner.cert-mananger-webhook.noshoes.xyz. This is what you'll use in your ClusterIssuer. Pay attention to the spelling - some versions have a typo: "mananger" instead of "manager".

Step 6: Set Up RBAC Permissions

cert-manager needs permission to call the webhook, and the webhook needs permission to read the Hetzner API secret.

Create a file called rbac.yaml:

# Allow webhook to read the Hetzner API token
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: cert-manager-webhook-hetzner-secret-access
  namespace: cert-manager
rules:
- apiGroups: [""]
  resources: ["secrets"]
  resourceNames: ["hcloud-api-token-secret"]
  verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: cert-manager-webhook-hetzner-secret-access
  namespace: cert-manager
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: cert-manager-webhook-hetzner-secret-access
subjects:
- kind: ServiceAccount
  name: cert-manager-webhook-hetzner
  namespace: cert-manager
---
# Allow cert-manager to call the webhook
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: cert-manager-webhook-hetzner
rules:
- apiGroups:
  - hetzner.cert-mananger-webhook.noshoes.xyz
  resources:
  - hetzner
  verbs:
  - create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: cert-manager-webhook-hetzner
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cert-manager-webhook-hetzner
subjects:
- kind: ServiceAccount
  name: cert-manager
  namespace: cert-manager
Enter fullscreen mode Exit fullscreen mode

Important: Replace hetzner.cert-mananger-webhook.noshoes.xyz in the ClusterRole with whatever kubectl api-resources | grep hetzner showed you.

Apply it:

kubectl apply -f rbac.yaml
Enter fullscreen mode Exit fullscreen mode

Step 7: Create the ClusterIssuer

The ClusterIssuer tells cert-manager how to request certificates from Let's Encrypt using Hetzner DNS.

Create a file called cluster-issuer.yaml:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-dns-hetzner
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: admin@example.com
    privateKeySecretRef:
      name: letsencrypt-dns-hetzner-key
    solvers:
      - dns01:
          webhook:
            groupName: hetzner.cert-mananger-webhook.noshoes.xyz
            solverName: hetzner
            config:
              secretName: hcloud-api-token-secret
              zoneName: example.com
              apiUrl: https://dns.hetzner.com/api/v1
Enter fullscreen mode Exit fullscreen mode

Replace:

  • admin@example.com with your email
  • groupName with the exact value from kubectl api-resources | grep hetzner
  • zoneName with your root domain (e.g., example.com)

For testing/staging (to avoid Let's Encrypt rate limits), use this server instead:

server: https://acme-staging-v02.api.letsencrypt.org/directory
Enter fullscreen mode Exit fullscreen mode

Apply it:

kubectl apply -f cluster-issuer.yaml
Enter fullscreen mode Exit fullscreen mode

Verify the issuer is ready:

kubectl describe clusterissuer letsencrypt-dns-hetzner
Enter fullscreen mode Exit fullscreen mode

Look for Status: True in the conditions.

Step 8: Create Certificate Resources

Now create a Certificate resource for each domain you want to secure.

Create a file called certificates.yaml:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: example-tls
  namespace: default
spec:
  secretName: example-tls
  issuerRef:
    name: letsencrypt-dns-hetzner
    kind: ClusterIssuer
  commonName: example.com
  dnsNames:
    - example.com
    - "*.example.com"
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: api-tls
  namespace: default
spec:
  secretName: api-tls
  issuerRef:
    name: letsencrypt-dns-hetzner
    kind: ClusterIssuer
  commonName: api.example.com
  dnsNames:
    - api.example.com
Enter fullscreen mode Exit fullscreen mode

Replace:

  • Namespace with your application's namespace
  • Domain names with your actual domains
  • Certificate names as appropriate

Apply it:

kubectl apply -f certificates.yaml
Enter fullscreen mode Exit fullscreen mode

Step 9: Monitor Certificate Creation

Watch the certificates being created:

kubectl get certificate -A -w
Enter fullscreen mode Exit fullscreen mode

Check detailed status:

kubectl describe certificate example-tls -n default
Enter fullscreen mode Exit fullscreen mode

Watch the challenges:

kubectl get challenge -n default -w
Enter fullscreen mode Exit fullscreen mode

Check the webhook logs to see DNS records being created:

kubectl logs -n cert-manager -l app=cert-manager-webhook-hetzner -f --tail=50
Enter fullscreen mode Exit fullscreen mode

You should see messages like:

Added TXT record result: {"record":{"id":"...","type":"TXT","name":"_acme-challenge.example",...}}
Presented txt record _acme-challenge.example.com.
Delete TXT record result: {"error":{}}
Enter fullscreen mode Exit fullscreen mode

Once the certificate shows Ready: True, the secret is ready to use.

Step 10: Use Certificates in Traefik Ingress

For standard Ingress with Traefik:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-ingress
  namespace: default
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-dns-hetzner
spec:
  ingressClassName: traefik
  tls:
    - hosts:
        - example.com
      secretName: example-tls
  rules:
    - host: example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: my-service
                port:
                  number: 80
Enter fullscreen mode Exit fullscreen mode

For Traefik IngressRoute with HTTP-to-HTTPS redirect:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: redirect-https
  namespace: default
spec:
  redirectScheme:
    scheme: https
    permanent: true
---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: example-http
  namespace: default
spec:
  entryPoints:
    - web
  routes:
    - match: Host(`example.com`)
      kind: Rule
      middlewares:
        - name: redirect-https
          namespace: default
      services:
        - name: my-service
          port: 80
---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: example-https
  namespace: default
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`example.com`)
      kind: Rule
      services:
        - name: my-service
          port: 80
  tls:
    secretName: example-tls
Enter fullscreen mode Exit fullscreen mode

Troubleshooting Guide

Problem 1: Certificate Stuck in "Pending" State

Symptom: kubectl get certificate shows READY: False for hours

Solution:

Check the certificate details:

kubectl describe certificate example-tls -n default
Enter fullscreen mode Exit fullscreen mode

Look at the events section. The most common causes are listed below.

Problem 2: "Unable to get secret" Error

Error message:

unable to get secret `hcloud-api-token-secret/cert-manager`; secrets "hcloud-api-token-secret" is forbidden
Enter fullscreen mode Exit fullscreen mode

Cause: The RBAC permissions are not set up correctly. The webhook's service account can't read the Hetzner API token secret.

Solution:

  1. Verify the secret exists: 
kubectl get secret hcloud-api-token-secret -n cert-manager
Enter fullscreen mode Exit fullscreen mode
  1. Reapply the RBAC rules: 
kubectl apply -f rbac.yaml
Enter fullscreen mode Exit fullscreen mode
  1. Make sure the Role has the correct secret name: 
rules:
- apiGroups: [""]
  resources: ["secrets"]
  resourceNames: ["hcloud-api-token-secret"]  # Exact match required
  verbs: ["get"]
Enter fullscreen mode Exit fullscreen mode

Problem 3: "Key 'api-key' not found in secret"

Error message:

unable to get api-key from secret `hcloud-api-token-secret`; key "api-key" not found in secret data
Enter fullscreen mode Exit fullscreen mode

Cause: The secret key is named wrong. The webhook expects exactly api-key.

Solution:

Delete and recreate the secret with the correct key name:

kubectl delete secret hcloud-api-token-secret -n cert-manager
Enter fullscreen mode Exit fullscreen mode

Create it again with api-key:

apiVersion: v1
kind: Secret
metadata:
  name: hcloud-api-token-secret
  namespace: cert-manager
type: Opaque
stringData:
  api-key: "your-token-here"
Enter fullscreen mode Exit fullscreen mode

Problem 4: "cannot create resource 'hetzner' is forbidden"

Error message:

User "system:serviceaccount:cert-manager:cert-manager" cannot create resource "hetzner" in API group "hetzner.cert-manager-webhook.noshoes.xyz"
Enter fullscreen mode Exit fullscreen mode

Cause: cert-manager service account doesn't have permission to call the webhook.

Solution:

Reapply the ClusterRole and ClusterRoleBinding:

kubectl apply -f rbac.yaml
Enter fullscreen mode Exit fullscreen mode

Verify the ClusterRoleBinding exists:

kubectl get clusterrolebinding | grep hetzner
Enter fullscreen mode Exit fullscreen mode

Problem 5: "the server could not find the requested resource"

Error message:

the server could not find the requested resource (post hetzner.cert-manager-webhook.noshoes.xyz)
Enter fullscreen mode Exit fullscreen mode

Cause: The groupName in your ClusterIssuer doesn't match what the webhook registered.

Solution:

Check the actual API group:

kubectl api-resources | grep hetzner
Enter fullscreen mode Exit fullscreen mode

This outputs something like:

hetzner    hetzner.cert-mananger-webhook.noshoes.xyz/v1alpha1    false    ChallengePayload
Enter fullscreen mode Exit fullscreen mode

Copy the exact API group (e.g., hetzner.cert-mananger-webhook.noshoes.xyz) and update your ClusterIssuer:

solvers:
  - dns01:
      webhook:
        groupName: hetzner.cert-mananger-webhook.noshoes.xyz  # Use exact match
        solverName: hetzner
Enter fullscreen mode Exit fullscreen mode

Problem 6: The Webhook Has a Typo in the API Group Name

Issue: The webhook registers as hetzner.cert-mananger-webhook.noshoes.xyz (with "mananger" - missing 'e')

Cause: This is a known issue in some versions of the Hetzner webhook Helm chart.

Solution:

You have two options:

Option A (Quickest): Just use the misspelled group name in your ClusterIssuer. It works fine despite the typo:

groupName: hetzner.cert-mananger-webhook.noshoes.xyz
Enter fullscreen mode Exit fullscreen mode

Option B (Proper fix): Reinstall the webhook with a corrected version or use a different webhook implementation that doesn't have this typo.

Problem 7: Challenge Not Being Presented

Symptom: Challenge status shows Presented: false but no errors in logs

Cause: Webhook pod might have crashed or become unhealthy

Solution:

Check webhook health:

kubectl get pods -n cert-manager | grep hetzner
Enter fullscreen mode Exit fullscreen mode

Check pod logs:

kubectl logs -n cert-manager -l app=cert-manager-webhook-hetzner
Enter fullscreen mode Exit fullscreen mode

Restart the webhook:

kubectl rollout restart deployment/cert-manager-webhook-hetzner -n cert-manager
Enter fullscreen mode Exit fullscreen mode

Delete the challenge to trigger a retry:

kubectl delete challenge -n default --all
Enter fullscreen mode Exit fullscreen mode

Problem 8: DNS Record Not Being Created in Hetzner

Symptom: No TXT record appears in Hetzner DNS console

Cause: Usually an invalid API token or zone name mismatch

Solution:

  1. Verify the API token is correct: 
kubectl get secret hcloud-api-token-secret -n cert-manager -o jsonpath='{.data.api-key}' | base64 -d
Enter fullscreen mode Exit fullscreen mode
  1. Verify the zone name matches your domain: 
zoneName: example.com  # Must match your actual domain
Enter fullscreen mode Exit fullscreen mode
  1. Check the webhook logs for errors: 
kubectl logs -n cert-manager -l app=cert-manager-webhook-hetzner -f | grep -i error
Enter fullscreen mode Exit fullscreen mode
  1. Test the API token manually: 
curl -H "Auth-API-Token: your-token" https://dns.hetzner.com/api/v1/zones
Enter fullscreen mode Exit fullscreen mode

Problem 9: Certificate Ready but HTTPS Still Shows Old/Invalid Certificate

Cause: Ingress is not using the new secret or old ingress resource is still active

Solution:

  1. Verify the secret exists: 
kubectl get secret example-tls -n default
Enter fullscreen mode Exit fullscreen mode
  1. Check the Ingress is referencing it: 
kubectl describe ingress example-ingress -n default
Enter fullscreen mode Exit fullscreen mode
  1. If using both Ingress and IngressRoute, delete the old Ingress: 
kubectl delete ingress example-ingress -n default
Enter fullscreen mode Exit fullscreen mode
  1. Wait for the load balancer to update (usually 30 seconds to 2 minutes)

Verification Checklist

Once everything is set up, verify with this checklist:

# 1. cert-manager is running
kubectl get pods -n cert-manager

# 2. Webhook is running
kubectl get pods -n cert-manager | grep hetzner

# 3. ClusterIssuer is ready
kubectl describe clusterissuer letsencrypt-dns-hetzner

# 4. Certificate is ready
kubectl get certificate -A

# 5. Secret is created
kubectl get secret example-tls -n default

# 6. Ingress/IngressRoute is configured
kubectl get ingress,ingressroute -A

# 7. DNS records exist
nslookup _acme-challenge.example.com

# 8. HTTPS works
curl -I https://example.com
Enter fullscreen mode Exit fullscreen mode

Automatic Renewal

cert-manager automatically renews certificates 30 days before expiration. You don't need to do anything.

Monitor renewal status:

kubectl describe certificate example-tls -n default
Enter fullscreen mode Exit fullscreen mode

Check cert-manager logs for renewal activity:

kubectl logs -n cert-manager -l app=cert-manager | grep -i renew
Enter fullscreen mode Exit fullscreen mode

Best Practices

  1. Use ClusterIssuer, not Issuer - ClusterIssuer is namespace-agnostic and reusable
  2. Use production Let's Encrypt - Only use staging for testing to avoid rate limits
  3. Keep API token secret - Don't commit it to git, use sealed secrets or external secret management
  4. Monitor certificate expiration - Set up alerts for certificates not renewing
  5. Use separate secrets per domain - Makes it easier to rotate certs individually
  6. Test in staging first - Use the Let's Encrypt staging server before production
  7. Keep webhook updated - Regularly update the webhook Helm chart for security patches

Conclusion

You now have a fully automated SSL certificate system that:

  • Issues certificates automatically from Let's Encrypt
  • Validates using DNS (no need for HTTP validation)
  • Renews certificates automatically before expiration
  • Works across all your subdomains under one root domain
  • Integrates seamlessly with Traefik

If you encounter issues, the troubleshooting guide above covers the most common problems and their solutions.

Top comments (0)