Automating APISIX TLS Secrets with CertManager using Argo Events: A Hands-On Guide

Vitalii Liubimov — Tue, 27 May 2025 16:34:04 +0000

When it comes to managing APIs in Kubernetes, APISIX offers a powerful and flexible platform. Recently, I encountered a unique challenge while integrating TLS certificates with APISIX,specially when automating certificate updates managed by cert-manager. cert-manager generates secrets in a standardized format (tls.crt, tls.key, ca.crt), but APISIX expects a different format: (cert, key) - ApisixTls CRD. Here's my journey and the solution that eventually worked, with Argo Events providing a real-time, automated workflow.

Problem Overview

Challenge: cert-manager automatically generates and renews certificates but stores them in a Kubernetes Secret object with keys like tls.crt and tls.key. However, APISIX requires a specific format - a Kubernetes secret with keys named cert (for the certificate) and key (for the private key) - interesting why Apache APISIX? Without the correct format, APISIX wouldn't be able to use the updated certificates.

Possible solutions:

When provisioning with Terraform, you can create the Secret and ApisixTls CRD using an intermediate secret. However, remapping the TLS secret during Terraform provisioning doesn’t update the mapped secret and can only handle the key’s state.

Goals:

Automate the TLS certificate injection into APISIX.
Ensure the process triggers whenever a TLS secret is updated or created by cert-manager.
Use Argo Events to listen for changes in TLS secrets, transforming them and injecting them in the APISIX-compatible format.

Solution Architecture Overview

Here's how the solution is structured:

cert-manager generates the TLS certificates.
Argo Events listens for events related to TLS secret creation or updates and sends corresponding API requests to the APISIX Admin API.
APISIX ingests the transformed secrets and applies them automatically.

Tools and Technologies

APISIX: Manages and routes API traffic.
cert-manager: Manages and renews TLS certificates automatically.
Argo Events: Observes changes in TLS secrets and triggers automation pipelines.

Prerequisites: Installing APISIX, cert-manager, and Argo Events

Step 1: Install APISIX

To install APISIX, follow the APISIX documentation. In this article, we assume you’ve installed APISIX with the Admin API in the gateway namespace.

Step 2: Set up cert-manager

cert-manager installation is straightforward using Helm:

helm repo add jetstack https://charts.jetstack.io
helm repo update
helm install cert-manager jetstack/cert-manager --namespace cert-manager --create-namespace

Step 3: Install Argo Events

Argo Events can also be installed with Helm:

helm repo add argo https://argoproj.github.io/argo-helm
helm repo update
helm install argo-events argo/argo-events --namespace argo --create-namespace

The installation process is described briefly. Please refer to the corresponding documentation or contact me for
details.

Solution: Configuring Argo Events to Watch for cert-manager TLS Secrets

To make the solution work, I set up Argo Events to watch for changes in cert-manager's TLS secrets. Here's the YAML manifest setup I used to transform cert-manager secrets into a format compatible with APISIX.

Step 1: Create Service Account and Roles

# serviceAccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: argo-events-sa
  namespace: argo
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: secrets-mapper-role
rules:
  - apiGroups: [ "" ]
    resources: [ "secrets" ]
    verbs: [ "*" ]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: argo-secret-mapper-role-binding
subjects:
  - kind: ServiceAccount
    name: argo-events-sa
    namespace: argo
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin

Step 2: Configure the EventBus

Define an EventBus to manage NATS messaging within Argo Events.

# eventBus.yaml
apiVersion: argoproj.io/v1alpha1
kind: EventBus
metadata:
  name: default
  namespace: argo
spec:
  nats:
    native: { }

Step 3: Set Up the Event Source for Watching TLS Secret Changes

This event source watches for any changes to Kubernetes TLS secrets annotated with apisix.io/tls. When it detects an
addition or update, it will trigger a transformation process.

# eventSource.yaml
apiVersion: argoproj.io/v1alpha1
kind: EventSource
metadata:
  name: secrets-mapper-event-source
  namespace: argo
spec:
  template:
    serviceAccountName: argo-events-sa
  resource:
    apisix-tls-secret:
      version: v1
      resource: secrets
      eventTypes:
        - ADD
        - UPDATE
        - DELETE
      filter:
        fields:
          - key: metadata.annotations.apisix\.io\/tls
            operation: "="
            value: "true"
    apisix-mtls-secret:
      version: v1
      resource: secrets
      eventTypes:
        - ADD
        - UPDATE
        - DELETE
      filter:
        fields:
          - key: metadata.annotations.apisix\.io\/tls
            operation: "="
            value: "mtls"

Step 4: Create a secret with Admin token

# apisix-creds.yaml
kind: Secret
metadata:
  name: apisix-admin-credentials
  namespace: argo
data:
  admin: base64-encoded-apisix-admin-token

We use the reflector’s annotations to create the secret in the gateway namespace, automatically updating the secret in
the Argo namespace during credential rotation. That’s another story, though!

Step 5: Create the Sensor to Transform and Inject Secrets into APISIX

The sensor will transform the secret fields and inject them into APISIX. This involves reading the secret data,converting it to base64-decoded strings, and sending it to the APISIX admin API.

# sensor.yaml
apiVersion: argoproj.io/v1alpha1
kind: Sensor
metadata:
  name: tls-apisix-registrar
  namespace: argo
spec:
  template:
    serviceAccountName: argo-events-sa
    container:
      env:
        - name: LOG_LEVEL
          value: error
  dependencies:
    - name: tls-secret
      eventSourceName: secrets-mapper-event-source
      eventName: apisix-tls-secret
      transform:
        jq: |
          .url = "http://apisix-admin.gateway.svc.cluster.local:9180/apisix/admin/ssls/" +
                 .body.metadata.annotations["cert-manager.io/certificate-name"] + "-" +
                 .body.metadata.namespace |
          .method = if .type == "ADD" or .type == "UPDATE" then "PUT" else "DELETE" end |
          .certs.cert = (.body.data["tls.crt"] | @base64d) |
          .certs.key = (.body.data["tls.key"] | @base64d) |
          .certs.snis = (.body.metadata.annotations["cert-manager.io/alt-names"] | split(",")) |
          .certs.type = "server"
    - name: mtls-secret
      eventSourceName: secrets-mapper-event-source
      eventName: apisix-mtls-secret
      transform:
        jq: |
          .url = "http://apisix-admin.gateway.svc.cluster.local:9180/apisix/admin/ssls/" +
                 .body.metadata.annotations["cert-manager.io/certificate-name"] + "-" +
                 .body.metadata.namespace |
          .method = if .type == "ADD" or .type == "UPDATE" then "PUT" else "DELETE" end |
          .certs.cert = (.body.data["tls.crt"] | @base64d) |
          .certs.key = (.body.data["tls.key"] | @base64d) |
          .certs.ca = (.body.data["ca.crt"] | @base64d) |
          .certs.snis = (.body.metadata.annotations["cert-manager.io/alt-names"] | split(",")) |
          .certs.type = "server"
  triggers:
    - template:
        name: log-trigger
        conditions: "tls-secret || mtls-secret"
        log:
          intervalSeconds: 20
    - template:
        name: apisix-register-tls
        conditions: "tls-secret"
        http:
          url: "{url}"
          method: "PUT"
          headers:
            content-type: "application/json"
          payload:
            - src:
                dependencyName: tls-secret
                dataKey: certs.snis
                useRawData: true
              dest: snis
            - src:
                dependencyName: tls-secret
                dataKey: certs.cert
              dest: cert
            - src:
                dependencyName: tls-secret
                dataKey: certs.key
              dest: key
            - src:
                dependencyName: tls-secret
                dataKey: certs.type
              dest: type
          secureHeaders:
            - name: "X-API-KEY"
              valueFrom:
                secretKeyRef:
                  name: apisix-admin-credentials
                  key: admin
      parameters:
        - src:
            dependencyName: tls-secret
            dataKey: url
          dest: http.url
        - src:
            dependencyName: tls-secret
            dataKey: method
          dest: http.method

    - template:
        name: apisix-register-mtls
        conditions: "mtls-secret"
        http:
          url: "{url}"
          method: "PUT"
          headers:
            content-type: "application/json"
          payload:
            - src:
                dependencyName: mtls-secret
                dataKey: certs.snis
                useRawData: true
              dest: snis
            - src:
                dependencyName: mtls-secret
                dataKey: certs.cert
              dest: cert
            - src:
                dependencyName: mtls-secret
                dataKey: certs.key
              dest: key
            - src:
                dependencyName: mtls-secret
                dataKey: certs.type
              dest: type
            - src:
                dependencyName: mtls-secret
                dataKey: certs.ca
              dest: client.ca
          secureHeaders:
            - name: "X-API-KEY"
              valueFrom:
                secretKeyRef:
                  name: apisix-admin-credentials
                  key: admin
      parameters:
        - src:
            dependencyName: mtls-secret
            dataKey: url
          dest: http.url
        - src:
            dependencyName: mtls-secret
            dataKey: method
          dest: http.method
      retryStrategy:
        steps: 3
        duration: 3s
      policy:
        status:
          allow:
            - 200
            - 201

Replace the Apisix Admin API url if it doesn't match your configuration: http://apisixadmin.gateway.svc.cluster.local:9180/

Apply all manifests via kubectl or other tools.

kubectl apply -f .

Result verification:

Check the pods on the argo namespace, verify specific pods are running and ready.

kubectl get pods | grep -E 'bus|mapper|registrar'

eventbus-default-stan-0                                          2/2     Running   0          76m
eventbus-default-stan-1                                          2/2     Running   0          76m
eventbus-default-stan-2                                          2/2     Running   0          75m
secrets-mapper-event-source-eventsource-wkrr9-7cd7d7d6cb-db74n   1/1     Running   0          76m
tls-apisix-registrar-sensor-rwh7f-fcf8d5bcc-6qtsd                1/1     Running   0          18m

Testing certificate registration:

# certificate/test.yaml
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: selfsigned-issuer
  namespace: argo
spec:
  selfSigned: { }
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: apisix-example-cert
  namespace: argo
spec:
  secretName: apisix-example-tls  # Name of the Secret that will be created
  duration: 2160h # 90d
  renewBefore: 360h # 15d
  commonName: example.com
  dnsNames:
    - example.com
    - www.example.com
    - api.example.com
  issuerRef:
    name: selfsigned-issuer
    kind: Issuer
  secretTemplate:
    annotations:
      apisix.io/tls: "true" # annotation for eventsource

Apply the issuer and certificate:

kubectl apply -f ./certificate/

Verify the certificate added on Apisix Dashboard:

Delete the test certificate – verify it has been deleted on the APISIX Dashboard or API:

For debugging, check the logs from the corresponding pod:
tls-apisix-registrar-sensor-…

Important Note:

This solution does not automatically register existing certificates in APISIX. To sync previously created certificates, you should:

Delete the existing TLS Secret (managed by Cert-Manager).

Let Cert-Manager recreate it - triggering Argo Events to register it in APISIX.

Conclusion:

Managing TLS certificates in Kubernetes can be complex, especially when integrating with API gateways like APISIX.
Cert-Manager simplifies certificate issuance and renewal, but APISIX requires certificates in a specific format – creating a need for automation.

In this guide, we explored how Argo Events bridges this gap by:
Detecting changes to Cert-Manager-generated TLS Secrets in real time.
Transforming the certificate data into APISIX's expected format (cert and key).
Automatically updating APISIX via its Admin API whenever a certificate is created, updated, or deleted.

Key Takeaways

Fully Automated Certificate Sync - No manual intervention needed for new or renewed certificates.

Real-Time Updates - APISIX stays in sync with Cert-Manager without delays.

Scalable & Kubernetes-Native - Works seamlessly with existing K8s tooling.

🚀 Need a Production-Ready Kubernetes Solution?

I've battle-tested this solution in my platform Laralord.dev where the base package includes fully automated provisioning of:

🚦 APISIX (API Gateway)
🔐 Cert-Manager (TLS Automation)
⚡ ArgoCD (GitOps)
...and more!

✨ Why Choose Laralord?

Feature	Benefit
Pre-Configured Automation	APISIX and Cert-Manager ready out-of-the-box - save dozens of configuration hours	⚙️
Simplified Deployments	Designed for Laravel but works with any app stack - perfect for SaaS/microservices	🚀
Enterprise-Grade Infrastructure	Built-in multi-tenancy, auto-scaling and security hardening	🔒

👉 Visit Laralord.dev to supercharge your Kubernetes workflows today!

📚 Further Reading

💬 Have questions or experiences to share?

Drop a comment below! 👇 We'd love to hear about your implementation!

DEV Community: Vitalii Liubimov