Deploy linkerd to Kubernetes using Helm and Terraform

So I've been trying to install Linkerd service mesh on our AKS clusters this week. There are a few options to install: using their command line utility or using Helm.

Command line tool works well, it will generate CA and issuer certificates for you. However, you have to download it when deploying via a pipeline, and there's a small problem with running linkerd check --pre - it will only succeed on first installation and fail after linkerd is installed.

So I chose to install it with Helm, along with linkerd-cni and linkerd viz dashboard. You can find all their charts here. I don't use Helm command line in pipelines, preferring to use Terraform helm provider. This is because I can install and configure many Helm releases using a single set of Terraform commands.

Generating CA and issuer certs
You need to generate your own CA and issuer certificates if you choose to install linkerd with Helm. They give you instructions on how to generate them using step command line utility. However, we can also generate them using Terraform tls provider.

resource "tls_private_key" "ca" {
  algorithm   = "ECDSA"
  ecdsa_curve = "P256"
}

resource "tls_self_signed_cert" "ca" {
  private_key_pem       = tls_private_key.ca.private_key_pem
  is_ca_certificate     = true
  set_subject_key_id    = true
  validity_period_hours = 87600
  allowed_uses = [
    "cert_signing",
    "crl_signing"
  ]
  subject {
    common_name = "root.linkerd.cluster.local"
  }
}

resource "tls_private_key" "issuer" {
  algorithm   = "ECDSA"
  ecdsa_curve = "P256"
}

resource "tls_cert_request" "issuer" {
  private_key_pem = tls_private_key.issuer.private_key_pem
  subject {
    common_name = "identity.linkerd.cluster.local"
  }
}

resource "tls_locally_signed_cert" "issuer" {
  cert_request_pem      = tls_cert_request.issuer.cert_request_pem
  ca_private_key_pem    = tls_private_key.ca.private_key_pem
  ca_cert_pem           = tls_self_signed_cert.ca.cert_pem
  is_ca_certificate     = true
  set_subject_key_id    = true
  validity_period_hours = 8760
  allowed_uses = [
    "cert_signing",
    "crl_signing"
  ]
}

Configuring linkerd Helm releases

Now all I need is to add 3 helm_release resources for linkerd CNI, linkerd and linkerd viz dashboard, providing outputs of certificate resources:

resource "helm_release" "linkerd_cni" {
  name             = "linkerd-cni"
  namespace        = "linkerd-helm"
  repository       = "https://helm.linkerd.io/stable"
  chart            = "linkerd2-cni"
  version          = "2.11.4"
  create_namespace = true
}

resource "helm_release" "linkerd" {
  name             = "linkerd"
  namespace        = helm_release.linkerd_cni.namespace
  repository       = "https://helm.linkerd.io/stable"
  chart            = "linkerd2"
  version          = "2.11.4"
  create_namespace = false
  set {
    name  = "cniEnabled"
    value = "true"
  }
  set {
    name  = "identityTrustAnchorsPEM"
    value = tls_locally_signed_cert.issuer.ca_cert_pem
  }
  set {
    name  = "identity.issuer.tls.crtPEM"
    value = tls_locally_signed_cert.issuer.cert_pem
  }
  set {
    name  = "identity.issuer.tls.keyPEM"
    value = tls_private_key.issuer.private_key_pem
  }
}

resource "helm_release" "linkerd_viz" {
  name             = "linkerd-viz"
  namespace        = helm_release.linkerd_cni.namespace
  repository       = "https://helm.linkerd.io/stable"
  chart            = "linkerd-viz"
  version          = "2.11.4"
  create_namespace = false
}

Namespaces

Helm needs a namespace to create the release in, but you cannot pre-create default linkerd namespaces. Their Helm chart deploys and configures them with appropriate annotations. Instead I create tell Helm resource to create a new namespace called linkerd-helm which houses only Helm releases but no linkerd resources. I also use implicit dependency on helm_release_linkerd_cni.namespace so CNI is deployed before linkerd and viz dashboard.

Once I run through the standard terraform init, plan, apply I end up with this:

kubectl get namespaces
NAME                       STATUS   AGE
<truncated>
linkerd                    Active   2d19h
linkerd-cni                Active   2d19h
linkerd-helm               Active   2d19h
linkerd-viz                Active   37h

helm ls --namespace linkerd-helm
NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                   APP VERSION
linkerd         linkerd-helm    4               2022-07-29 09:59:38.7045481 +0000 UTC   deployed        linkerd2-2.11.4         stable-2.11.4
linkerd-cni     linkerd-helm    1               2022-07-28 14:40:00.021646641 +0000 UTC deployed        linkerd2-cni-2.11.4     stable-2.11.4
linkerd-viz     linkerd-helm    1               2022-07-29 20:26:34.495285272 +0000 UTC deployed        linkerd-viz-2.11.4      stable-2.11.4

P.S. Caveat for Azure Kubernetes Service: Do not use Open Service Mesh add-on if you want to install linkerd. It kept crashing for me.