DEV Community: Luke Livingstone

Hashicorp Vault at Super

Luke Livingstone — Thu, 22 May 2025 14:41:55 +0000

At Super, we use HashiCorp Vault to securely store the secrets required by our microservices running on Kubernetes.

We’ve been long-time fans of Vault. Our Platform team has previous experience deploying and maintaining it, so choosing Vault for our current setup was an easy decision from a knowledge and reliability standpoint.

Drawing on lessons from past implementations, we were able to build something robust and scalable. Our infrastructure is hosted entirely on AWS and is segmented across multiple accounts. We maintain three separate workload accounts, Staging, Mock, and Production each running Super's microservices in Kubernetes along side a Infrastructure account, for Platform tooling.

Rather than deploying and maintaining a separate Vault cluster for each environment, we opted for a centralised approach. This decision reduced operational overhead and significantly improved the developer experience, avoiding the complexity of managing and switching between multiple Vault interfaces.

To get started, we deployed our Vault infrastructure via Terraform. Vault’s storage backend is powered by Amazon S3, with DynamoDB providing high availability. We also use AWS KMS for auto-unseal functionality, eliminating the need for manual intervention when restarting Vault. Vault itself is installed using the official HashiCorp Helm chart.

Next, we provisioned an internal Network Load Balancer (NLB) and exposed it through a VPC Endpoint Service. This design choice enables secure, cross-account connectivity to Vault using VPC Interface Endpoints—avoiding the complexity and security risks of VPC peering.

To simplify service discovery within our Kubernetes clusters, we created human-readable internal services that resolve super.vault to the appropriate VPC interface endpoint. This gives our services a clean and consistent way to talk to Vault, regardless of the environment they’re running in.

That wraps up our simple yet effective centralized Vault infrastructure here at Super. By consolidating our setup, we've kept operations streamlined, secure, and developer-friendly across all environments.

If you're interested in hearing more or want us to dive deeper into any aspect of our Vault implementation—be it authentication flows, secret injection, or scaling—feel free to reach out. We'd love to share more!

Karpenter on EKS Fargate

Luke Livingstone — Wed, 17 Apr 2024 15:04:30 +0000

We're reinventing payments. Super powers free payments for businesses and more rewarding shopping for customers, so that everyone wins. https://www.superpayments.com/

We're using Karpenter to manage our Kubernetes node scaling.

We're big fans of how fast Karpenter can provision just-in-time nodes for us across our EKS clusters but there was one sticking point, for obvious reasons the Karpenter controller pods can't run on Karpenter managed nodes.

To get around this we used AWS EKS managed node groups as init nodes and pinned Karpenter to said nodes. We provisioned a node group with a minimum and maximum of 2 nodes for Karpenter mostly (although other pods could run on these nodes too, to avoid wasting compute resources!)

The downside is that updating managed node groups is slow; updating two nodes, with a maximum of one available at a time, took between six and ten minutes, and we wanted to speed up this process.

The simple solution? Remove the init nodes! But then, where do we run Karpenter? Enter Fargate.

We created an EKS Fargate profile via our EKS Terraform module with a selector for the Karpenter namespace:

resource "aws_eks_fargate_profile" "karpenter" {
  cluster_name           = aws_eks_cluster.cluster.name
  fargate_profile_name   = "karpenter"
  pod_execution_role_arn = aws_iam_role.fargate.arn
  subnet_ids             = var.private_subnets

  selector {
    namespace = "karpenter"
  }
}

Pod Execution Role

If you've ever used ECS, you'll be familiar with the pod execution role. For Fargate and EKS, it's a straightforward role with two AWS managed policies attached:

resource "aws_iam_role" "fargate" {
  name = "${var.cluster_name}-fargate"

  assume_role_policy = jsonencode({
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "eks-fargate-pods.amazonaws.com"
      }
    }]
    Version = "2012-10-17"
  })
}

resource "aws_iam_role_policy_attachment" "fargate" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSFargatePodExecutionRolePolicy"
  role       = aws_iam_role.fargate.name
}

resource "aws_iam_role_policy_attachment" "fargate_eks_cni" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
  role       = aws_iam_role.fargate.name
}

Karpenter Helm Install

We use the hashicorp/helm Terraform provider to install both the Karpenter and CRD charts directly from our EKS module. This ensures that Karpenter is up and running before anything else, ready to provision compute.

Next, we set the namespace for the Karpenter chart to match the selector in the Fargate profile, which in our case is karpenter, and we're off to the races!

NAME                         READY   STATUS    RESTARTS   AGE   IP              NODE                                                  NOMINATED NODE 
karpenter-75c664b7cb-9z9lr   1/1     Running   0          5d    <snip>   fargate-<snip>.eu-west-2.compute.internal   <none>           <none>
karpenter-75c664b7cb-fxhb2   1/1     Running   0          5d    <snip>    fargate-<snip>.eu-west-2.compute.internal

Notes

By default we're using Fargate's minimum resources which are 0.25 vCPU and 0.5GB RAM per task.

Currently you can't specify ARM when creating Fargate tasks on EKS so we're currently using x86 but the cost is around $20 per month for both tasks.

We've generally reduced the number of nodes across our EKS clusters too, resulting in some cost savings but much less waiting around for the Platform team!

Terraform Modules at Super

Luke Livingstone — Thu, 28 Mar 2024 12:30:55 +0000

We're reinventing payments. Super powers free payments for businesses and more rewarding shopping for customers, so that everyone wins. https://www.superpayments.com/

Like most startups, we use Terraform to manage and deploy our infrastructure. This post covers how we use Terraform modules at Super to adhere to the DRY principle.

Early in our Terraform refactor, we aimed to invest in modules. Our goal was to promote high reusability while minimising code.

At the time of writing, Super has around 70 Terraform modules in use across 10 providers. Some of the modules are small (e.g. IAM Role) and some are larger (e.g. EKS Cluster).

Template Module & Code Style 📝

In order to keep module creation in line with a style guide we have a template module. Some of the rules below are best practice and some are specific to Super.

We don't include provider configurations
We don't include any backend configuration
data.tf file is used for all data resources
outputs.tf file is used for all output resources
variables.tf file is used for all variables
versions.tf file is used for required_providers and required_version

Why no provider?! 😱

The primary reason we avoid including a provider in our modules is to facilitate nesting modules. Nesting modules can be beneficial to keep resources used in a standardised format across modules.

When using a module inside of a module Terraform deems it incompatible with count, for_each, and depends_on if the module in question has its own local provider configuration.

We started out only removing the providers of modules nested, but decided that we can make use of Terragrunt's generate and include to remove providers from all modules.

Let's take the following directory structure for AWS as an example. We have a folder for the AWS region (eu-west-2) and we also have a few hcl files.

├── super-staging
│   ├── eu-west-2
│   ├── aws.hcl
│   ├── terragrunt.hcl
│   └── vault.hcl

The aws.hcl file uses a Terragrunt generate block to arbitrarily generate a file in the terragrunt working directory (where terraform is called).

generate "aws" {
  path      = "aws.tf"
  if_exists = "overwrite_terragrunt"
  contents  = <<EOF
provider "aws" {
  region = "eu-west-2"
  default_tags {
    tags = {
      environment = "staging",
    }
  }
}
EOF
}

When using a module with Terragrunt you can then use the include block with the find_in_parent_folders function.

include "aws" {
  path   = find_in_parent_folders("aws.hcl")
}

terraform {
  source = "git@github.com:organisation/terraform-example-module.git?ref=v1.0.0"
}

Remote State

We use S3 as our state store along with DynamoDB for locking all encrypted with KMS.

The terragrunt.hcl at the root of the directory includes three things. The terragrunt remote_state block, iam_role and some default inputs.

remote_state {
  backend = "s3"

  generate = {
    path      = "backend.tf"
    if_exists = "overwrite"
  }

  config = {
    bucket                = "super-staging-eu-west-2-example-bucket"
    key                   = "${path_relative_to_include()}/terraform.tfstate"
    region                = "eu-west-2"
    encrypt               = true
    dynamodb_table        = "super-staging-eu-west-2-example-table"
    kms_key_id            = "alias/s3-super-staging-eu-west-2-example-kms"
    disable_bucket_update = true
  }
}

iam_role = "arn:aws:iam::<snip>:role/example-role"

inputs = {
  environment    = "staging"
  aws_account_id = "<snip>"
  service_owner  = "devops"
}

We then add the include like we do with the AWS provider. By default find_in_parent_folders will search for the first terragrunt.hcl file.

include "root" {
  path   = find_in_parent_folders()
  expose = true
}

Versioning 🔢

Our Platform team are enthusiasts of semantic versioning and we also use conventional commits.

We have a simple Github Action job on each module repository that uses the semantic-release-action. We use the @semantic-release/commit-analyzer plugin with the conventionalcommits preset.

      - name: Release
        uses: cycjimmy/semantic-release-action@v4
        with:
          semantic_version: 23.0.2
          extra_plugins: |
            @semantic-release/changelog@6.0.3
            @semantic-release/git@10.0.1
            conventional-changelog-conventionalcommits@7.0.2
        env:
          GITHUB_TOKEN: ${{ secrets.CI_GITHUB_TOKEN }}