How to get an ALB running with kOps in 2021

Hayder Sharhan — Fri, 22 Jan 2021 19:34:27 +0000

Intro

Yesterday was a painful day because even though the folks who created https://github.com/kubernetes-sigs/aws-load-balancer-controller are great developers. They're really shit at documenting

I needed to use an ALB because ELBs don't support automatic http to https upgrades. And they have a ton more features that I'll use in the future. So in this guide I'll show you how to make your ALB do that auto redirect.

Getting your cluster ready

So let me guide you through the bs and save you 8+ hours of trial and error and intense research though github issues.

Please keep in mind my kubernetes version is ~1.15 so I need to use some legacy things. You might need to use some more recent configs.

The subnets

You need to make sure you have at least two subnets setup for your kubernetes cluster and these subnets need to be in different availability zones. Take a look at this for more information but basically you need to do kops edit cluster and make sure your subnets look something like this:

  subnets:
  - cidr: 172.20.32.0/19
    name: us-east-1a
    type: Public
    zone: us-east-1a
  - cidr: 172.20.64.0/19
    name: us-east-1b
    type: Public
    zone: us-east-1b

Then run kops update cluster --yes

The extra policies

For aws-load-balancer-controller to run properly, it needs extra policies to be attached to your cluster nodes so it can configure some things for you. To do that first we need to download this file which contains the policy description: curl -o iam-policy.json https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.1.0/docs/install/iam_policy.json

Then creating a policy for it:

aws iam create-policy \
    --policy-name AWSLoadBalancerControllerIAMPolicy \
    --policy-document file://iam-policy.json

Take note of the arn that gets returned.. something that looks like "aws:arn:iam:123456789000:policy:test-policy"

Now go edit you cluster again kops edit cluster and attach the policy like so:

spec:
  externalPolicies:
    node:
    - aws:arn:iam:123456789000:policy:test-policy

Now hit it with a kops update cluster --yes

Installing dependencies on your cluster

First you need to install cert manager which will allow you to manage certs for the different services and deployments that we'll be doing:

kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v1.1.0/cert-manager-legacy.yaml

Second installing the actual load balancer controller:

wget https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.1.0/docs/install/v2_1_0_full.yaml

open that file and go to where it says --cluster-name= and change it to your cluster's name.

save and run kubectl apply -f v2_1_0_full.yaml

Configuring the ALB

Make sure you have your deployment looking something like this:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app: my-app
  name: my-app-deployment
  namespace: dev
spec:
  replicas: 2
  selector:
    matchLabels:
      app.kubernetes.io/name: my-app
  template:
    metadata:
      labels:
        app.kubernetes.io/name: my-app
    spec:
      containers:
      - image: my-app:latest
        imagePullPolicy: Always
        name: my-app
        ports:
        - containerPort: 3000
          name: app-port
          protocol: TCP
      restartPolicy: Always

Now you need a service classic LoadBalancer (I think this can be a NodePort instead) to point to this port:

apiVersion: v1
kind: Service
metadata:
  namespace: dev
  name: my-app-service
spec:
  ports:
    - port: 3000
      targetPort: 3000
      protocol: TCP
  type: LoadBalancer
  selector:
    app.kubernetes.io/name: my-app

And the actual ALB ingress:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/certificate-arn: YOUR-ARN-CERT-HERE
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
    alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
  name: my-app-alb
  namespace: dev
spec:
  rules:
  - http:
      paths:
      - path: /*
        backend:
          serviceName: ssl-redirect
          servicePort: use-annotation
      - path: /*
        backend:
          serviceName: my-app-service
          servicePort: 3000

That's it! Run kubectl -n dev get ingress and go to that address. Enjoy it!