AWS ALB with NGINX INGRESS CONTROLLER

So if you are wondering how to integrate the AWS application load balancer with Nginx/Istio to get the traffic into the cluster so this is it.

You might ask why this article so let me tell you what I'm looking for when I say aws alb/nlb and ingress controller.

I know there are bunch of components missing but you got the idea right!! only 1 ALB/NLB unless u have some other use case where you need bunch of LB

LB: LoadBalancer; A: Application; N: Network
just to avoid any confusions.

Before moving forward let's discuss why this setup

Before

If you know Kubernetes(K8s) you know they support 3 type of networking, build natively we called them service

ClusterIP
NodePort
LoadBalancer

you can read more about them from K8s official docs
We are not interested in above 2 only LoadBalancer so what it does is that when you are using cloud services in our case EKS we can directly provision a loadbalancer for our services to ingress the traffic.
But the problem with this is how many LB we can create until we ran out of money coz we use many services and according to this there's going to be a lot of LB.

So you get the problem right we can't create LB everytime we need to let the traffic inside, so the great guys at K8s decided we need something and they came up with INGRESS the whole idea was to save cost and reduce management this is what I think 😁 coz it sure does for me!

AFTER

Moving on we can create the setup which showed in the starting only 1 LB is sufficient for K8s cluster we don't need to create different LB for different Services.

PROCESS

There are 4 stage in this from creating the cluster to deploying the application and then opening it in browser.

STAGE I

Spin Up the EKS cluster, you can do it from the aws console, cli. I love cli and there's a best tool to do this eksctl
Setup this tool you need aws cli configure plus you must have the permissions.

eksctl create cluster -f cluster.yaml

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
  name: DevOps-Prod
  region: ap-south-1
  version: "1.26"
managedNodeGroups:
  - name: ng-1
    instanceType: m5.large
    desiredCapacity: 2
    volumeSize: 80
    labels: { role: workers }

Give it 15mins or so it will complete the creation and the cluster will be up then

STAGE II

Now we'll install something called AWS LoadBalancer Controller.
I'm not going into details coz AWS blogs are great you just need to follow them here's the link
TIP: I prefer the eksctl

STAGE III

Make sure the pods are running by running the command
kubectl get po -n kube-system
If your namespace is kube-system

Now we need to install the Nginx Ingress Controller again not going into deep here's the link

So we need to do some changes over here to achieve our noble goal of having 1 LB
TIP: Install it with Helm
make sure you change the service type of chart from LoadBalancer to NodePort.
You find it in the values.yaml or if you're using manifest file the look for the service block just make sure you do this.

NOTICE: It's important that you do this otherwise it won't work

Now after successfully deploying this you need to check which NodePort is being utilized it's mostly in above 3000 make sure you wrote it down somewhere.
And check the readiness-probe for the deployment of nginx controller

readinessProbe:
failureThreshold: 30
httpGet:
path: /healthz

something like this check it for yourself and now we are going to last phase.

STAGE IV

After everything is deployed and you have the NodePort and Path of the health-check you going to apply the below file.

You also need the certificate imported in a service called ACM
Amazon Certificate Manager from which you will get the arn of that certificate which you need to update in the file

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: alb-ingress-connect-nginx
  namespace: kube-system
  annotations:
    # Ingress Core Settings
    kubernetes.io/ingress.class: "alb"
    alb.ingress.kubernetes.io/scheme: internet-facing
    # Health Check Settings
    alb.ingress.kubernetes.io/healthcheck-protocol: HTTP
    alb.ingress.kubernetes.io/healthcheck-port: 30000 # Make sure you add the NodePort over here
    alb.ingress.kubernetes.io/healthcheck-path: /healthz # Put the path of readiness probe over here
    alb.ingress.kubernetes.io/healthcheck-interval-seconds: '15'
    alb.ingress.kubernetes.io/healthcheck-timeout-seconds: '5'
    alb.ingress.kubernetes.io/success-codes: '200'
    alb.ingress.kubernetes.io/healthy-threshold-count: '2'
    alb.ingress.kubernetes.io/unhealthy-threshold-count: '2'
    ## SSL Settings
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}, {"HTTP":80}]'
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:eu-west-2:122221113322:certificate/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx # make sure you update your certificate arn over here
    #alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-TLS-1-1-2017-01 #Optional (Picks default if not used)
    # redirect all HTTP to HTTPS
    alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
spec:
  rules:
  - http:
      paths:
        - path: /*
          pathType: ImplementationSpecific
          backend:
            service:
              name: ssl-redirect
              port:
                name: use-annotation
        - path: /*
          pathType: ImplementationSpecific
          backend:
            service:
              name: ingress-nginx-controller # Make sure you name the service correctly by checking the name of it nginx ingress controller service nothing else
              port:
                number: 80

If you open the above file in vscode or text-editor you'll find the comment that I wrote for you guys where you need to supply the NodePort and Health-check path and name of the service of ingress controller, arn of the certificate after doing all the changes apply with following

kubectl apply -f ingress.yaml

After running this command you'll see that there's a ALB is provisioning and target group is created to check the instance health check once it's checked and alb is fully provisioned you can hit the url of the alb see the 404 NOT FOUND from nginx
and this confirm that response is coming from nginx.

ENDING

You can deploy the application and expose the service with ingress and try to hit the url mentioned in the ingress file before this you need to update the records too if using ROUTE53 or any other domain registrar and that's it

CONCLUSION

I hope this works for you if not plz reach out to me I'm happy to help make sure the to update the values which I commented and it will work for you.

Comments

[alon-cheq]

Hi Ashutosh,

When getting to deploy the application - I can't make it work, getting 404 error on the nginx default backend.
Any example for the app ingress.yaml how to point the nginx internal ingress to the app backend?

Best,
Alon

[rhaskins]

When I attempt to deploy the Stage IV "ingress.yaml" manifest, I am getting the following error: 'error: unable to decode "ingress.yaml": json: cannot unmarshal number into Go struct field ObjectMeta.metadata.annotations of type string'. Any ideas on what I am doing wrong?

[rhaskins]

I figured this out -- every value in the annotations needs to be quoted.

[Thức Phạm]

this not work