1. Introduction
Overview:
This SOP explains the purpose, scope, and best practices for setting up a ALB Ingress Controller.The AWS Load Balancer Controller manages AWS Elastic Load Balancers for a Kubernetes cluster. You can use the controller to expose your cluster apps to the internet. The controller provisions AWS load balancers that point to cluster Service or Ingress resources. In other words, the controller creates a single IP address or DNS name that points to multiple pods in your cluster.
The controller watches for Kubernetes Ingress or Service resources. In response, it creates the appropriate AWS Elastic Load Balancing resources. You can configure the specific behavior of the load balancers by applying annotations to the Kubernetes resources. For example, you can attach AWS security groups to load balancers using annotations.
Objective:
The objective of this document is to provide a comprehensive guide for implementing the AWS ALB Ingress Controller on AWS. This setup will enable efficient routing and load balancing of incoming traffic to Kubernetes services running on an Amazon Elastic Kubernetes Service (EKS) cluster using Application Load Balancers.
Key Components:
AWS ALB Ingress Controller:
The AWS ALB Ingress Controller is a Kubernetes resource that manages AWS Application Load Balancer configuration to route incoming traffic to Kubernetes services.
Amazon EKS:
Amazon Elastic Kubernetes Service is a managed Kubernetes service provided by AWS that simplifies the process of deploying, managing, and scaling containerized applications using Kubernetes.
Amazon Route 53:
Amazon Route 53 is a scalable and highly available Domain Name System (DNS) web service provided by AWS. It enables routing traffic to AWS resources, including the AWS ALB Ingress Controller.
AWS Application Load Balancer (ALB):
ALB automatically distributes incoming application traffic across multiple targets, such as EC2 instances, containers, and IP addresses, within one or more Availability Zones.
Prerequisites:
Before proceeding with the implementation, ensure the following prerequisites are met:
AWS Account:
Access to an AWS account with permissions to create and manage resources such as EKS clusters, Route 53 records, and Application Load Balancers.
Kubernetes Cluster:
An Amazon EKS cluster should be provisioned and running. Ensure the cluster is properly configured with networking, IAM roles, and necessary node groups.
kubectl CLI:
Install and configure the kubectl command-line tool to interact with the Kubernetes cluster.
Helm (Optional):
If using Helm for deploying the AWS ALB Ingress Controller, ensure Helm is installed and configured.
Access to DNS:
Have access to manage DNS records, as you will need to create DNS records to route traffic to the AWS ALB Ingress Controller.
Procedure
Step Create IAM Role using eksctl
Create an IAM policy.
Download an IAM policy for the AWS Load Balancer Controller that allows it to make calls to AWS APIs on your behalf.
2.Create an IAM policy using the policy downloaded in the previous step.
aws iam create-policy \
--policy-name AWSLoadBalancerControllerIAMPolicy \
--policy-document file://iam_policy.json
Create IAM Role using eksctl
Replace my-cluster with the name of your cluster, 111122223333 with your account ID, and then run the command. If your cluster is in the AWS GovCloud (US-East) or AWS GovCloud (US-West) AWS Regions, then replace arn:aws: with arn:aws-us-gov:.
eksctl create iamserviceaccount \
--cluster=my-cluster \
--namespace=kube-system \
--name=aws-load-balancer-controller \
--role-name AmazonEKSLoadBalancerControllerRole \
--attach-policy-arn=arn:aws:iam::111122223333:policy/AWSLoadBalancerControllerIAMPolicy \
--approve
eksctl create iamserviceaccount \
--cluster=demo-cluster \
--namespace=kube-system \
--name=aws-load-balancer-controller \
--role-name AmazonEKSLoadBalancerControllerRole \
--attach-policy-arn=arn:aws:iam::975050347443:policy/AWSLoadBalancerControllerIAMPolicy \
--region=us-east-1
--approve
The above error is came due to OIDC is not enabled
Commands to configure IAM OIDC provider
export cluster_name=demo-cluster
oidc_id=$(aws eks describe-cluster --name $cluster_name --query "cluster.identity.oidc.issuer" --output text | cut -d '/' -f 5)
Check if there is an IAM OIDC provider configured already
eksctl utils associate-iam-oidc-provider --cluster $cluster_name --region us-east-1 --approve
Step Install AWS Load Balancer Controller
Install AWS Load Balancer Controller using Helm V3:
Add the eks-charts Helm chart repository. AWS maintains this repository on GitHub.
helm repo add eks https://aws.github.io/eks-charts
Update your local repo to make sure that you have the most recent charts.
helm repo update eks
To view the available versions of the Helm Chart and Load Balancer Controller, use the following command:
helm search repo eks/aws-load-balancer-controller --versions
3.Install the AWS Load Balancer Controller.
Replace my-cluster with the name of your cluster. In the following command, aws-load-balancer-controller is the Kubernetes service account that you created in a previous step.
For more information about configuring the helm chart, see values.yaml on GitHub.
Ref Link:
https://github.com/aws/eks-charts/blob/master/stable/aws-load-balancer-controller/values.yaml
helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
-n kube-system \
--set clusterName=my-cluster \
--set serviceAccount.create=false \
--set serviceAccount.name=aws-load-balancer-controller
If you're deploying the controller to Amazon EC2 nodes that have restricted access to the Amazon EC2 instance metadata service (IMDS), or if you're deploying to Fargate, then add the following flags to the helm command that follows:
--set region=region-code
--set vpcId=vpc-xxxxxxxx
helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
-n kube-system \
--set clusterName=demo-cluster \
--set serviceAccount.create=false \
--set serviceAccount.name=aws-load-balancer-controller \
--set autoDiscoverAwsRegion=true \
--set autoDiscoverAwsVpcID=true
--set region=us-east-1 \
--set vpcId=vpc-08ce046624ab1b564
Step Verify that the controller is installed
Verify that the controller is installed.
kubectl get deployment -n kube-system aws-load-balancer-controller
Step Deploy a sample application
Prerequisites:
At least one public or private subnet in your cluster VPC.
Have the AWS Load Balancer Controller deployed on your cluster. For more information, see What is the AWS Load Balancer Controller?. We recommend version 2.7.2 or later.
Deploy the game 2048 as a sample application.
To verify that the AWS Load Balancer Controller creates an AWS ALB as a result of the ingress object.
Complete the steps for the type of subnet you're deploying to.
If you're deploying to Pods in a cluster that you created with the IPv6 family, skip to the next step.
If Public execute below command
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.7.2/docs/examples/2048/2048_full.yaml
If Private follow below steps
Download the manifest.
Edit the file and find the line that says alb.ingress.kubernetes.io/scheme: internet-facing.
Change internet-facing to internal and save the file.
Apply the manifest to your cluster.
kubectl apply -f 2048_full.yaml
If you're deploying to Pods in a cluster that you created with the IPv6 family, complete the following steps.
Download the manifest.
Open the file in an editor and add the following line to the annotations in the ingress spec.
alb.ingress.kubernetes.io/ip-address-type: dualstack
If you're load balancing to internal Pods, rather than internet facing Pods,
Change the line that says
alb.ingress.kubernetes.io/scheme: internet-facing to alb.ingress.kubernetes.io/scheme: internal
Save the file.
Apply the manifest to your cluster.
kubectl apply -f 2048_full.yaml
After a few minutes, verify that the ingress resource was created with the following command.
For testing here i have used public
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.7.2/docs/examples/2048/2048_full.yaml
Check the deployed application is working properly or not.
kubectl get all -n game-2048
Check the ingress
kubectl get ingress -n game-2048
Check the application using the endpoint
Upto now it is completed ACM is in progress
Step Create a ACM certifiacte with the required domain.
Sign in to the AWS Management Console and open the ACM console
Choose Request a certificate.
In the Domain names section, type your domain name.
Validate the Certificate by adding the records in DNS.
Add the records in Route53
Updating the Controller with ACM
helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
-n kube-system \
--set clusterName=demo-cluster \
--set serviceAccount.create=false \
--set serviceAccount.name=aws-load-balancer-controller \
--set autoDiscoverAwsRegion=true \
--set autoDiscoverAwsVpcID=true \
--set region=us-east-1 \
--set vpcId=vpc-08ce046624ab1b564 \
--set enableShield=false \ # optional: enable AWS Shield Advanced for the ALB
--set enableWaf=false \ # optional: enable AWS WAF (Web Application Firewall) for the ALB
--set acm.enabled=true \ # enable ACM integration
--set acm.defaultRegion=us-east-1 \ # specify the default region for ACM
--set acm.managed=false \ # set to false as you're providing the certificate ARN
--set acm.certArn=arn:aws:acm:us-west-2:XXXXXXXX:certificate/XXXXXX-XXXXXXX-XXXXXXX-XXXXXXXX
Step Final Testing Phase with ACM by deploying Prom Stack.
Testing the ingress deploy prom-stack
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
helm install prometheus prometheus-community/kube-prometheus-stack --create-namespace -n monitoring
Ingreess rules creation
Create a ingress file for prometheus
vi prometheus-ingress.yaml
Replace the Service and port of the related service.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/proxy-body-size: 5m
name: prod-prom-grafana-monitoring
namespace: monitoring
spec:
rules:
- host: grafana-prod-mumbai.illusto.com
http:
paths:
- backend:
service:
name: prom-stack-grafana
port:
number: 80
path: /
pathType: ImplementationSpecific
- host: prom-prod-mumbai.illusto.com
http:
paths:
- backend:
service:
name: prom-stack-kube-prometheus-prometheus
port:
number: 9090
path: /
pathType: ImplementationSpecific
- host: alert-prod-mumbai.illusto.com
http:
paths:
- backend:
service:
name: prom-stack-kube-prometheus-alertmanager
port:
number: 9093
path: /
pathType: ImplementationSpecific
Create the ingress rules
k apply -f prometheus-ingress.yaml
Check the ingress.
k get ingress -n monitoring
Add the Route53 records:
Grafana Dashboard:
Prometheus Dashboard:
AlertManager Dashboard:
Scope
This SOP covers the deployment and management of the AWS Load Balancer Controller in a Kubernetes cluster. It is intended for use in exposing cluster applications to the internet through AWS Elastic Load Balancers. The document applies to system administrators, DevOps engineers, and any team members responsible for maintaining Kubernetes clusters on AWS. It also includes best practices for managing Ingress resources and configuring load balancers to meet security and scalability requirements.
Roles and Responsibilities
DevOps Engineers:
- Ensure the AWS Load Balancer Controller is installed and configured correctly in the Kubernetes cluster.
- Monitor load balancer performance and ensure compliance with the organization’s network and security policies.
- Define and apply Kubernetes Ingress or Service resources with proper annotations for load balancer configuration.
- Automate the deployment of load balancers using Infrastructure as Code (IaC) tools like Terraform or Helm.
- Configure AWS security groups to allow necessary traffic to and from the load balancers.
- Verify that DNS names and IP addresses assigned to the load balancers are correctly pointing to the Kubernetes resources.
- Ensure the setup adheres to security standards and regulatory requirements, including data protection and access control policies.
Enforcement
This SOP is enforced by regular audits of the Kubernetes and AWS infrastructure to ensure compliance with defined configurations. Automated monitoring tools, such as Prometheus, should be used to track load balancer health, traffic patterns, and security group configurations. Any deviations from the established guidelines should be logged, reported, and addressed promptly.
Conclusion
Implementing the AWS Load Balancer Controller is a critical step in securely and efficiently managing application traffic in a Kubernetes cluster. By following the practices outlined in this SOP, organizations can ensure reliable application delivery while maintaining scalability and security. Proper configuration and monitoring of the controller and associated resources will significantly reduce the risks of downtime, unauthorized access, or performance bottlenecks.
Ref Links
https://docs.aws.amazon.com/eks/latest/userguide/aws-load-balancer-controller.html
https://github.com/kubernetes-sigs/aws-load-balancer-controller
https://docs.aws.amazon.com/eks/latest/userguide/lbc-helm.html
https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html
https://github.com/aws/eks-charts/blob/master/stable/aws-load-balancer-controller/values.yaml
https://docs.aws.amazon.com/eks/latest/userguide/lbc-manifest.html
https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html
https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.7/guide/ingress/annotations/
Top comments (4)
Excellent work
Great work
well done
Excellent Work