Configure Autoscaler on EKS

Overview
Amazon EKS is a fully managed service, kubernetes allows the scalability to adjust to sudden traffic changes seamlessly. Cluster Autoscaler automatically adjusts the number of nodes in your cluster when pods fail or are rescheduled onto other nodes. The Cluster Autoscaler is typically installed as a Deployment in your cluster. It uses leader election to ensure high availability, but scaling is done by only one replica at a time.

Prerequisites

1. An existing Amazon EKS cluster
2. An existing IAM OIDC provider for your cluster. Determine whether you have one or need to create one.
3. Node groups with Auto Scaling groups tags

Create IAM policy and role
Create an IAM policy. Paste the following content into a file named cluster-autoscaler-policy.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "autoscaling:SetDesiredCapacity",
                "autoscaling:TerminateInstanceInAutoScalingGroup"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/k8s.io/cluster-autoscaler/my-cluster": "owned"
                }
            }
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "autoscaling:DescribeAutoScalingInstances",
                "autoscaling:DescribeAutoScalingGroups",
                "ec2:DescribeLaunchTemplateVersions",
                "autoscaling:DescribeTags",
                "autoscaling:DescribeLaunchConfigurations"
            ],
            "Resource": "*"
        }
    ]
}

then create the policy with the following command. You can change the value for policy-name.

aws iam create-policy \
--policy-name AmazonEKSClusterAutoscalerPolicy \
--policy-document file://cluster-autoscaler-policy.json

then attach the above policy as shown:

eksctl create iamserviceaccount \
  --cluster=<YOUR CLUSTER NAME> \
  --namespace=kube-system \
  --name=cluster-autoscaler \
--attach-policy-arn=arn:aws:iam::ACCOUNT_ID:policy/ClusterAutoscaler-autoDiscovery \
  --override-existing-serviceaccounts \
  --approve

Deploy cluster autoscaler
To deploy the Cluster Autoscaler:

Download the Cluster Autoscaler YAML file.

wget curl -o cluster-autoscaler-autodiscover.yaml https://raw.githubusercontent.com/kubernetes/autoscaler/master/cluster-autoscaler/cloudprovider/aws/examples/cluster-autoscaler-autodiscover.yaml

Modify the YAML file and replace with your cluster name. Also consider replacing the cpu and memory values as determined by your environment.

Apply the YAML file to your cluster.

kubectl apply -f cluster-autoscaler-autodiscover.yaml

Annotate the cluster-autoscaler service account with the ARN of the IAM role that you created previously. Replace the example values with your own values.

kubectl annotate serviceaccount cluster-autoscaler \
 -n kube-system \  eks.amazonaws.com/role-arn=arn:aws:iam::ACCOUNT_ID:role/AmazonEKSClusterAutoscalerRole

Patch the deployment to add the cluster-autoscaler.kubernetes.io/safe-to-evict annotation to the Cluster

kubectl annotate serviceaccount cluster-autoscaler \
  -n kube-system \
  eks.amazonaws.com/role-arn=arn:aws:iam::ACCOUNT_ID:role/AmazonEKSClusterAutoscalerRole

Edit the Cluster Autoscaler deployment with the following command.

kubectl -n kube-system edit deployment.apps/cluster-autoscaler

Add the following lines as shown

--balance-similar-node-groups
--skip-nodes-with-system-pods=false

Find the latest Cluster Autoscaler version here that matches the Kubernetes major and minor version of your cluster.

Set the Cluster Autoscaler image tag to the version that you recorded in the previous step with the following command.

kubectl set image deployment cluster-autoscaler \
  -n kube-system \
  cluster-autoscaler=k8s.gcr.io/autoscaling/cluster-autoscaler:v1.23.0

View your Cluster Autoscaler logs

kubectl -n kube-system logs -f deployment.apps/cluster-autoscaler

Read more here

Conclusion
EKS platform is eminently scalable across multiple change vectors. Cluster autoscaler handles change in expected workload behavior situations or unexpected scenarios while meeting business needs.