Karpenter is an open-source, high-performance Kubernetes cluster autoscaler developed by AWS. Amazon Elastic Kubernetes Service (EKS) provides a powerful and flexible platform for running containerized applications.
Step 1 : Create an IAM Role
Role Name : KarpenterNodeRole-santhosh"
KarpenterNodeRole-santhosh"
Role attached below policy:
AmazonEKSWorkerNodePolicy
AmazonEKS_CNI_Policy
AmazonEC2ContainerRegistryReadOnly
AmazonSSMManagedInstanceCore
Step 2 : Create a Controller IAM Role
Role Name : KarpenterControllereRole-santhosh
Modify Trust relationships
Add OIDC
Account ID
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::${AWS_ACCOUNT_ID}:oidc-provider/oidc.eks.ap-south-1.amazonaws.com/id/6B407ED9BFC9CE681546033D7AD4156A"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"oidc.eks.ap-south-1.amazonaws.com/id/6B407ED9BFC9C46033D7AD4156A:aud": "sts.amazonaws.com",
"oidc.eks.ap-south-1.amazonaws.com/id/6B407ED9BFC9CE633D7AD4156A:sub": "system:serviceaccount:karpenter:karpenter"
}
}
}
]
}
Step 3: Create a KarpenterControllerPolicy-santhosh" policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Karpenter",
"Effect": "Allow",
"Action": [
"ssm:GetParameter",
"ec2:DescribeImages",
"ec2:RunInstances",
"ec2:DescribeSubnets",
"ec2:DescribeSecurityGroups",
"ec2:DescribeLaunchTemplates",
"ec2:DescribeInstances",
"ec2:DescribeInstanceTypes",
"ec2:DescribeInstanceTypeOfferings",
"ec2:DeleteLaunchTemplate",
"ec2:CreateTags",
"ec2:CreateLaunchTemplate",
"ec2:CreateFleet",
"ec2:DescribeSpotPriceHistory",
"pricing:GetProducts"
],
"Resource": "*"
},
{
"Sid": "ConditionalEC2Termination",
"Effect": "Allow",
"Action": "ec2:TerminateInstances",
"Resource": "*",
"Condition": {
"StringLike": {
"ec2:ResourceTag/karpenter.sh/nodepool": "*"
}
}
},
{
"Sid": "PassNodeIAMRole",
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "arn:aws:iam::961489959441:role/MF-MS-PRE-PROD-CLUSTER-NodeInstanceRole"
},
{
"Sid": "EKSClusterEndpointLookup",
"Effect": "Allow",
"Action": "eks:DescribeCluster",
"Resource": "arn:aws:eks:ap-south-1:961489959441:cluster/MF-MS-PRE-PROD-CLUSTER"
},
{
"Sid": "AllowScopedInstanceProfileCreationActions",
"Effect": "Allow",
"Action": [
"iam:CreateInstanceProfile"
],
"Resource": "*",
"Condition": {
"StringEquals": {
"aws:RequestTag/kubernetes.io/cluster/MF-MS-PRE-PROD-CLUSTER": "owned",
"aws:RequestTag/topology.kubernetes.io/region": "ap-south-1"
},
"StringLike": {
"aws:RequestTag/karpenter.k8s.aws/ec2nodeclass": "*"
}
}
},
{
"Sid": "AllowScopedInstanceProfileTagActions",
"Effect": "Allow",
"Action": [
"iam:TagInstanceProfile"
],
"Resource": "*",
"Condition": {
"StringEquals": {
"aws:ResourceTag/kubernetes.io/cluster/MF-MS-PRE-PROD-CLUSTER": "owned",
"aws:ResourceTag/topology.kubernetes.io/region": "ap-south-1",
"aws:RequestTag/kubernetes.io/cluster/MF-MS-PRE-PROD-CLUSTER": "owned",
"aws:RequestTag/topology.kubernetes.io/region": "ap-south-1"
},
"StringLike": {
"aws:ResourceTag/karpenter.k8s.aws/ec2nodeclass": "*",
"aws:RequestTag/karpenter.k8s.aws/ec2nodeclass": "*"
}
}
},
{
"Sid": "AllowScopedInstanceProfileActions",
"Effect": "Allow",
"Action": [
"iam:AddRoleToInstanceProfile",
"iam:RemoveRoleFromInstanceProfile",
"iam:DeleteInstanceProfile"
],
"Resource": "*",
"Condition": {
"StringEquals": {
"aws:ResourceTag/kubernetes.io/cluster/MF-MS-PRE-PROD-CLUSTER": "owned",
"aws:ResourceTag/topology.kubernetes.io/region": "ap-south-1"
},
"StringLike": {
"aws:ResourceTag/karpenter.k8s.aws/ec2nodeclass": "*"
}
}
},
{
"Sid": "AllowInstanceProfileReadActions",
"Effect": "Allow",
"Action": "iam:GetInstanceProfile",
"Resource": "*"
}
]
}
Step 4 : Add tags to subnets and security groups
aws ec2 create-tags --tags "Key=karpenter.sh/discovery,Value=MF-MS-PRE-PROD-CLUSTER" --resources "sg-06ee27bb7de43cf6e"
Step 5 : Update aws-auth ConfigMap:
We need to allow nodes that are using the node IAM role we just created to join the cluster. To do that we have to modify the aws-auth ConfigMap in the cluster.
kubectl edit configmap aws-auth -n kube-system
you will need to add a section to the mapRoles that looks something like this.
- groups:
- system:bootstrappers
- system:nodes
# - eks:kube-proxy-windows
rolearn: arn:aws:iam::${AWS_ACCOUNT_ID}:role/KarpenterNodeRole-santhosh
username: system:node:{{EC2PrivateDNSName}}
The full aws-auth configmap should have two groups. One for your Karpenter node role and one for your existing node group.
Step 6: Deploy Karpenter
helm install karpenter oci://public.ecr.aws/karpenter/karpenter --namespace "karpenter" --create-namespace \
--set "settings.clusterName=santhosh" \
--set "serviceAccount.annotations.eks\.amazonaws\.com/role-arn=arn:aws:iam::${AWS_ACCOUNT_ID}:role/KarpenterControllerRole-santhosh" \
--set controller.resources.requests.cpu=1 \
--set controller.resources.requests.memory=1Gi \
--set controller.resources.limits.cpu=1 \
--set controller.resources.limits.memory=1Gi
output:
[ec2-user@ip-172-31-0-244 ~]$ kubectl get po -A
NAMESPACE NAME READY STATUS RESTARTS AGE
karpenter karpenter-7d4c9cbd84-vpbfw 1/1 Running 0 29m
karpenter karpenter-7d4c9cbd84-zjwz4 1/1 Running 0 29m
kube-system aws-node-889mt 2/2 Running 0 16m
kube-system aws-node-rnzsk 2/2 Running 0 51m
kube-system coredns-6c55b85fbb-4cj87 1/1 Running 0 54m
kube-system coredns-6c55b85fbb-nxwrg 1/1 Running 0 54m
kube-system kube-proxy-8jmbr 1/1 Running 0 16m
Step 7 : Fetch AMI ID
We need to create a default NodePool so Karpenter knows what types of nodes we want for unscheduled workloads.
You can retrieve the image ID of the latest recommended Amazon EKS optimized Amazon Linux AMI with the following command
fetch AMI ID using command line:
aws ssm get-parameter --name /aws/service/eks/optimized-ami/1.30/amazon-linux-2-arm64/recommended/image_id --query Parameter.Value --output text
aws ssm get-parameter --name /aws/service/eks/optimized-ami/1.30/amazon-linux-2/recommended/image_id --query Parameter.Value --output text
aws ssm get-parameter --name /aws/service/eks/optimized-ami/1.30/amazon-linux-2-gpu/recommended/image_id --query Parameter.Value --output text
Step 8: Create nodeclass with KMS encryption:
apiVersion: karpenter.k8s.aws/v1
kind: EC2NodeClass
metadata:
name: karpenter-core-class
spec:
amiFamily: AL2023
role: arn:aws:iam::961489959441:role/Axis-MF-MSIL-PRE-PROD-CLUSTER-NodeInstanceRole
subnetSelectorTerms:
- tags:
kubernetes.io/cluster/Axis-MF-MSIL-PRE-PROD-CLUSTER: owned
securityGroupSelectorTerms:
- tags:
karpenter.sh/discovery: Axis-MF-MSIL-PRE-PROD-CLUSTER
amiSelectorTerms:
- id: ami-060175f7c2d4690ba
blockDeviceMappings:
- deviceName: /dev/xvda
ebs:
volumeSize: 50Gi
volumeType: gp3
encrypted: true
kmsKeyID: arn:aws:kms:ap-south-1:3668421:key/c2b329ac-bf-47cd-9e054379b
Step 9: Ceate nodepool
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
name: karpenter-core-pool
spec:
template:
spec:
requirements:
- key: kubernetes.io/arch
operator: In
values: ["amd64"]
- key: kubernetes.io/os
operator: In
values: ["linux"]
- key: karpenter.sh/capacity-type
operator: In
values: ["on-demand"]
- key: karpenter.k8s.aws/instance-type
operator: In
values:
- m5.xlarge
- r5.xlarge
- m5.2xlarge
- r5.2xlarge
nodeClassRef:
group: karpenter.k8s.aws
kind: EC2NodeClass
name: karpenter-core-class
expireAfter: 720h
limits:
cpu: 1000
disruption:
consolidationPolicy: WhenEmptyOrUnderutilized
consolidateAfter: 2m
🚀 Conclusion: Why Karpenter Stands Out ?
Karpenter offers a modern, intelligent approach to Kubernetes node provisioning that significantly enhances cluster performance and operational efficiency. Compared to traditional solutions like Cluster Autoscaler, Karpenter delivers:
⚙️ Improved Node Scheduling Efficiency By dynamically selecting the optimal instance types and sizes, Karpenter ensures your workloads run on the most suitable infrastructure—eliminating the rigidity of predefined node groups.
⚡ Faster Scaling Karpenter reacts to cluster changes in seconds, enabling rapid scaling during traffic spikes or workload surges—far outpacing the slower response times of Cluster Autoscaler.
💰 Cost Optimization With its ability to choose cost-effective instance types and avoid over-provisioning, Karpenter helps organizations reduce cloud spend while maintaining performance.
🛠️ Simpler Configuration Developers benefit from a streamlined setup process, as Karpenter removes the need to manage complex node group configurations and handles provisioning automatically.
Top comments (0)