Capacity Planning AKS Azure Kubernetes Service Cluster

Capacity Planning for AKS Deployment

Deploying a microservice-based architecture into Azure Kubernetes Service (AKS) requires careful capacity planning to ensure optimal performance. Key considerations include networking, compute, and the size of the cluster's worker nodes.

Networking Plugins

When setting up a new Azure AKS cluster, you need to choose a networking plugin. Azure AKS supports two types of networking plugins:

Kubenet - The default networking plugin.
Azure CNI.

Kubenet

Node IP Addressing: Nodes receive an IP address from the Azure Virtual Network subnet.
Pod IP Addressing: Pods receive an IP address from a logically different address space than the node pool's subnet.
Network Address Translation (NAT): Configured so pods can reach resources on the Azure Virtual Network. The source IP address of the traffic is translated to the node's primary IP address.
Pod Communication: Pods can't communicate directly with each other across nodes. User Defined Routing (UDR) and IP forwarding are used for connectivity between pods across nodes. By default, UDRs and IP forwarding configurations are created and maintained by the AKS service, but you can bring your own route table for custom route management.

Azure CNI

Node IP Addressing: Nodes receive an IP address from the Azure Virtual Network subnet.
Pod IP Addressing: Each pod receives an IP address within the node pool's subnet and can directly communicate with other pods and services.
Cluster Size: Clusters can be as large as the IP address range specified. The IP address range must be planned in advance, as all IP addresses are consumed by the AKS nodes based on the maximum number of pods they can support.

Capacity Planning Considerations

When it comes to capacity planning, consider the following questions:

How many worker nodes do I need?
What is the size of the subnet?
How many PODs could be running on this cluster?

Some settings in an AKS cluster cannot be changed after deployment, such as the subnet IP range and the maximum number of pods per node. Here, I will discuss how to calculate the size of the AKS subnet and the cluster. I will not delve into the different services offered by AKS or the comparison between Kubenet and Azure CNI.

Example Calculation

Suppose we have an AKS cluster with the Azure CNI networking plugin and a single node. In this configuration, the AKS cluster supports a maximum of 30 pods per node by default. Here is the calculation:

Whenever a subnet is created inside a VNet, Microsoft reserves 5 IPs:
- 0 Network Address
- 1 Default Gateway
- 2 and 3 Reserved by Azure to map the Azure DNS IPs to the VNet space
- 255 Network broadcast address
Additional IPs required:
- 1 IP assigned to the network interface of the 1 node in your cluster
- 30 IPs reserved by Azure CNI, which is the maximum number of pods per node

So, the total number of required IPs is:
[ 5 + 1 + 30 = 36 ]

If we create the subnet with the range 10.0.0.1/27, then:
[ 32 - 27 = 5 ]
[ 2^5 = 32 ]

This is not enough to accommodate 36 IPs, so we need to create a subnet with the range 10.0.0.1/26:
[ 32 - 26 = 6 ]
[ 2^6 = 64 ]

This range is sufficient to accommodate 36 IPs.

For a two-node cluster:
[ 5 + (2 + 1) + (30 \times (2 + 1)) = 5 + 3 + 90 = 98 ]

So, the 10.0.0.1/25 subnet (which supports 128 IPs) is still sufficient.

For a three-node cluster, the subnet range 10.0.0.1/25 is not sufficient. In this case, you would need to reduce the number of pods per node from the default 30 to a lower number to fit within the subnet range, or expand the subnet range.

Each additional node automatically added during AKS node pool upgrades requires an additional IP.

REST API for IP Calculation

You can use the following REST API to calculate the number of IPs needed based on your cluster's pods and scaling factor:
[ https://akscnicalc.azurewebsites.net/api/akscnicalc ]

Sample payload:

{
  "nodes": 5,
  "pods": 30,
  "scale": 2,
  "ilbs": 1
}

nodes: Number of nodes in the AKS cluster
pods: Number of pods per node
scale: Scale-out capacity (e.g., 2 nodes)
ilbs: Number of Azure Internal Load Balancers you would like to use in the AKS subnet and setup for your container workloads

By understanding and configuring these networking options, you can ensure optimal connectivity and performance for your microservices workloads in AKS.

Thanks
Sreeni Ramadurai