An m5.xlarge running 10 pods holds 30 IP addresses from your subnet.
That's with the defaults: the Amazon VPC CNI in secondary-IP mode and WARM_ENI_TARGET=1. A /24 has 251 usable addresses, so it's full at eight such nodes, 80 pods. If those are small 250m / 512Mi pods, the eight nodes have 62% of their CPU and 31% of their memory requested. They're bored, and you still can't schedule pod number 81. Most teams find out from a pod stuck in ContainerCreating:
Warning FailedCreatePodSandBox kubelet
failed to assign an IP address to container
I went through the AWS docs, the VPC CNI source and every public IP-exhaustion story I could find, and ran the numbers for all 1,328 instance types with published ENI limits. Here's what eats your subnet, and the three ways out.
The warm ENI nobody asked for
With the default settings every pod gets a real VPC address from one of the node's ENIs (elastic network interfaces). The CNI keeps one spare ENI fully loaded (WARM_ENI_TARGET=1) so new pods start fast. An ENI on an m5.xlarge carries 15 addresses, so the node holds:
pods on node ENIs in use + warm IPs held IPs per pod
10 1 + 1 30 3.0
20 2 + 1 45 2.3
58 (max) 4 60 1.0
Why would anyone want that? Because attaching a new ENI can take up to 10 seconds, and the CNI docs say plainly that this is "the main reason why we have WARM_ENI_TARGET=1 as the default setting." Fast pod starts, paid for in addresses.
Small, half-empty nodes are the worst case. At a quarter of max pods, a t3.small holds 4 IPs per running pod, a t3.medium or t3.large 3, an m5.large 2.86, and the whole xlarge class (m5, m6i, c5, r5) 2.14. Across the fleet it gets silly: for 926 of 1,328 instance types, a node running a single pod holds 60 or more addresses, and for 604 types one full node plus its warm ENI is already bigger than a /24.
What it looks like when it happens to someone big
The best public write-up I found is Neon's: AWS CNI lessons from a Production Outage (June 2025, with a postmortem). They're not a small shop with a /24. They had three /20 subnets, 12,273 usable addresses, in us-east-1, starting around 500 pods a minute.
When the pod count grew from roughly 5,000 to 8,000, only 71% of the addresses were assigned to pods, but 99% were allocated to nodes. One node held about 450 IPs for 200 pods. In their words, "we never free IP addresses back to the subnet." Pod starts stalled for a total of 5.5 hours over two days.
The fix that sounds obvious made it worse first. With WARM_IP_TARGET=1, every pod deletion put an IP into a 30-second cooldown that still counted as "warm," so new pods waited. They ended up setting all three warm targets to 0.
It isn't a one-off:
- Wealthsimple watched one EKS subnet go from 80% to 100% in two weeks during a migration (November 2024). Short term, more subnets. Long term, custom networking.
- Adevinta got paged because DaemonSets on a small, mostly unused cluster failed with "failed to assign an IP address to container" (2024), and later paused their own migration because addresses ran low.
- A VPC CNI issue from 2020 shows that event repeated 23,123 times over 21 hours on m5 nodes.
The common thread: plenty of CPU, no addresses.
Max pods is an ENI limit, not a CPU limit
With secondary IPs:
max pods = ENIs × (IPs per ENI − 1) + 2
m5.large 3 × (10 − 1) + 2 = 29
t3.medium 3 × (6 − 1) + 2 = 17
m5.xlarge 4 × (15 − 1) + 2 = 58
The "+ 2" is aws-node and kube-proxy, which run on the host network and need no pod IP. These numbers match AWS's own eni-max-pods.txt for every one of the 1,302 types that appear in both lists.
Past that number pods stay Pending, however much CPU is free. And since EKS reserves 11 MiB of memory per possible pod, the same number also decides how much memory the node gives you. More on that here:
Here's a twist I didn't expect: for most clusters, max pods is not the limit that hits first. With 250m / 512Mi pods, only 132 of 1,264 non-GPU instance types run out of ENI slots before they run out of CPU or memory. An m5.large fits 6 such pods by resources and 26 by ENIs. Max pods only bites with tiny pods: at 100m / 128Mi it's the limit on 620 types. The thing that runs out first is usually the subnet.
And a confession. While checking the article, I found that my own EKS calculator got max pods wrong for 62 big instance types, the ones with several network cards, like c6in.32xlarge. AWS only counts the ENIs of network card 0, so the right answer is 394 pods, not the 786 my code showed. It's fixed now. The AWS script got the same fix in 2024, so at least I was wrong in good company.
Way out 1: fewer warm IPs
The cheapest fix is a config change on the aws-node DaemonSet: replace the warm ENI with WARM_IP_TARGET and MINIMUM_IP_TARGET. The node keeps a few spare addresses instead of a whole ENI's worth.
How many pods a subnet fits, for m5.xlarge nodes:
default (warm ENI) WARM_IP_TARGET=2 + prefix delegation
MINIMUM_IP_TARGET=pods
/24, 10 pods/node 8 nodes 80 pods 22 nodes 220 pods 7 nodes 70 pods
/24, 20 pods/node 5 nodes 100 pods 11 nodes 220 pods 5 nodes 100 pods
/22, 20 pods/node 22 nodes 440 pods 46 nodes 920 pods 20 nodes 400 pods
/20, 10 pods/node 136 nodes 1,360 pods 371 nodes 3,710 pods 123 nodes 1,230 pods
Same subnet, 2.7× the pods, one environment variable.
The price is more EC2 API calls (the CNI docs warn about throttling with low targets) and slower starts during bursts, because addresses are attached on demand. Neon's cooldown story above is what that looks like at 500 pods a minute. There's also a long-running issue where nodes report Ready before their IPs exist when these targets are set. Tune it, then test a burst before you trust it.
Way out 2: prefix delegation (on dense nodes)
Prefix delegation puts a /28 (16 addresses) into each ENI slot, so an m5.large can run 110 pods instead of 29. The cap is 110 for nodes with up to 30 vCPUs and 250 above that, and managed node groups enforce it. Attaching a prefix takes under a second instead of up to 10 for an ENI.
But the node now takes addresses in blocks of 16, plus one warm block. At 20 pods an m5.large holds 49 IPs with prefixes and 30 without. On sparse nodes, prefixes cost more addresses, not fewer, which is why the prefix column in the table above never wins on a /24. AWS's own guidance is honest about it: "If you can afford (in the worst case) up to 15 unused IPs per node, we strongly recommend using the newer prefix assignment."
It also needs contiguous free /28 blocks. A fragmented subnet fails with InsufficientCidrBlocks even when plenty of single addresses are free. CloudKeeper wrote up exactly that: a /25 with prefix delegation, the console showing free IPs, and zero intact /28 blocks left.
A subnet CIDR reservation for prefixes avoids that. Set it up before the subnet fragments, not after.
Way out 3: custom networking
The usual way out at scale: add a secondary VPC CIDR (AWS recommends 100.64.0.0/10) and put pods there with custom networking. The node's primary ENI no longer hosts pods, so max pods drops: an m5.large goes from 29 to 20. Wealthsimple hit exactly that number in their migration: "large instance types are limited to 20 pods, whereas without custom networking we could run 29."
You trade a few pods per node for a pod address space that's no longer shared with everything else in the VPC. If your subnets are already tight, it's the only fix that doesn't depend on how bursty you are.
The checklist I'd run today
- Count allocated, not assigned. The VPC console's "available IPs" for your node subnets is the real number. Pods-per-node math hides the warm pool.
- Look at your smallest, emptiest nodes. They hold the most addresses per pod.
-
Change
WARM_ENI_TARGETdeliberately. Either keep it and size subnets for it, or switch toWARM_IP_TARGET/MINIMUM_IP_TARGETand test a burst. - Prefix delegation for dense nodes only, with a CIDR reservation.
- Plan custom networking before the migration, not during it. eksctl already creates /19 subnets by default for a reason.
Do the math before the subnet does
I built a calculator for this in NodeSize. Pick an instance type and CNI mode, enter nodes and pods per node, and it shows max pods, IPs per node including the warm pool, and whether your subnet fits: EKS max pods and subnet IP calculator. It covers the ENI limits of 1,328 instance types, with the formulas and AWS sources on the page (and now the right answer for multi-card instances).
Your turn
When your subnet ran dry, what did you do: bigger CIDR, prefixes, warm-IP tuning, or custom networking?
Top comments (0)