A checkout API runs on Lambda (serverless compute, Node.js, 1,024 MB, attached to a VPC for RDS access). Cold starts were causing P99 spikes of 3.8 seconds on Monday mornings when traffic ramps up after the weekend.
The team enabled Provisioned Concurrency (pre-initializes a fixed number of execution environments so they are warm before requests arrive — billed per environment, around the clock, whether or not traffic uses them): 10 pre-initialized execution environments, configured to stay warm at all times. Additional monthly cost: approximately $110.
The following Monday, P99 still spiked — this time to 3.1 seconds on the first wave of requests. The provisioned environments showed 0 invocations in CloudWatch (AWS monitoring service) during the spike window. The function was being invoked. The provisioned environments were running. Nothing was using them.
What went wrong?
A) 10 provisioned environments is not enough — the Monday spike requires at least 50 to absorb the concurrency burst
B) Provisioned Concurrency was configured on the $LATEST version (the mutable, always-updating function version — Provisioned Concurrency cannot target it) — it must be applied to a published version or an alias pointing to one, and the event source must invoke that version or alias
C) The VPC attachment (places Lambda inside the customer VPC for private resource access) is re-initializing the ENI on each provisioned environment at invocation time, overriding the warm state
D) Provisioned Concurrency requires the function to be behind API Gateway (managed API layer) — direct invocation bypasses the warm pool
Answer in the comments.
Top comments (5)
The answer is B.
Provisioned Concurrency (pre-initializes environments, billed per environment regardless of traffic) cannot be applied to
$LATEST. AWS requires you to configure it on a published version (e.g., version 12) or an alias (a named pointer to a specific published version) that resolves to a published version. And the routing must match: if your event source — API Gateway (managed API layer), ALB (load balancer), Function URL — is invoking$LATEST, it invokes$LATEST. The provisioned pool attached to version 12 is never reached. The pre-initialized environments sit warm, billed, and idle while$LATESTcold-starts in front of your users.CloudWatch (AWS monitoring service) confirms this directly: 0 invocations against the provisioned concurrency configuration while the function itself is being invoked. Warm capacity and live traffic are pointing at two different targets.
The fix: publish a version with
aws lambda publish-version, configure Provisioned Concurrency against that version or an alias pointing to it, then update the API Gateway stage or ALB target to invoke the alias instead of$LATEST. The alias is the piece teams forget — it is what lets you shift the warm pool to a new version on deploy without touching the event source again.Worth knowing while you're here: the money only works if the routing works. Ten warm environments at 1 GB run roughly $110/month in idle charges ($0.0000041667 per GB-second, around the clock) before any invocation costs. Misrouted, that is $110/month for a space heater.
A — The CloudWatch signal rules this out completely: 0 invocations on the provisioned config. Undersized capacity would show the pool saturated and overflowing — high utilization, spillover cold starts. A pool at zero is not too small; it is unreachable.
C — VPC ENI attachment was a real cold start contributor before 2019, when AWS moved to a Hyperplane architecture with pre-provisioned ENIs shared across functions in the same VPC and subnet. For most workloads, VPC attachment (places Lambda inside the customer VPC) no longer adds meaningful cold start latency — and it would not explain warm environments receiving zero invocations.
D — Provisioned Concurrency works with any invocation path: API Gateway (managed API layer), ALB (load balancer), Function URLs, SDK direct invocation, and event source mappings. There is no gateway requirement. What every path shares is the actual requirement: it must invoke the version or alias that carries the provisioned configuration.
Also, it would mean a lot to me if you could support my content and stay in touch 🙏
YouTube: youtube.com/@system-design-lab
LinkedIn: linkedin.com/in/joud-awad/
Medium Blog: joudwawad.medium.com/
Substack: joudawad.substack.com/