We moved a small API to Cloud Run and immediately started getting complaints about the first request of the day being slow. So I spent a week measuring cold starts instead of guessing.
How I measured
Not with synthetic benchmarks, but with the actual requests: the service logs a timestamp on entry, and I recorded the gap between the request arriving at the load balancer and the first line of our handler executing. That includes platform time we can't control, which is the honest number a user experiences.
I ran this for a week, then made one change at a time and watched the same metric.
What moved the needle
Container size was the biggest single factor. Our first image was built from a full node image with dev dependencies, about 900MB. A multi-stage build with a distroless base brought it to under 100MB, and the p50 cold start dropped from roughly 4 seconds to under 2.
Minimum instances made the tail disappear. We set --min-instances=1 on the service that humans actually wait on, and kept zero for the internal batch jobs. The cost difference for a small instance was about the same as one coffee a week, which I'm happy to pay for not getting "is it down?" messages.
Startup work was the sneaky one. We had a config loader that fetched a dozen secrets over the network on boot. Moving those to environment variables configured at deploy time removed almost a second, and made the service start deterministically.
What didn't help
More memory. I tried bumping to a larger instance class and the cold start barely changed. It does help throughput, not startup.
Parallelism settings: they change how many requests an instance handles, not how fast it starts. Worth tuning separately, and only if you have the traffic to justify it.
The current setup
Small multi-stage image, secrets via env vars, min instances of 1 on the user-facing API, zero everywhere else, and an alert when the p95 startup exceeds our old baseline. Nothing clever. The main lesson was that cold starts are mostly a build-and-boot problem, and the platform will only fix the part it owns.
Top comments (3)
Was the min-instances=1 call harder to justify than slimming the image?
Measuring at the load balancer is the right call. Our synthetic numbers were way too optimistic by comparison.
The config loader over the network is a sneaky one. We hit the same thing with a feature-flag client on boot.