We turned on autoscaling and slept better for exactly one month, until the invoice arrived. Traffic had grown maybe fifteen percent that month, a good and healthy number. The cloud bill had grown closer to sixty. Something in the machinery we had trusted to save us money was instead quietly spending it, and it took a genuinely uncomfortable afternoon of reading metrics to understand that the autoscaler had been doing precisely what we told it to do. We had just told it something careless without realizing.
The problem was that our scaling trigger was tied to a metric that spiked for reasons that had nothing to do with real demand. A slow downstream dependency made our own requests pile up, CPU climbed as threads waited, the autoscaler saw the climb and added machines, and the new machines made more requests to the same slow dependency, which made everything slower still. We had built a feedback loop that responded to a bottleneck by throwing money at the one place money could not help. The system scaled itself enthusiastically in exactly the wrong direction.
This is the thing about autoscaling that the marketing never mentions. It does not give you fewer decisions to make. It gives you a smaller number of far more consequential decisions, made once, in advance, that then execute thousands of times without a human in the loop. Choose the wrong signal, forget to set an upper bound, and you have not automated cost control. You have automated cost, and handed the automation your credit card with no ceiling.
Now I treat every autoscaling rule as a small, sharp financial instrument that deserves the same scrutiny as the code it runs. What exactly triggers a scale-up, and is that metric a true proxy for demand or just a symptom that could have other causes? What is the maximum, in machines and in money, that I am willing to let this thing spend before it stops and calls a human? Autoscaling without a hard ceiling is not elasticity. It is an open-ended bet that your triggers are perfect.
Elastic infrastructure is a wonderful thing, right up until it stretches in the wrong direction with your money. Automate the scaling, absolutely, but never automate away the limit.
– Serguey Shinder
Top comments (0)