You don’t have to be a Heroku veteran to setup autoscaling. But the surprising thing is that 63% of Heroku App Owners don’t setup autoscaling. By investing 15 min of your time, you can get started adding autoscaling to your app. Let’s walk through the simple, straightforward process to get autoscaling setup on your application so you can start saving money today.
Autoscaling is a useful tool to help you improve your application availability. It allows you to scale up or scale down dynos automatically in response to system traffic. This is useful because it can increase the number of dynos during traffic spikes and decrease the number of dynos during quiet periods to save cost but maintain availability.
Is Heroku's Autoscaling a good fit for me?
Heroku offers dyno autoscaling but there are limitations to their solution. Before setting up, lets review those limitations.
Are you running performance dynos?
If yes, Heroku will allow your application to autoscale. If you are running standard Dynos, unfortunately Heroku doesn't support Autoscaling at this time. Checkout another solution detailed below, that supports standard dyno Heroku Autoscaling.
Do all HTTP requests take about the same time?
If yes, Heroku scaling can be a good fit for you. If your request have a high variance in backend latency (200-500ms), its likely that Heroku's autoscaling will trigger many false positives and cost you more in unneeded dyno time. This is due to Heroku's autoscaling metric being Response time instead of Queue time.
If you have this problem checkout another solution detailed below, that can handle Heroku Autoscaling with high variance in backend latency.
Are you running in a private space?
If yes, then you are in luck. Heroku supports scaling for applications hosted on a private space. If you are not on a private space checkout the other solution detailed below.
Heroku Autoscaling
Configuration
Navigate to the Resources tab of your application on the Heroku Dashboard.
Click on the Enable Autoscaling button next to your web dyno details.
Select the upper and lower bounds for your applications autoscale range. A preview of the monthly cost range will appear below the dyno range. Your dyno count will always be scaled to a quantity inside the range you specify.
Next you set the desired p95 Response Time for your application. This value is the threshold used by Heroku's autoscaling engine to determine how to scale your dynos. A recommended p95 response time is listed below this field.
Then, if you would like to receive an Email Notification when your application is autoscaled to the max dynos check the box. This notification is sent at most once per day.
Finally, click Confirm to save your autoscale configuration. This will immediately adjust the dynos of your application to conform to the autoscale configuration.
Autoscaling Logic
The autoscaling engine uses the Desired p95 Response Time set in the prior step to determine when to scale your app. The autoscaling algorithm looks back over the past hour to calculate the number of dynos required to achieve the desired response time for 95% of incoming requests.
When the engine deems the number of dynos incorrect, a single dyno is either added or removed from your app. No more than 1 Autoscaling event can occur in 1 minute.
The autoscaling engine scales down less quickly than it scales up. This is to protect against a substantial downscaling from a temporary lull in requests resulting in high latency if demand spikes upward soon after.
If there is no request throughput your application for 3 minutes, dynos will scale down at 1-minute intervals until throughput resumes.
Monitoring Autoscale Events
Navigate to the Metrics tab of your Heroku application. Scaling events can be seen in the Events chart. By hovering over a scaling event you can see what initiated it. Heroku autoscaling events are labeled “Dyno Autoscaling”. If multiple autoscaling events occur in the same period, only the step where the scaling changed direction is shown.
Disabling Autoscaling
To disable heroku autoscaling navigate to the Resources tab of your application on the Heroku Dashboard. Then click on the Disable Autoscaling button and select a fixed dyno count.
Dynoscale Autoscaling
Dynoscale is a Heroku Autoscaling Addon that solves many of the limitations of Heroku's autoscaling engine. It works for both standard and performance dynos, letting you decide the proper resources for your application. In addition, Dynoscale uses queues time instead of response time to scale your application. This ensures that long running application code doesn't scale based on a false positives from a Response Time metric.
Configuration
Dynoscale currently supports Ruby and Python applications. To install for a Rails/Ruby Application, follow these getting started instructions. To install for a Python Application, follow these getting started instructions.
Once you have installed the Dynoscale addon, next lets setup the scaling rules by specifying the Autoscaling Logic. For those with the Heroku CLI installed, in a terminal run heroku addons:open dscale. For those who do not have the Heroku CLI installed, navigate to the Resource tab of your application in the Heroku Dashboard and click on Dynoscale.
Autoscaling Logic
Next navigate to the Web Autoscale Rules to setup the autoscaling logic. Click on the blue edit rule button.
On the upper left side of the rule you can see the upper and lower bounds for dynos. These bounds set the max/min dynos this rule will permit. For a relatively low traffic application with some periods of high traffic its sensible to set the lower bound to 1 and the upper bound to 2.
On the lower left side of the rule you can see the Upscale and Downscale Configuration. On the Upscale configuration, the first field is the request queue threshold. It determines when more dynos are needed to meet the traffic needs of your application. Once the threshold is exceeded, Dynoscale will upscale your application by an increment set in the next field. Finally, to avoid frequent scale ups, the last field delays incremental scaleups with a buffer time.
On the Downscale configuration, the first field is the request queue threshold. It determines when to reduce the dynos needed to meet the traffics needs of your application. Once the request queue drops below the threshold, Dynoscale will downscale your application by an increment set in the next field. Finally, to avoid frequent scaling, the last field delays incremental scale downs with a buffer time.
Once you are happy with a rule, click on the green save button in the upper right side of the rule. Alternatively, if you don't want to save changes, click on the black cancel button.
Finally click on the toggle on the upper right side of the page. Red indicates that your rules are off and green your rules are active.
Monitoring Autoscale Events
In addition to the Heroku Dashboard, Scaling information can be found on the Web Dyno Activity Page. This page displays queue time and dyno scaling charts for the past 24 hours.
Disabling Autoscaling
To turn off autoscaling for your Heroku application. Navigate to the Web Autoscaling Rules page and click on the toggle in the upper right-hand side of the screen. Red means the rule is off.
Conclusion
This article explores the scenarios for which Heroku Autoscaling is a good fit for your application. In addition, it details the setup procedure for Heroku Autoscaling.
If Heroku Autoscaling is not a good fit this article details an alternative called Dynoscale and outlines quick, easy setup instructions.
Top comments (0)