DEV Community: eabraham

Level up your Heroku autoscaling in 15 minutes

eabraham — Tue, 05 Apr 2022 12:31:34 +0000

One of the most powerful features of Dynoscale’s Heroku autoscaling is the ability to configure it to meet the needs of your app without needing to write complex code. This tutorial will show you how to make advanced use of Dynoscale’s Heroku autoscaling to meet the needs of your app’s unique performance requirements. It will also show you how to enable the highest levels of autoscaling for your app with just a few configuration steps.

This post covers advanced Heroku autoscaling topics that build upon a prior post covering the basics.

Tuning Web Request Queuing

For a great end-user experience on your website, lowering backend latency is crucial. Amazon discovered ten years ago that every 100ms of latency cost them 1% in sales. Google discovered that adding 0.5 seconds to the generation time of a search page reduced traffic by 20%. We can match the correct thresholds for your traffic to reduce queue time by spending a few minutes examining the Web Dyno Activity report. Your users and customers will be able to load a page of your site faster if the queue time is reduced.

Navigate to the Web Dyno Activity report. Dynoscale sets your web upscale and downscale thresholds to 100 and 50 milliseconds, respectively, by default. These are useful settings, but it's possible that your application has a unique traffic pattern, in which case different thresholds would improve your autoscaling. Take the case above: the queue time was high from 1:00 AM to 1:30 AM, however the auto upscale was not triggered. This means that queries in the p95 category experienced roughly 100 milliseconds of additional backend request time due to queue time.

In this case, I recommend lowering the upscale threshold to 90ms and monitoring to see if auto scaling performance improves. Keep in mind not to overfit your thresholds based on a specific scenario. This frequently results in completely unanticipated behavior outside of the specific scenario.

Cost cutting during known periods of lower traffic

Imagine a scenario where your application provides a service for business users who mostly use it during weekdays. On weekends, your application receives significantly less traffic and you would like to run one web dyno instead of two during this period to cut costs.

Navigate to Web Dyno Auto Scaling Page and create a new rule.

In the new rule set the lower dyno bound to one and match the upper bound to the default rule’s. Set the Start At and End At to "12:00:00 AM" and "11:59:59 PM", respectively, and the Day of Week to "Weekend". Click Save to commit the rule and downscale to one dyno on weekends.

Preparing for a period of planned higher traffic

Consider a scenario in which your app's marketing team is about to launch a major campaign at 10 a.m. the next day (a Tuesday). You need to increase the dyno ranges to four and eight for a twelve-hour period to ensure the application can manage the increased traffic.

In advance of the campaign, navigate to Web Dyno Auto Scaling Page and create a new rule.

In the new rule set the lower dyno bound to four and the upper bound to eight. Next, set the Day of Week to "Tuesday" and the Start At and End At to "10:00:00 AM" and "10:00:00 PM" respectively. Click Save to create the new rule.

Once the campaign has run its course, delete this new auto scaling rule.

Optimizing for cost savings over traffic demand

When analyzing the Web Dyno Activity report, it may become clear that small bursts of traffic are driving your application to auto scale up. However, the traffic drops off soon, and you end up running extra dynos. Reducing the upscale dyno increment and the downscale cooldown time would be beneficial.

The scale up increment is two, and the downscale cool down is one hundred and twenty seconds in this situation.

Navigate to Web Dyno Auto Scaling Page and edit the active rule.

Reduce the upscale increment from two to one and the downscale cooldown to sixty seconds. Click Save to commit the rule and cut costs.

Optimizing for traffic demand over cost savings

When checking the Web Dyno Activity report, it may be obvious that heavy loads are driving your service to auto scale up, but scaling up to meet demand can take a long time. Users of your application will encounter increased queue time and a worse experience during this scaling up phase. Increasing the upscale dyno increment and reducing the upscale cooldown time would be useful.

The scale up increment is two, and the autoscale cool down is sixty seconds in this instance.

Navigate to Web Dyno Auto Scaling Page and edit the active rule.

Increase the upscale increment from two to four and decrease the upscale cooldown to thirty seconds. Click Save to commit the rule and improve request queue time.

Conclusion

The flexibility to tailor Dynoscale's Heroku autoscaling to fit your app's needs without having to write sophisticated code is one of its most powerful features. This tutorial showed you how to use Dynoscale's Heroku autoscaling in a more complex way to fulfill the performance requirements of your app. It also demonstrated how to adjust just a few configuration settings to enable the maximum capabilities of autoscaling for your app.

How to setup heroku autoscaling in under 15 minutes?

eabraham — Mon, 28 Mar 2022 12:46:43 +0000

You don’t have to be a Heroku veteran to setup autoscaling. But the surprising thing is that 63% of Heroku App Owners don’t setup autoscaling. By investing 15 min of your time, you can get started adding autoscaling to your app. Let’s walk through the simple, straightforward process to get autoscaling setup on your application so you can start saving money today.

Autoscaling is a useful tool to help you improve your application availability. It allows you to scale up or scale down dynos automatically in response to system traffic. This is useful because it can increase the number of dynos during traffic spikes and decrease the number of dynos during quiet periods to save cost but maintain availability.

Is Heroku's Autoscaling a good fit for me?

Heroku offers dyno autoscaling but there are limitations to their solution. Before setting up, lets review those limitations.

Are you running performance dynos?

If yes, Heroku will allow your application to autoscale. If you are running standard Dynos, unfortunately Heroku doesn't support Autoscaling at this time. Checkout another solution detailed below, that supports standard dyno Heroku Autoscaling.

Do all HTTP requests take about the same time?

If yes, Heroku scaling can be a good fit for you. If your request have a high variance in backend latency (200-500ms), its likely that Heroku's autoscaling will trigger many false positives and cost you more in unneeded dyno time. This is due to Heroku's autoscaling metric being Response time instead of Queue time.

If you have this problem checkout another solution detailed below, that can handle Heroku Autoscaling with high variance in backend latency.

Are you running in a private space?

If yes, then you are in luck. Heroku supports scaling for applications hosted on a private space. If you are not on a private space checkout the other solution detailed below.

Heroku Autoscaling

Configuration

Navigate to the Resources tab of your application on the Heroku Dashboard.

Click on the Enable Autoscaling button next to your web dyno details.

Select the upper and lower bounds for your applications autoscale range. A preview of the monthly cost range will appear below the dyno range. Your dyno count will always be scaled to a quantity inside the range you specify.

Next you set the desired p95 Response Time for your application. This value is the threshold used by Heroku's autoscaling engine to determine how to scale your dynos. A recommended p95 response time is listed below this field.

Then, if you would like to receive an Email Notification when your application is autoscaled to the max dynos check the box. This notification is sent at most once per day.

Finally, click Confirm to save your autoscale configuration. This will immediately adjust the dynos of your application to conform to the autoscale configuration.

Autoscaling Logic

The autoscaling engine uses the Desired p95 Response Time set in the prior step to determine when to scale your app. The autoscaling algorithm looks back over the past hour to calculate the number of dynos required to achieve the desired response time for 95% of incoming requests.

When the engine deems the number of dynos incorrect, a single dyno is either added or removed from your app. No more than 1 Autoscaling event can occur in 1 minute.

The autoscaling engine scales down less quickly than it scales up. This is to protect against a substantial downscaling from a temporary lull in requests resulting in high latency if demand spikes upward soon after.

If there is no request throughput your application for 3 minutes, dynos will scale down at 1-minute intervals until throughput resumes.

Monitoring Autoscale Events

Navigate to the Metrics tab of your Heroku application. Scaling events can be seen in the Events chart. By hovering over a scaling event you can see what initiated it. Heroku autoscaling events are labeled “Dyno Autoscaling”. If multiple autoscaling events occur in the same period, only the step where the scaling changed direction is shown.

Disabling Autoscaling

To disable heroku autoscaling navigate to the Resources tab of your application on the Heroku Dashboard. Then click on the Disable Autoscaling button and select a fixed dyno count.

Dynoscale Autoscaling

Dynoscale is a Heroku Autoscaling Addon that solves many of the limitations of Heroku's autoscaling engine. It works for both standard and performance dynos, letting you decide the proper resources for your application. In addition, Dynoscale uses queues time instead of response time to scale your application. This ensures that long running application code doesn't scale based on a false positives from a Response Time metric.

Configuration

Dynoscale currently supports Ruby and Python applications. To install for a Rails/Ruby Application, follow these getting started instructions. To install for a Python Application, follow these getting started instructions.

Once you have installed the Dynoscale addon, next lets setup the scaling rules by specifying the Autoscaling Logic. For those with the Heroku CLI installed, in a terminal run heroku addons:open dscale. For those who do not have the Heroku CLI installed, navigate to the Resource tab of your application in the Heroku Dashboard and click on Dynoscale.

Autoscaling Logic

Next navigate to the Web Autoscale Rules to setup the autoscaling logic. Click on the blue edit rule button.

On the upper left side of the rule you can see the upper and lower bounds for dynos. These bounds set the max/min dynos this rule will permit. For a relatively low traffic application with some periods of high traffic its sensible to set the lower bound to 1 and the upper bound to 2.

On the lower left side of the rule you can see the Upscale and Downscale Configuration. On the Upscale configuration, the first field is the request queue threshold. It determines when more dynos are needed to meet the traffic needs of your application. Once the threshold is exceeded, Dynoscale will upscale your application by an increment set in the next field. Finally, to avoid frequent scale ups, the last field delays incremental scaleups with a buffer time.

On the Downscale configuration, the first field is the request queue threshold. It determines when to reduce the dynos needed to meet the traffics needs of your application. Once the request queue drops below the threshold, Dynoscale will downscale your application by an increment set in the next field. Finally, to avoid frequent scaling, the last field delays incremental scale downs with a buffer time.

Once you are happy with a rule, click on the green save button in the upper right side of the rule. Alternatively, if you don't want to save changes, click on the black cancel button.

Finally click on the toggle on the upper right side of the page. Red indicates that your rules are off and green your rules are active.

Monitoring Autoscale Events

In addition to the Heroku Dashboard, Scaling information can be found on the Web Dyno Activity Page. This page displays queue time and dyno scaling charts for the past 24 hours.

Disabling Autoscaling

To turn off autoscaling for your Heroku application. Navigate to the Web Autoscaling Rules page and click on the toggle in the upper right-hand side of the screen. Red means the rule is off.

Conclusion

This article explores the scenarios for which Heroku Autoscaling is a good fit for your application. In addition, it details the setup procedure for Heroku Autoscaling.

If Heroku Autoscaling is not a good fit this article details an alternative called Dynoscale and outlines quick, easy setup instructions.