Understanding Autoscale Throughput in Azure Cosmos DB

Provisioning Autoscale containers and databases in Azure Cosmos DB is simple and helps our apps perform better.

Back in January 2020, I wrote on article on Azure Cosmos DB’s ‘Autopilot’ mode that they released in November 2019, which was still in preview at the time of writing.

Not only has this feature gone GA (Generally Available), but it also has a much better name (In my opinion), Autoscale!

In this article, I will cover the following topics:

• How Throughput works in Azure Cosmos DB.
• Life before Autoscale Throughput.
• What is Autoscale Throughput and how does it work?
• The benefits of Autoscaled Throughput in Azure Cosmos DB.
• When would you opt for Autoscale over manually provisioned throughput?
• Creating a container with Autoscale Throughput provisioned.

To follow along with this article, it would help to be familiar with Azure Cosmos DB. If you’re not, no worries! 😊 Check out the following documentation as a quick primer and then feel free to come back here and read on!

How Throughput works in Azure Cosmos DB

In Azure Cosmos DB, when we make operations against our database, each operation costs something called Request Units in order to perform the operation. The cost of this varies depending on the complexity of the operation.

Request Units (or RU’s for short) are a abstraction of system resources like CPU, IOPS and memory required to perform the operations. In order for our operations against our Cosmos DB databases to execute successfully, we need to provision enough throughput.

Here’s a high-level overview of how RU’s work in Cosmos DB:

If you want to learn more about Request Units, check out the following documentation.

As of the time of writing, there are three ways that we can provision throughput on our Cosmos DB accounts:

1. Provisioned Throughput (Manual) — This provisions the number of RU/s for your application that you need on a per-second basis. This is provisioned in increments of 100 RU/s and you can change this value at any time, either programmatically or through the portal (programmatically is cooler 😎). You are billed on a hourly basis and you can provision throughput this way at both the Database level and the Container level.
2. Serverless — In this case, you don’t actually provision any throughput. Essentially you get charged per operation that you make against your Serverless account. If you’re new to Serverless mode in Cosmos DB, I wrote an article that you can check out here.
3. Autoscale — This automatically and instantly scales the amount of throughput of either you databases and containers based on the amount of operations being performed. This happens without impacting the availability or performance of your application.

When I first started working with Azure Cosmos DB, Provisioned throughput was the only way we could provision throughput in Cosmos DB, which caused a few problems.

Life before Autoscale Throughput.

Before Autoscale was available, you had to set a hard maximum limit on how many RU/s a container or database could have. It was simple enough to set up, but a lot of guesswork was involved as to how much you needed to set this hard limit to.

Say if we have an application that we know will experience a burst in traffic for 15 minutes a day. Using Provisioned Throughput we need to provision enough RU/s in order to accommodate the burst in traffic.

This had financial implications. In Provisioned Throughput mode, you are charged for this maximum limit on a hourly basis, so even though you may only use the maximum amount for 15 minutes a day, you would still be charged for the whole hour.

There are tools that we can use to change the maximum amount of throughput that we provision. We can either use the portal or use SDK’s, AZ CLI commands or PowerShell scripts to change the amount programmatically. While this works, writing and maintaining these scripts created an overhead for something that should have just been part of the service to begin with right?

Thankfully, this is where Autoscale Throughput can really help us out.

What is Autoscale Throughput and how does it work?

Autoscale Throughput automatically scales the amount of throughput provisioned on your containers and databases based on the amount of traffic that is being executed against your containers and databases.

When you use Autoscale, you set the maximum amount of throughput that you want that particular container or database to scale to. When the container is not is use, Azure Cosmos DB will set the throughput on that container/database to 10% of the defined maximum.

So for example, if you have a container that uses Autoscale and you set the maximum RU/s level to be 4,000 RU/s, when this container is not in use, it will rest at 400 RU/s.

The benefits of Autoscaled Throughput in Azure Cosmos DB.

Having the ability to use Autoscale Throughput for our Cosmos DB accounts provides us with some awesome benefits. No longer do we have to spend time messing around with scripts to change the amount of throughput that we provision.

Taking our 15 minute burst in traffic example from earlier, if we know how many request units we need to accommodate for this burst, we can set this as our maximum throughput level and let Cosmos DB handle scaling up and down as needed for us.

This also helps us to save money (got to keep our bosses happy right? 😉). Yes you still pay for Autoscale on a hourly basis, but rather than paying for that maximum throughput level for each hour of the day, we would only pay for that amount for the hour that we need it.

When would you opt for Autoscale over manually provisioned throughput.

For variable and unpredictable workloads, Autoscale provides huge benefits for us! No more messing about with calculators and scripts to determine how many RU/s we need to provision. Autoscale takes this concern away from us, so we can focus on developing our apps without having to manage a PaaS service.

It’s also great when we’re first starting out with Cosmos DB. We can combine performance testing and metrics to determine what kind of range we should set our Autoscale to based on what we know about our application and then let Cosmos accounts handle the scaling for us.

For infrequent workloads or dev/test environments, Serverless may be the better option (particularly from a cost perspective), but depending on what infrequent traffic means to your application, Autoscale is still a pretty good option in this scenario.

Creating a container with Autoscale Throughput provisioned.

There are a couple of ways to create a container with Autoscale Throughput. We can do this via the Portal, CLI, PowerShell and various SDKs. For this section, I’m going to save a bit of time and assume that you already have a Cosmos DB account provisioned.

Via the Portal:

Go into your Cosmos DB account and click on ‘New Container’. When choosing a throughput option, check the ‘Autoscale’ radio button and enter the maximum amount of RU/s you want your container to scale to. When you enter the Maximum amount, you’ll be given a estimate on how much this container will cost per month, which is pretty handy. Give your container a name, partition key and assign it to a new database or existing one and create it.

The great thing about Autoscale is that if you like, you can switch between Autoscale and Manual. You can do this in the portal through the settings tab of your container:

Via the CLI:

Assuming that we have a Cosmos DB account that uses the SQL API, we can create a container with Autoscale Throughput like so:

az cosmosdb sql container create -g MyResourceGroup -a MyAccount -d MyDatabase -n MyContainer --partition-key-path "/my/path" --idx @policy-file.json --ttl 1000 --max-throughput 4000

These commands differ depending on the API that your Cosmos DB account uses. You can find the exhaustive CLI commands for Cosmos DB here.

Via Powershell:

Again, I’m going to use the SQL API as an example. Here’s a sample script that we can use to provision a collection with Autoscale using PowerShell:

# Create an Azure Cosmos DB container with default indexes and autoscale throughput at 4000 RU
$resourceGroupName = "myResourceGroup"
$accountName = "mycosmosaccount"
$databaseName = "myDatabase"
$containerName = "myContainer"
$partitionKeyPath = "/myPartitionKey"
$autoscaleMaxThroughput = 4000 #minimum = 4000
New-AzCosmosDBSqlContainer `
    -ResourceGroupName $resourceGroupName `
    -AccountName $accountName `
    -DatabaseName $databaseName `
    -Name $containerName `
    -PartitionKeyKind Hash `
    -PartitionKeyPath $partitionKeyPath `
    -AutoscaleMaxThroughput $autoscaleMaxThroughput

For other PowerShell cmdlets to manage your Cosmos DB account, check out the following articles:

• Manage Azure Cosmos DB Core (SQL) API resources using PowerShell
• Azure PowerShell samples for Azure Cosmos DB Cassandra API
• Azure PowerShell samples for Azure Cosmos DB API for MongoDB
• Azure PowerShell samples for Azure Cosmos DB Gremlin API
• Azure PowerShell samples for Azure Cosmos DB Table API

Any Catches?

There are somethings to keep in mind when it comes to storage. The limit in GB for each max RU/s level is the Maximum RU/s that you set for a container or database divided by 100.

For example, if I set my container at a maximum of 20,000 RU/s, the maximum storage for that container will be 200GB. If we exceed this storage limit, Azure Cosmos DB will automatically increase the maximum throughput to support that storage amount.

Conclusion

Hopefully in this article, you now understand the benefits of using Autoscale Throughput for your Azure Cosmos DB containers and databases.

I’m so glad to see this feature go GA. I started working with the feature when it was still known as Autopilot and it’s grown so much since then. For my production applications, it has solved a lot of headaches for me and allowed me to just focus on writing my applications without worrying about how I’m going to manage throughput.

As always, if you have any questions for me, please feel free to ask in the comment section below, or reach out to me on Twitter.