Provisioning Azure Cosmos DB resources with Terraform

Thanks to Terraform, we can deploy Azure Cosmos DB accounts, databases and collections easily in just a few lines of code!

In a DevOps world, we want to have the ability to make changes to our cloud infrastructure easily and intuitively. This is especially true when deploying our databases, as we need to be able to iterate quickly to accommodate any changes in requirements that our applications need to go through.

Azure Cosmos DB is a globally distributed database service that provides developers with an enormous amount of flexibility when it comes to developing globally distributed applications. Thanks to Infrastructure as Code tools like Terraform, we can deploy Cosmos DB accounts and resources easily with just a few lines of code.

In this article, I’m going to show you how you can use Terraform to spin up Cosmos DB Accounts, Databases and Collections quickly and configure them according to the requirements of your applications.

If you want to follow along with this tutorial or even just spin up what I’ve created, you can check out the full sample on my GitHub here.

What is Terraform?

Terraform is a Infrastructure as Code (IaC) tool created by HashiCorp that allows us to define and provision infrastructure using Hashicorp Configuration Language (HCL).

Terraform supports a number of different cloud providers, including Azure, and it allows us to code up blueprints of our infrastructure, share that code, re-use it and version it.

If you’re brand new to Terraform, I recommend that you check out their tutorial section before continuing with this tutorial. It will show you how to install Terraform on your computer and how you can use it to deploy resources to Azure.

What is Azure Cosmos DB?

Azure Cosmos DB is a globally distributed, multi-model database services that allows you to elastically scale in both throughput and storage. You can use document, key-value, columnar and graph databases in Azure Cosmos DB thanks to their multiple API offerings.

Cosmos DB is a foundational service in Azure and is available in all Azure regions worldwide. You can build highly responsive and highly available applications worldwide, bringing your user’s data closer to where they are.

This article won’t go over Cosmos DB in huge detail, but if you want to dive deeper into Cosmos DB, check out the docs here.

Cosmos DB Topology

When we provision a Azure Cosmos DB account, we are creating an account to manage our databases, containers and items. Check out the below diagram to see how the hierarchy of entities work in Cosmos DB:

At the very top of the Cosmos DB food chain, we have our accounts. This is where we manage data by creating databases and containers. We can create multiple databases within our account and within our databases, we can create multiple containers.

In Terraform, we can create our Azure Cosmos DB account and then create databases and containers inside the account.

Creating an Account

Before even creating our account, we’ll need to tell Terraform that we’re deploying our resources to Azure. We can do this by using the Azure Provider. This is used to configure infrastructure in Azure. We’ll also use this provider to tell Terraform which subscription we are using. For that, we’ll need to provide our Subscription Id and our Tenant Id.

To find your Subscription Id, go to your Subscriptions list in Azure and then copy + paste the value underneath Subscription ID:

To find your Tenant Id,Go to your Azure Active Directory view in the Portal and you should see your Tenant ID. Copy and paste it:

We’ll stick both the Subscription and Tenant Id in a file called variables.tf like so:

variable "subscription_id" {
  default = "<PUT_YOUR_SUBSCRIPTION_ID_HERE>"
}
variable "tenant_id" {
  default = "<PUT_YOUR_TENANT_ID_HERE>"
}

Now that we’ve set up our Azure Provider, add the following variables to your variables.tf file:

variable "resource_group_name" {
  default = "VelidaCosmosDBTerraform-rg"
}
variable "resource_group_location" {
  default = "australiaeast"
}
variable "cosmos_db_account_name" {
  default = "velidacosmosterraform"
}
variable "failover_location" {
  default = "australiasoutheast"
}

Then create a file called main.tf and write the following code:

provider "azurerm" {
  version = "~> 1.34.0"
  subscription_id = "${var.subscription_id}"
  tenant_id = "${var.tenant_id}"
}
resource "azurerm_resource_group" "rg" {
  name = "${var.resource_group_name}"
  location = "${var.resource_group_location}"
}
resource "azurerm_cosmosdb_account" "acc" {
  name = "${var.cosmos_db_account_name}"
  location = "${azurerm_resource_group.rg.location}"
  resource_group_name = "${azurerm_resource_group.rg.name}"
  offer_type = "Standard"
  kind = "GlobalDocumentDB"
  enable_automatic_failover = true
consistency_policy {
    consistency_level = "Session"
  }

  geo_location {
    location = "${var.failover_location}"
    failover_priority = 1
  }
geo_location {
    location = "${var.resource_group_location}"
    failover_priority = 0
  }
}

Let’s go through this block by block.

First up, we are setting the provider. There are different versions of providers that support different resources. I’m also setting the subscription_id and tenant_id to tell Terraform which Azure subscription I want to deploy my Cosmos DB resources to.

I’m then creating a resource group for my Cosmos DB account. Resource groups are logical collections of resources in Azure. Typically we would create one resource group for an application, but this depends on your needs. Check out this article for more information on Resource Groups. In this block, I’m defining the name of my resource group and the location. I’ve set the default values of this in my variables.tf file and I’m referencing those values in my main.tf file.

Now we can set up our Cosmos DB account! This is done via the azurerm_cosmosdb_account resource. Here, I’m giving it a name, assigning it to my resource group, giving it a location, defining the offer type, the kind of Cosmos DB account to create, defining the Consistency level of my Cosmos DB account, enabling geo_location for geo-replication and failover location priorities.

If you want more details on the cosmosdb_account resource in Terraform, check out this article.

Creating Databases

Now that we have our account, I’m going to create a SQL API database within that account. Add the following resource definition to your main.tf file:

resource "azurerm_cosmosdb_sql_database" "db" {
  name = "products"
  resource_group_name = "${azurerm_cosmosdb_account.acc.resource_group_name}"
  account_name = "${azurerm_cosmosdb_account.acc.name}"
}

Here, I’m giving my database a name called “products”. I’m then telling Terraform to deploy my database in the same resource group as my Cosmos DB account as well as assigning the database to my account name.

We can do this because when we define resources in Terraform, we can export attributes of that resource so that other resources can be linked to it. So in this example, we can assign this database to our Cosmos DB account that we defined earlier because when we defined our account, it exported the name that we defined earlier.

If you want more information on how to create Cosmos DB databases in Terraform, check out the documentation.

Creating Collections

Let’s create a collection to live underneath our database. We can do this with the azurerm_cosmosdb_sql_container resource definition. Add the following Terraform code to your main.tf file:

resource "azurerm_cosmosdb_sql_container" "coll" {
  name = "Clothes"
  resource_group_name = "${azurerm_cosmosdb_account.acc.resource_group_name}"
  account_name = "${azurerm_cosmosdb_account.acc.name}"
  database_name = "${azurerm_cosmosdb_sql_database.db.name}"
  partition_key_path = "/ClothesId"
}

In this resource, I’ve given my container a name (‘Clothes’), assigned it to my resource group, account and database and then given my container a Partition Key of “/ClothesId”.

Partitioning is a vital concept in Cosmos DB so if you want to learn more about it (you should), please read this article.

Complete Terraform file

Our main.tf file now looks like this:

provider "azurerm" {
  version = "~> 1.34.0"
  subscription_id = "${var.subscription_id}"
  tenant_id = "${var.tenant_id}"
}
resource "azurerm_resource_group" "rg" {
  name = "${var.resource_group_name}"
  location = "${var.resource_group_location}"
}
resource "azurerm_cosmosdb_account" "acc" {
  name = "${var.cosmos_db_account_name}"
  location = "${azurerm_resource_group.rg.location}"
  resource_group_name = "${azurerm_resource_group.rg.name}"
  offer_type = "Standard"
  kind = "GlobalDocumentDB"
  enable_automatic_failover = true
  consistency_policy {
    consistency_level = "Session"
  }

  geo_location {
    location = "${var.failover_location}"
    failover_priority = 1
  }
  geo_location {
    location = "${var.resource_group_location}"
    failover_priority = 0
  }
}
resource "azurerm_cosmosdb_sql_database" "db" {
  name = "products"
  resource_group_name = "${azurerm_cosmosdb_account.acc.resource_group_name}"
  account_name = "${azurerm_cosmosdb_account.acc.name}"
}
resource "azurerm_cosmosdb_sql_container" "coll" {
  name = "Clothes"
  resource_group_name = "${azurerm_cosmosdb_account.acc.resource_group_name}"
  account_name = "${azurerm_cosmosdb_account.acc.name}"
  database_name = "${azurerm_cosmosdb_sql_database.db.name}"
  partition_key_path = "/ClothesId"
}

Our completed variables.tf file looks like this:

variable "resource_group_name" {
  default = "VelidaCosmosDBTerraform-rg"
}
variable "resource_group_location" {
  default = "australiaeast"
}
variable "subscription_id" {
  default = "<PUT_YOUR_SUBSCRIPTION_ID_HERE>"
}
variable "tenant_id" {
  default = "<PUT_YOUR_TENANT_ID_HERE>"
}
variable "cosmos_db_account_name" {
  default = "velidacosmosterraform"
}
variable "failover_location" {
  default = "australiasoutheast"
}

Everything has now been set up, so let’s deploy it to Azure!

Deploying to Azure

First, we’ll need to login to Azure via the command line. If you haven’t got the AZ CLI installed yet, follow this article.

If you’ve got it already, run the following command:

az login

This should open up a browser which will allow you to login. If successful, you’ll get a response with the Azure Subscriptions that are associated with your Microsoft account. Verify that your subscription id and tenant id are correct.

Once you’ve logged in, we can now start deploying resources to our Azure account.

When we build infrastructure in Terraform, we need to perform the following three steps:

1. Terraform Init
2. Terraform Plan
3. Terraform Apply

terrafrom init will create a new configuration for our Terraform environment. It will create a directory called .terraform and download any plugins needed for the configuration. It will also configure the back-end for the Terraform state. Each Terraform provider will have it’s own binaries that are separate from Terraform itself, so the init command will download and install these based on what provider we are using.

Run terraform init in the same directory as your Terraform files to initialize Terraform. If successful, you should see the following response:

Before we create infrastructure in Terraform, we need to generate an execution plan. This will tell us what actions Terraform will take in order for our configuration to deploy resources to Azure and in which order those actions will be taken.

terraform plan also checks our syntax and attempts to connect to our Azure account to check if there are any differences between our Terraform code and our current state. We can save our plan to a file by specifying a file name using the -out argument. But we won’t do this for this tutorial.

Type in terraform plan and you should get the following response:

Now, we can run terraform apply. This command is used to apply our changes. We should run this command in the same directory as the main.tf file.

The output will show the execution plan and describe what actions Terraform will take to change the infrastructure. Before the changes are actually applied, we need to give Terraform approval to make the changes:

Type yes, and our changes will start to take affect. This can take a while (it took around 7 minutes on my machine, so grab a cup of green tea while you wait).

If the execution was successful, we should see the following output. We’ve created three resources, one for our account, one for our database and one for our collection.

Head to the portal and you should see your brand new Cosmos DB account ready and waiting for you to use!

If we don’t want to use this resource anymore and you just want to delete it for the purpose of the demo, we can do this via the terraform destroy command.

Once again, this may take a view minutes. Once this is done, you’ll see the following output:

And there you have it! You’ve just deployed a Distributed NoSQL database to Azure via Terraform in less than 70 lines of code. Did you get that green tea yet? Because you’ve earned it!

What you can’t do

While we can create Accounts, Databases and Collections in Cosmos DB via Terraform, as of writing, we can’t define custom indexing policies, deploy any server-side programming artifacts (stored procs, triggers and user-defined functions).

If you want to do this, I’d highly recommend checking out the CosmosDB Powershell module or using the REST API.

Interestingly enough, I tried setting throughput at the container level, but I kept getting errors :( Might raise a issue on GitHub for it.

Conclusion

Even though this example is very basic, I hope you can see how easy it is to create Azure Cosmos DB resources using Terraform. In more complex scenarios, we can create other types of Azure resources and associate them with our Cosmos DB infrastructure, such as Azure Key Vault to protect our keys and connection strings or Log Analytics to help us monitor activity inside our Cosmos DB accounts.

As always, if you have any questions or comments, please feel free to ask and I’ll be happy to help.