Terraform module for custom Azure policies

Terraform modules are used to create reusable components that include groups of resources meant to be deployed together. This is a natural fit for custom Azure policies and initiatives since it allows organizations to implement all those definitions in a centralized component.

This post goes through an example that shows how to implement and test a Terraform module that defines custom Azure policies and initiatives. The full version of the code can be found in the following repo:

beandrad / terraform-azurerm-policy-sample

Minimal Terraform module defining Azure policies and initiatives

Module structure

Hashicorp published a set of guidelines on module definitions stating some naming conventions and the general module structure.

According to those guidelines, module repository names follow the format terraform-<PROVIDER>-<NAME>, where PROVIDER in our case is AzureRM and NAME is a label describing the infrastructure provided. In our case, the Terraform module is called terraform-azurerm-policy-sample.

The module structure follows the standard practices recommended by Hashicorp; note, however, that some of the paths defined are specific to this module.

• main.tf. Defines the configuration requirements. It may also configure other resources.

• variables.tf. Declares the module input variables.

• outputs.tf. Declares the module outputs; in this case, the initiative IDs.

• /policies. (Module specific) Includes the definitions of custom Azure policies. Each policy has its own folder, where the file policy-rule.json has the definition and policy-parameters.json, the parameters if applicable.

• <label>-initiative.tf. (Module specific) Configures the definition of policies and initiatives in Azure. Policies are grouped into initiatives based on the resources they affect and/or by industry-level standards (such as the CIS Hardening Guidelines).

• initiative-parameters.tf. (Module specific) Declares the input parameters of all the initiatives defined in the module.

• /tests. Implements the module acceptance tests.

Definition of policies and initiatives

This module defines custom policies and initiatives under a management group (definition_management_group in variables.tf) or current subscription if the management group name is not defined.

Custom policy definitions are created using the azurerm_policy_definition resource and built-in policies are imported using the azurerm_policy_definition data resource. Both resources are included in the corresponding initiatives Terraform configuration files; unless they are shared across initiatives, in which case they are defined in the main.tf file.

It is important to note that policy data resources should be imported using its policy name (as opposed to the displayName). The reason is that the displayName is not unique and it may change, whereas the name is unique and it remains the same until the policy is deleted. In the configuration, the displayName appears commented out as it describes the policy being imported.

Acceptance tests

The acceptance tests deploy resources to Azure to check whether the defined initiatives actually work: non-compliant resources cannot be deployed whereas compliant ones are allowed to be deployed.

The following conventions were followed when testing the policy module:

• Tests check that the initiative is working as expected, as opposed to testing individual policies. The reason is that this module only outputs initiatives, all the policies are linked from an initiative.

• Only the behavior of custom policies is tested; built-in policies are expected to work.

Tests are implemented using the Go testing framework together with the Terratest module. This configuration allows calling the Terraform configuration from the Go tests.

The lifecycle of the tests is as follows:

• Setup: load the policy module to define policies and initiatives and assign initiatives in Azure (tests/terraform/main.tf).
• Run: try to create compliant resources (for example, tests/terraform/resource-location-allow) and non-compliant resources (for example, tests/terraform/resource-location-audit).
• Assert: check whether the policy has been correctly applied using the returned error from the Terraform apply for deny effect or the policy state for audit effect.
• Teardown: delete test resources from Azure.

Running this kind of tests is slow, in particular, those checking effects other than deny. There are two main components that cause this delay: the first one is policy definition and assignment, and the second one is policy evaluation (which, as stated above, is required to check the audit effect).

In order to speed up the tests, test cases are run in parallel using the Parallel() function.

Finally, regarding the teardown, it is important to note that it is done in two different steps: one for the resources provisioned during the setup and the other for the resources deployed by each of the test cases. In the first case, the Cleanup function is used; defer wouldn't work since deferred functions are run before parallel subtests are executed. On the other hand, resources created by the test cases are destroyed in a deferred function.

Final thoughts

As stated in the beginning, the aim of this post is to provide a baseline example of an Azure policy module; however, it is important to point out that quite a few opinionated design decisions have been made and, therefore, this example shouldn't be taken as the only correct way of implementing an Azure policy module.

Having said that, I hope this post helps those devs out there looking for some guidance on how to implement Terraform modules, define Azure custom policies and initiatives, and test those definitions.