Exploring The Possibilities of Infrastructure as Code

We've all been there. Development is complete on the brand new shiny app. The development team have done a great demo, the product owner is happy, the stakeholders are trained up and the QA team have given their approval in a dedicated test environment.

So on day one, the team excitedly deploy the new application to the live environment... but something isn't working. It turns out that Tom had misconfigured a setting in the live environment.

But it's not really Tom's fault. They should never have been expected to remember every bit of configuration that was needed.

With infrastructure as code, we can prevent this from ever happening. We can remove the human dependency and easily allow our infrastructure changes to be reviewed before they are made.

What is infrastructure as code?

Infrastructure as code (IaC) is the managing and provisioning of infrastructure through code instead of manual processes. Configuration and code files are used to define your infrastructure clearly and concisely.

IaC, like other architecture concepts such as Microservices, has matured from being something of an industry trend to a way for organisations to make massive operational improvements.

There are two approaches to IaC - Imperative and Declarative.

An imperative approach is to define the specific steps needed to provision the infrastructure, for example, using a PowerShell script to run Azure CLI commands.

A declarative approach is to define the desired state of the infrastructure and an IaC tool will configure it for you.

Generally, a declarative approach is preferred as configuration with an IaC tool is easier and quicker to create than with an imperative approach.

Examples of IaC tools include Azure Resource Manager (ARM) Templates (with Azure Bicep), Terraform, Chef and Ansible. At Audacia we use Terraform, a declarative IaC tool created by HashiCorp.

Why we use IaC

Traditionally, managing and provisioning infrastructure is a time-consuming, manual process. A key part of DevOps is to automate processes so that they can be quicker and more reliable. Using IaC allows us to achieve this automation and reliability so that we can more be confident when releasing.

Using IaC provides us with many advantages.

Version control

The configuration can be stored in a Version Control System either alongside our code or separately. This means that all changes to the infrastructure must go through our code review process. We also get all the benefits of using version control such as being able to easily revert changes if something goes wrong in a test environment.

Provisioning speed

There is no longer a dependency on a SysAdmin to do the provisioning. The developer can now be responsible for it whilst developing their feature. This also means that the SysAdmin has more time to do other tasks.

Prevents environment drift

Environment drift is where small, manual tweaks are made to infrastructure over time, causing them to drift from the expected configuration. An environment can become a snowflake, where it is in a unique configuration that cannot be reproduced automatically.

By preventing environment drift, we can keep infrastructure consistent across test and production environments, improving our testing reliability and confidence.

We are now able to lock down manual changes to environments with permissions. Team members cannot modify resources manually, all changes must be made through code reviewed IaC.

Templating

New projects can quickly get environments provisioned by using templates. Our projects are usually hosted in Azure with a SQL Server and App Services for a Web UI, an API and an Authentication Service. We can share these templates and significantly reduce the time between starting a project and deploying it to a test environment.

Infrastructure experience

More junior developers can be exposed to how infrastructure works and therefore gain valuable experience at an earlier stage in their career.

Terraform

As mentioned earlier, at Audacia we use Terraform as our IaC tool. We chose Terraform due to the fact it has multi-cloud support and can integrate with other SaaS solutions such as MongoDB and SendGrid. The only drawback we have encountered so far is that some Azure features are missing, for example, you cannot set an App Service setting to be sticky to a slot.

Terraform uses a custom HashiCorp Configuration Language (HCL) for its configuration files. This allows for concise descriptions of resources using blocks, arguments and expressions.

HCL follows a simple structure:

resource "<resource_type>" "<label>" {
  <identifier> = <expression>
}

• A resource block indicates that something should be provisioned
• The <resource_type> defines the resource to provision, e.g., azurerm_app_service is an Azure App Service
• The <label> provides a reference to the resource
• The resource is then made up of <identifier>s which are used to set the configuration of the resources
• An <expression> can be a simple value such as a string, number, Boolean or a more complex nested object

Example

Let's try an example of a simple setup in Azure. We will provision a Resource Group that contains an App Service Plan and an App Service for a .NET Core Weather Forecast API.

Configuration

Terraform configuration is stored in .tf files, the convention being to use a file named main.tf.

The first thing we need to do is define that we want to use the Azure Provider. A Provider is a logical abstraction in HCL of an upstream API. Here we specify that there is a required_provider called azurerm.

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "=3.0.2"
    }
  }
}

provider "azurerm" {
  features {}
}

Next, we can create a Resource Group to store our API using the "azurerm_resource_group" resource. We provide a name and a valid Azure location.

resource "azurerm_resource_group" "resource_group" {
  name     = "weather-forecast-resource-group"
  location = "UK South"
}

We can now add an App Service Plan with the "azurerm_app_service_plan" resource. We give it a name and configure the SKU we want.

To set the location we can use the label of the Resource Group resource, ensuring they are in the same location without specifying it multiple times.

The syntax for accessing a previously defined resource is <resource_type>.<label>.<property>. This is used to access the given resource type with a specified label, then get the location property from it. We can do the same thing to get the name of the Resource Group to put the App Service Plan in.

resource "azurerm_service_plan" "app_service_plan" {
  name                = "weather-forecast-plan"
  location            = azurerm_resource_group.resource_group.location
  resource_group_name = azurerm_resource_group.resource_group.name
  os_type             = "Windows"
  sku_name            = "S1"
}

The final step is to add an App Service with the "azurerm_windows_web_app" resource. Again, we set the name (this must be globally unique in Azure) and use the labels of previous steps to ensure it is created under the correct resources.

resource "azurerm_windows_web_app" "app_service_api" {
  name                = "jp01-weather-forecast-api"
  location            = azurerm_resource_group.resource_group.location
  resource_group_name = azurerm_resource_group.resource_group.name
  service_plan_id     = azurerm_service_plan.app_service_plan.id

  site_config {}
}

Provisioning

Now that we have a fully constructed Terraform configuration file, we can use the Terraform CLI to validate our configuration and provision the infrastructure.

First, we run terraform init. This will set up our working directory for further commands as well as downloading any providers that we have used.

Next, we run terraform validate to ensure the configuration we have written is valid. If all goes well, we should see a success message.

Then we can use terraform plan to see what Terraform is going to do when it is run. This will show a diff like output showing the resources that will be added, changed and destroyed. The summary correctly states that 3 new resources will be added and the diff shows that all of our resources will be created (for brevity, the App Service has been removed).

Once we are happy with all of the changes that the plan says will be made, we can run terraform apply for the provisioning to take place.

With that completed, we can now see that our resources have been created in Azure.

Further Steps

Now that we have some resources being managed and provisioned by Terraform, there are a lot more possibilities to expand on.

Modifying the Configuration

We can of course modify the configuration that we have set up to make changes to our resources. We could modify the SKU of the App Service Plan to a higher tier, or provide a more advanced config to the App Service.

A developer may add a Web UI to our Weather Forecasting project. Whilst they are developing the feature, they can easily add the required infrastructure by updating the Terraform configuration. Adding another "azurerm_windows_web_app", a new App Service can be provisioned for the Web UI to be deployed to. This change to the infrastructure would then be code reviewed at the same time as the code change required for the feature.

Multiple Environments

Our projects are usually deployed to at least three environments - QA, UAT and Production. It would be impractical to copy the configuration per environment and would potentially lead to different infrastructure in each.

Terraform allows us to set variables, which can then be changed when plan and apply are run.

In our configuration, we specify that there is an input variable called environment. We also create a local variable fullName which concatenates our weather-forecast name with the environment input variable.

variable "environment" {
  type = string
}

locals {
  fullName = "weather-forecast-${var.environment}"
}

We can then use our fullName variable throughout the configuration to change the names of our resources, for example, the Resource Group.

resource "azurerm_resource_group" "resource_group" {
  name     = "${local.fullName}-resource-group"
  location = "UK South"
}

To set our input variable, we can specify a value from the CLI when we run apply.

terraform apply -var="environment=qa"

Changing the value of the variable allows us to deploy the same configuration multiple times. We can run the apply command again but set the environment value to uat instead.

After running with both values, we end up with all of these resources created in Azure.

Using in Continuous Integration and Continuous Delivery

Whilst we have been able to use IaC to automate and improve the reliability of provisioning our infrastructure, we can automate further. Using the CLI commands we can easily add steps to both our Continuous Integration (CI) and Continuous Delivery (CD) pipelines.

In CI we could add the validate command to our Pull Request Build Validation. With this, a developer will not be able to complete a PR unless the Terraform configuration is valid. This means the code in our dev branch is always ready to be deployed.

For CD we can apply our Terraform Configuration just before we deploy to each environment, changing the environment input variable each time. Any features merged in by a developer that requires an infrastructure change will always be deployed together.

Conclusion

Using infrastructure as code has allowed us to remove the manual and time-consuming process of managing and provisioning infrastructure.

By integrating Terraform into our CI and CD pipelines we can ensure that our test and production environments are always up-to-date and consistent with one another.

At Audacia, we aim to implement Terraform on all of our new projects and are looking to expand usage to existing projects where possible. We are also keen to try out Azure Bicep as another tool in our belt, giving us more opportunities to gain the many benefits of using infrastructure as code.

By Jack Percy, Senior Software Developer at Audacia