Infrastructure deployment with Terraform and Azure DevOps CI-CD Pipelines

In this post, I’d be deploying the demo AKS infrastructure in an automated way using terraform and Azure DevOps. The shopping list for this recipe is as:

1. Azure subscription (Imp: Ensure to keep an eye on billing)
2. Code editor of your choice (In my case, I have cloned VS Code with Azure Repos)
3. Azure DevOps account
4. Storage account and a container (Storing remote backend for terraform) In my case I have created a resource group, a storage account, and a container within it (can be deployed later as well). In addition to having a storage account, it is necessary to create a service principal and capture the following information to be updated in terraform variable file :

Client id : xxxxxxxxxxxxxxxxxxx
Client secret : xxxxxxxxxxxxxxxxxxx
Tenant id : xxxxxxxxxxxxxxxxxxx
Subscription id : : xxxxxxxxxxxxxxxxxxx

This is a terraform configuration snip from my VS code editor wherein the specifics for terraform and azure provider version is been called out along with the rest of the deployable configuration and are committed in Azure Repos.

I have created a new project named Terraform-K8S-deploy in DevOps to start with ci-cd pipelines. Though there are different ways for setting up the pipelines however I have chosen classic editor while setting up the pipeline just to keep it simple.

Until this point, we have our org and project setup ready. Let us now set up our build pipelines. My build pipeline will source the code from Azure Repos, however, you can make your own Repos selection based on your code availability.

Let us start configuring the CI pipeline, we will copy and publish our terraform code to artifacts directory and make it available for further deployment through CD pipeline in Azure.

Select repository based on your source code > click continue. The next page will prompt for the template. Select “empty job ” as at this stage there is no defined template for terraforming.

To configure the copy file and artifacts publish job > click the agent job plus sign (+) for adding a task.
On the top right corner search for copy files and publish build artifacts tasks and add them to the job list by clicking the add button.

Select copy files task and configure the source folder location which essentially the terraform code location in repos. Add the target folder location where the files will be copied.

When the copy files task configuration is finished, select publish artifact task > configure “path to publish” to publish the artifacts.

Now the build pipeline is ready – just save and run!!

The progress window confirms that the agents have done their job and the artifacts are ready to be consumed by the CD pipeline.

At this stage we are done with creating a CI pipeline, in the next stage let us setup a CD pipeline.

Our build artifacts are ready, now it’s time to pass those artifacts through release pipeline for further deployment.

On the DevOps portal, select pipeline > release > new release pipeline - Currently no defined terraform designer template is available therefore we will start by selecting “empty job” from pipeline create wizard.

• Rename the release pipeline if required according to your need.
• Select and add the artifacts (which we already build earlier at the time of setting up the CI pipeline. Refer to picture)
• Select Build as the source type and populate the respective project field information (Refer to picture)

Click “ADD”

Now the artifacts are ready holding terraform files! let’s create the deployment stages by configuring respective jobs and tasks.

Search terraform on the top right corner and select terraform installer– configure it based on the terraform version you would like to use.

Similarly search and add “terraform CLI” job from the marketplace. However!! before we go ahead and configure it - let us understand terraform deployment process and phases for simplicity.

Deployment through terraform is a 3-stage process which involves:

• Terraform init – “Init” parameter installs the terraform modules and selected providers plugins – for example, Azure, AWS, GCP, etc..etc
• Terraform plan – “Plan” parameter performs a code configuration sanity check, allows a dry run to visualize the resources to be deployed in the infrastructure.
• Terraform apply - If the plan looks good, initiating “apply” command executes the infrastructure building defined as part of IaC

Initiating the above command prepares and fires up the infra deployment. Let us configure our jobs.

After adding the “terraform CLI” agent jobs –

• Setup job “display name” to suit the requirement
• Select “init” from the “command” drop-down list
• Select a configuration directory where we have a compiled terraform files (artifacts location)

The backend configuration will be a placeholder to store terraform statefile, continue populating the backend configuration fields which requires RG name, storage account and container information (supposed to be created in the beginning), no worries if it is not yet created – check the “create backend” box and fill the details as needed.

Once all the fields are properly populated, clone the “init” task twice by right-clicking on it to configure for “plan” and “apply” job parameters.

In the terraform plan task – select “plan” from the “command” drop-down list rest of the field would remain as is.
Similarly, for the next cloned task for “terraform apply” select “apply” from the “command” drop-down list in addition to adding “-auto-approve” command parameter option for approval consent.

(Refer to picture)

In the meanwhile, a quick AKS snip from the azure portal – no AKS service is available at this moment.

Voila!! Our release pipeline is ready. Just click Save and create a release to trigger the pipeline

And Bingo!! After 6 min and 29 seconds, glad to see the agent progress outcome.

A successful Azure Kubernetes services deployment available for consumption

To conclude –
In this article we have configured the CICD pipeline in Azure DevOps leveraging terraform as our primary source for infrastructure as code. We have utilized the terraform jobs in Azure DevOps to further deploy infrastructure in Azure.

I hope you'll find this post useful!