DEV Community

Cover image for Terraform plan in Devops GUI using templates
Arindam Mitra
Arindam Mitra

Posted on • Edited on

Terraform plan in Devops GUI using templates

#azure #devops #terraform #iac

Greetings my fellow Technology Advocates and Specialists!!!

This Blog post is a follow-up to my previous post - Publish Terraform Plan in Azure DevOps GUI

In this Session, I will demonstrate how to Publish Terraform Plan in Azure DevOps GUI Using PIPELINE TEMPLATES.

THIS IS HOW IT LOOKS AT THE END!!!
Image description
REQUIREMENTS:-
  1. Azure Subscription.
  2. Azure DevOps Organisation and Project.
  3. Service Principal with Delegated Graph API Rights and Required RBAC (Typically Contributor on Subscription or Resource Group)
  4. Azure Resource Manager Service Connection in Azure DevOps.
  5. Azure Pipelines Terraform Tasks Extension by Charles Zipp Installed in Azure DevOps.
CODE REPOSITORY:-

GitHub logo arindam0310018 / 06-July-2022-DevOps__Publish-Terraform-Plan-In-DevOps-GUI-Using-Pipeline-Templates

TERRAFORM PLAN IN DEVOPS GUI USING TEMPLATES

TERRAFORM PLAN IN DEVOPS GUI USING TEMPLATES

Greetings my fellow Technology Advocates and Specialists!!!

This Blog post is a follow-up to my previous post - [Publish Terraform Plan in Azure DevOps GUI] (https://dev.to/arindam0310018/terraform-plan-in-devops-gui-52fp)

In this Session, I will demonstrate how to Publish Terraform Plan in Azure DevOps GUI Using PIPELINE TEMPLATES.

THIS IS HOW IT LOOKS AT THE END!!!
Image description
REQUIREMENTS:-
  1. Azure Subscription.
  2. Azure DevOps Organisation and Project.
  3. Service Principal with Delegated Graph API Rights and Required RBAC (Typically Contributor on Subscription or Resource Group)
  4. Azure Resource Manager Service Connection in Azure DevOps.
  5. Azure Pipelines Terraform Tasks Extension by Charles Zipp Installed in Azure DevOps.
EXTENSION DETAILS:-
NAME:
Azure Pipelines Terraform Tasks
WHERE TO FIND:
https://marketplace.visualstudio.com/items?itemName=charleszipp.azure-pipelines-tasks-terraform&targetId=11c5414f-ba26-4659-87a7-aa40610cf74a&utm_source=vstsproduct&utm_medium=ExtHubManageList
Image description
INSTALLED IN AZURE DEVOPS ORGANISATION:
Image description
OBJECTIVE:-
Deploy a Resource Group and Log Analytics Workspace.
Publish the Terraform Plan in Azure DevOps GUI using Pipeline Templates.
HOW DOES MY CODE PLACEHOLDER LOOKS LIKE:-
Image description
ORDER
…
View on GitHub
EXTENSION DETAILS:-
NAME:
Azure Pipelines Terraform Tasks
WHERE TO FIND:
https://marketplace.visualstudio.com/items?itemName=charleszipp.azure-pipelines-tasks-terraform&targetId=11c5414f-ba26-4659-87a7-aa40610cf74a&utm_source=vstsproduct&utm_medium=ExtHubManageList
Image description
INSTALLED IN AZURE DEVOPS ORGANISATION:
Image description
OBJECTIVE:-
Deploy a Resource Group and Log Analytics Workspace.
Publish the Terraform Plan in Azure DevOps GUI using Pipeline Templates.
HOW DOES MY CODE PLACEHOLDER LOOKS LIKE:-
Image description
ORDER OF THE PIPELINES:-
Below is the order, the Pipelines are called -
azure-pipelines-v1.0.yml
azure-pipelines-v1.0-Main.yml
azure-pipelines-v1.0-PublishPlan.yml
azure-pipelines-v1.0-Validate.yml
azure-pipelines-v1.0-Deploy.yml
ADVANTAGES WITH PIPELINE TEMPLATES:-
1. Scalable
2. Advocates DRY (Don't repeat yourself) Principle
3. Each YAML Pipeline file (Template) can be a Task or a Stage
PIPELINE CODE SNIPPET:-
AZURE DEVOPS YAML PIPELINE TEMPLATE (azure-pipelines-v1.0.yml):- 


trigger:
  none

stages:
  - template: ./azure-pipelines-v1.0-Main.yml
    parameters:
          container_name: "terraform"
          container_key: "PUBLISH-TF-PLAN/LogaPublishTFPlan.tfstate"
          environment_name: "NonProd"
          planfilename: "tfplan"
          tfvarfilename: "loga.tfvars"
          root_directory: "/Publish-TF-Plan-In-GUI-DevOps-Templates"
          service_connection_name: "amcloud-cicd-service-connection"
          backend_resource_group: "tfpipeline-rg"
          backend_storage_accountname: "tfpipelinesa"
          storageAccountSku: "Standard_LRS"
          pool:
            vmImage: "ubuntu-latest"  
          terraformVersion: "latest"
          environment_name_Job: "ARINDAM_DEPLOYMENT"
Enter fullscreen mode Exit fullscreen mode
NOTE:-
No User Input Required.
Any Parameters Value can be changed.
All YAML pipelines templates are build using Parameters. No Values are Hardcoded.
AZURE DEVOPS YAML PIPELINE TEMPLATE (azure-pipelines-v1.0-Main.yml):- 


parameters:
  container_name:
  container_key:
  environment_name:
  planfilename:
  tfvarfilename:
  root_directory:
  service_connection_name:
  backend_resource_group:
  backend_storage_accountname:
  storageAccountSku:
  environment_name_Job:
  terraformVersion: 
  pool:
    vmImage:

stages:

  - stage: Publish_Plan
    jobs:
    - template: azure-pipelines-v1.0-PublishPlan.yml
      parameters:
        container_name: ${{ parameters.container_name }}
        container_key: ${{ parameters.container_key }}
        environment_name: ${{ parameters.environment_name }}
        planfilename: ${{ parameters.planfilename }}
        tfvarfilename: ${{ parameters.tfvarfilename }}
        root_directory: ${{ parameters.root_directory }}
        service_connection_name: ${{ parameters.service_connection_name }}
        backend_resource_group: ${{ parameters.backend_resource_group }}
        backend_storage_accountname: ${{ parameters.backend_storage_accountname }}
        backendAzureRmStorageAccountSku: ${{ parameters.backend_storageAccountSku }}
        terraformVersion: ${{ parameters.terraformVersion }}
        pool: ${{ parameters.pool }}

  - stage: Validate
    jobs:
    - template: azure-pipelines-v1.0-Validate.yml
      parameters:
        container_name: ${{ parameters.container_name }}
        container_key: ${{ parameters.container_key }}
        environment_name: ${{ parameters.environment_name }}
        planfilename: ${{ parameters.planfilename }}
        tfvarfilename: ${{ parameters.tfvarfilename }}
        root_directory: ${{ parameters.root_directory }}
        service_connection_name: ${{ parameters.service_connection_name }}
        backend_resource_group: ${{ parameters.backend_resource_group }}
        backend_storage_accountname: ${{ parameters.backend_storage_accountname }}
        backendAzureRmStorageAccountSku: ${{ parameters.backend_storageAccountSku }}
        terraformVersion: ${{ parameters.terraformVersion }}
        pool: ${{ parameters.pool }}

  - stage: Deploy
    condition: |
      and(succeeded(),
       eq(variables['build.sourceBranch'], 'refs/heads/main') 
      )
    dependsOn: "Validate"
    jobs:
      - deployment: ${{ parameters.environment_name_job }}
        pool: 
          vmImage: ${{ parameters.vmImage }}
        displayName: Deploy
        environment: ${{ parameters.environment_name }}
        strategy:
          runOnce:
            deploy:
              steps:
              - template: azure-pipelines-v1.0-Deploy.yml
                parameters:
                  container_name: ${{ parameters.container_name }}
                  container_key: ${{ parameters.container_key }}
                  environment_name: ${{ parameters.environment_name }}
                  tfvarfilename: ${{ parameters.tfvarfilename }}
                  root_directory: ${{ parameters.root_directory }}
                  service_connection_name: ${{ parameters.service_connection_name }}
                  backend_resource_group: ${{ parameters.backend_resource_group }}
                  backend_storage_accountname: ${{ parameters.backend_storage_accountname }}
                  backendAzureRmStorageAccountSku: ${{ parameters.backend_storageAccountSku }}
                  terraformVersion: ${{ parameters.terraformVersion }}
Enter fullscreen mode Exit fullscreen mode
NOTE:-
  1. This is a 3 Stage Pipeline Template.
  2. The Names of the Stages are - a) PUBLISH_PLAN b) VALIDATE, and c) DEPLOY
  3. All Parameters defined in "azure-pipelines-v1.0.yml" are referenced in the above Pipeline "azure-pipelines-v1.0-Main.yml".
  4. PUBLISH_PLAN Stage References to "azure-pipelines-v1.0-PublishPlan.yml".
  5. VALIDATE Stage References to "azure-pipelines-v1.0-Validate.yml".
  6. DEPLOY Stage References to "azure-pipelines-v1.0-Deploy.yml".
  7. DEPLOY Stage will Execute only if the following conditions are met - a) VALIDATE Stage gets completed successfully. b) Source Branch = Main. If not, DEPLOY Stage will get Skipped Automatically.
  8. DEPLOY Stage will Execute only after Approval. The Approval is integrated with Environment defined in the Pipeline Parameters Section.
AZURE DEVOPS YAML PIPELINE TEMPLATE (azure-pipelines-v1.0-PublishPlan.yml):- 


parameters:
  container_name:
  container_key:
  environment_name:
  planfilename:
  tfvarfilename:
  root_directory:
  service_connection_name:
  backend_resource_group:
  backend_storage_accountname:
  storageAccountSku:
  environment_name_Job:
  terraformVersion: 
  pool:
    vmImage:

jobs:
  - job: Publish_Plan
    pool: ${{ parameters.pool }}
    steps:
# Install Terraform Installer in the Build Agent:-
      - task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0
        displayName: INSTALL TERRAFORM VERSION
        inputs:
          terraformVersion: ${{ parameters.terraformVersion }}
# Terraform Init:-
      - task: TerraformCLI@0
        displayName: TERRAFORM INIT
        inputs:
          backendType: 'azurerm'
          command: 'init'
          workingDirectory: '$(System.DefaultWorkingDirectory)/${{ parameters.root_directory }}'
          backendServiceArm: ${{ parameters.service_connection_name }} 
          backendAzureRmResourceGroupName: ${{ parameters.backend_resource_group }}
          backendAzureRmStorageAccountName: ${{ parameters.backend_storage_accountname }}
          backendAzureRmStorageAccountSku: ${{ parameters.backend_storageAccountSku }}
          backendAzureRmContainerName: ${{ parameters.container_name }}
          backendAzureRmKey: ${{ parameters.container_key }}
# Terraform Validate:-
      - task: TerraformCLI@0
        displayName: TERRAFORM VALIDATE
        inputs:
          backendType: 'azurerm'
          command: 'validate'
          workingDirectory: '$(System.DefaultWorkingDirectory)/${{ parameters.root_directory }}' 
          environmentServiceName: ${{ parameters.service_connection_name }}
# Terraform Plan:-
      - task: TerraformCLI@0
        displayName: TERRAFORM PLAN
        inputs:
          backendType: 'azurerm'
          command: 'plan'
          workingDirectory: '$(System.DefaultWorkingDirectory)/${{ parameters.root_directory }}'
          commandOptions: "--var-file=${{ parameters.tfvarfilename }} --out=${{ parameters.planfilename }}"
          environmentServiceName: ${{ parameters.service_connection_name }}
          publishPlanResults: 'tfplan'
Enter fullscreen mode Exit fullscreen mode
PUBLISH_PLAN STAGE PERFORMS BELOW:-
## TASKS
1. Terraform Installer installed in Azure DevOps Build Agent.
2. Terraform Init
3. Terraform Validate
4. Terraform Plan
5. Publish Terraform Plan in Azure DevOps GUI.
NOTE:- 


- task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0
Enter fullscreen mode Exit fullscreen mode
EXPLANATION:-
Instead of using TerraformInstaller@0 YAML Task, I have specified the Full Name. This is because I have two Terraform Extensions in my DevOps Organisation and with each of the Terraform Extension, exists the Terraform Install Task
The Names of the Extensions are listed below:-
1. Terraform by Microsoft DevLabs
2. Azure Pipelines Terraform Tasks by Charles Zipp
If Full Name is not provided, then below Error is Encountered:-
Image description

Alternatively, below can also be used as Full Name:-



- task: charleszipp.azure-pipelines-tasks-terraform.azure-pipelines-tasks-terraform-installer.TerraformInstaller@0
Enter fullscreen mode Exit fullscreen mode
AZURE DEVOPS YAML PIPELINE TEMPLATE (azure-pipelines-v1.0-Validate.yml):- 


parameters:
  container_name:
  container_key:
  environment_name:
  planfilename:
  tfvarfilename:
  root_directory:
  service_connection_name:
  backend_resource_group:
  backend_storage_accountname:
  storageAccountSku:
  environment_name_Job:
  terraformVersion: 
  pool:
    vmimage:

jobs:
  - job: Validate
    pool: ${{ parameters.pool }}
    steps:
# Install Terraform Installer in the Build Agent:-
      - task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0
        displayName: INSTALL TERRAFORM VERSION
        inputs:
          terraformVersion: ${{ parameters.terraformVersion }} 
# Terraform Init:-
      - task: TerraformCLI@0 
        displayName: TERRAFORM INIT
        inputs:
          backendType: 'azurerm'
          command: 'init'
          workingDirectory: '$(System.DefaultWorkingDirectory)/${{ parameters.root_directory }}'
          backendServiceArm: ${{ parameters.service_connection_name }} 
          backendAzureRmResourceGroupName: ${{ parameters.backend_resource_group }}
          backendAzureRmStorageAccountName: ${{ parameters.backend_storage_accountname }}
          backendAzureRmStorageAccountSku: ${{ parameters.backend_storageAccountSku }}
          backendAzureRmContainerName: ${{ parameters.container_name }}
          backendAzureRmKey: ${{ parameters.container_key }}
# Terraform Validate:-
      - task: TerraformCLI@0
        displayName: TERRAFORM VALIDATE
        inputs:
          backendType: 'azurerm'
          command: 'validate'
          workingDirectory: '$(System.DefaultWorkingDirectory)/${{ parameters.root_directory }}' 
          environmentServiceName: ${{ parameters.service_connection_name }}
# Terraform Plan:-
      - task: TerraformCLI@0
        displayName: TERRAFORM PLAN
        inputs:
          backendType: 'azurerm'
          command: 'plan'
          workingDirectory: '$(System.DefaultWorkingDirectory)/${{ parameters.root_directory }}'
          commandOptions: "--var-file=${{ parameters.tfvarfilename }} --out=${{ parameters.planfilename }}"
          environmentServiceName: '${{ parameters.service_connection_name }}'
# Copy Files to Artifacts Staging Directory:-
      - task: CopyFiles@2
        displayName: COPY FILES ARTIFACTS STAGING DIRECTORY
        inputs:
          SourceFolder: '$(System.DefaultWorkingDirectory)/${{ parameters.root_directory }}'
          Contents: |
            **/*.tf
            **/*.tfvars
            **/*tfplan*
          TargetFolder: $(build.artifactstagingdirectory)/AMTF
# Publish Artifacts:-
      - task: PublishBuildArtifacts@1
        displayName: PUBLISH ARTIFACTS
        inputs:
          targetPath: $(build.artifactstagingdirectory)/AMTF
          artifactName: 'drop'
Enter fullscreen mode Exit fullscreen mode
VALIDATE STAGE PERFORMS BELOW:-
## TASKS
1. Terraform Installer installed in Azure DevOps Build Agent.
2. Terraform Init
3. Terraform Validate
4. Terraform Plan
5. Copy the Terraform files (Most Importantly Terraform Plan Output) to Artifacts Staging Directory.
6. Publish Artifacts
AZURE DEVOPS YAML PIPELINE TEMPLATE (azure-pipelines-v1.0-Deploy.yml):- 


parameters:
  container_name:
  container_key:
  environment_name:
  planfilename:
  tfvarfilename:
  root_directory:
  service_connection_name:
  backend_resource_group:
  backend_storage_accountname:
  storageAccountSku:
  environment_name_Job:
  terraformVersion: 

steps:
  - download: none
  - task: DownloadBuildArtifacts@0
    displayName: 'Downloading ${{ parameters.environment_name }} Artifacts'
    inputs:
      buildType: 'current'
      downloadType: 'single'
      artifactName: 'drop'
      downloadPath: '$(System.ArtifactsDirectory)'
# Install Terraform Installer in the Build Agent:-
  - task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0
    displayName: INSTALL TERRAFORM VERSION
    inputs:
      terraformVersion: ${{ parameters.terraformVersion }}
# Terraform Init:-
  - task: TerraformCLI@0
    displayName: TERRAFORM INIT
    inputs:
      backendType: 'azurerm'
      command: 'init'
      workingDirectory: '$(System.ArtifactsDirectory)/drop/AMTF'
      backendServiceArm: ${{ parameters.service_connection_name }} 
      backendAzureRmResourceGroupName: ${{ parameters.backend_resource_group }}
      backendAzureRmStorageAccountName: ${{ parameters.backend_storage_accountname }}
      backendAzureRmStorageAccountSku: ${{ parameters.backend_storageAccountSku }}
      backendAzureRmContainerName: ${{ parameters.container_name }}
      backendAzureRmKey: ${{ parameters.container_key }}
# Terraform Apply:-
  - task: TerraformCLI@0
    displayName: TERRAFORM APPLY
    inputs:
      backendType: 'azurerm'
      command: 'apply'
      workingDirectory: '$(System.ArtifactsDirectory)/drop/AMTF'
      commandOptions: "--var-file=${{ parameters.tfvarfilename }}"
      environmentServiceName: ${{ parameters.service_connection_name }}
Enter fullscreen mode Exit fullscreen mode
DEPLOY STAGE PERFORMS BELOW:-
## TASKS
1. Download the Published Artifacts.
2. Terraform Installer installed in Azure DevOps Build Agent.
3. Terraform Init
4. Terraform Apply
DETAILS AND ALL TERRAFORM CODE SNIPPETS FOLLOWS BELOW:-
TERRAFORM (main.tf):- 


terraform {
  required_version = ">= 1.2.0"

   backend "azurerm" {
    resource_group_name  = "tfpipeline-rg"
    storage_account_name = "tfpipelinesa"
    container_name       = "terraform"
    key                  = "PUBLISH-TF-PLAN/LogaPublishTFPlan.tfstate"
  }
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 3.2"
    }   
  }
}
provider "azurerm" {
  features {}
  skip_provider_registration = true
}
Enter fullscreen mode Exit fullscreen mode
TERRAFORM (loga.tf):- 


# Azure Resource Group:-
resource "azurerm_resource_group" "rg" {
  name     = var.rg-name
  location = var.rg-location
}

## Azure log Analytics Workspace:-

resource "azurerm_log_analytics_workspace" "loga" {
  name                = var.loga-name
  resource_group_name = azurerm_resource_group.rg.name
  location            = azurerm_resource_group.rg.location
  sku                 = var.loga-sku 
  retention_in_days   = var.loga-retention

  depends_on          = [azurerm_resource_group.rg]
}
Enter fullscreen mode Exit fullscreen mode
TERRAFORM (variables.tf):- 


variable "rg-name" {
  type        = string
  description = "Name of the Resource Group"
}

variable "rg-location" {
  type        = string
  description = "Resource Group Location"
}

variable "loga-name" {
  type        = string
  description = "Name of the Log Analytics Workspace"
}

variable "loga-sku" {
  type        = string
  description = "SKU the Log Analytics Workspace"
}

variable "loga-retention" {
  type        = string
  description = "Retention Period of the Log Analytics Workspace"
}
Enter fullscreen mode Exit fullscreen mode
TERRAFORM (loga.tfvars):- 


rg-name         = "AMTESTRG100"
rg-location     = "West Europe"
loga-name       = "AMLOGA100"
loga-sku        = "PerGB2018"
loga-retention  = "30"
Enter fullscreen mode Exit fullscreen mode
ITS TIME TO TEST:-
DESIRED RESULT: Stages - PUBLISH_PLAN, VALIDATE and DEPLOY should Complete Successfully. Terraform Plan Gets Published Successfully in Azure DevOps GUI. Resource Group and Log Analytics Workspace Resources gets deployed.
PIPELINE STAGE PUBLISH_PLAN EXECUTED SUCCESSFULLY:-
Image description
TERRAFORM PLAN GETS PUBLISHED IN AZURE DEVOPS GUI:-
Image description
PIPELINE STAGE VALIDATE EXECUTED SUCCESSFULLY:-
Image description
PIPELINE STAGE DEPLOY WAITING APPROVAL:-
Image description
Image description
PIPELINE STAGE DEPLOY EXECUTED SUCCESSFULLY:-
Image description
PIPELINE OVERALL EXECUTION STATUS:-
Image description
Image description
VALIDATE RESOURCES DEPLOYED IN PORTAL:-
Image description
IMPORTANT TO NOTE:-
Terraform Plan are NOT PUBLISHED in Azure DevOps GUI unless there is a change in Infrastructure - ADD, DESTROY or CHANGE
In order to demonstrate, the same pipeline was re-run.
Image description
Image description
Image description

Hope You Enjoyed the Session!!!

Stay Safe | Keep Learning | Spread Knowledge

Top comments (0)

Read next

rokicool profile image

Flex Consumption is not cheap (when in private VNET)

Roman Kiprin -

adzhydra profile image

# Azure Functions Memory Issues: How to Handle Large Files (Complete Guide 2024)

Adam -

vaish_98 profile image

Multicloud Cost Management Guide for FinOps Practitioners

Vaishnavi -

adzhydra profile image

Azure Functions (dotnet): The Right Way to Work with Queue Storage

Adam -