How to use Terraform in AWS multi-account via AzureDevOps pipelines

In this session, we will see how to provision infrastructure using Terraform in AWS cloud multi-accounts, cross-account IAM roles, and AzureDevOps pipelines to automate the deployment.

Let's break down the topics below

1. Architecture diagram
2. AWS multi-account creation setup
3. Terraform infrastructure code
4. AzureDevOps pipelines as code to automate deployment

Architecture diagram

From the above architecture diagram, let's list down the workflow.

Workflow 1

We need to create AWS multi accounts, basically 3 accounts based on this example. The below steps need to be completed manually by creating via AWS console or AWS CLI commands.

- Management/DevOps account

1. In this account we will manage the IAM user(automation user for ex: vscode) and we will generate the secret key and access id for this user which will be used by Terraform

2. Assign an Administrator role or least privilege role by attaching only STS policy, so that the user 'vscode' can assume a role in trusting account like Development or Production account

- Development Account

1. Create a role for ex: ManageInDevAsAdmin with Administrator permission or least privilege that is required to manage this account and mention trusted account ARN number of Management/DevOps account

- Production Account

1. Create a role for ex: ManageInProdAsAdmin with Administrator permission or least privilege that is required to manage this account and mention trusted account ARN number of Management/DevOps account

Once these above steps are completed, Our terraform automation able to use those credentials for vscode and assume the IAM role to provision the resources in those different AWS accounts.

Workflow 2 and 3
Azure DevOps tool has a complete solution to manage source code repository, pipelines to automation deployment, and much more. In this use case, we use both repository and pipelines.

Azure DevOps Repository to keep the source code for Terraform infrastructure provisioning and Azure pipelines to execute the terraform commands.

Pre-requisites:

1. We need to upload a secure file in the Azure DevOps portal which will be used in pipelines for the Terraform to use the secret key and access id in the profile section.

Let's use azure-pipelines.yml pipelines as a code to automate the deployment using Terraform. The code has stages for Dev and Prod and it contains terraform commands.

The credentials file looks like below.

Secure file upload in AzureDevOps portal

azure-pipelines.yml file content.

# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml

trigger:
- main

pool:
  vmImage: ubuntu-latest

stages:    
  - stage: Dev
    displayName: Dev
    jobs:
      - job: plan
        pool:
          vmImage: ubuntu-latest
        displayName:  Terraform Dev Init and Apply
        steps:
        - task: DownloadSecureFile@1
          name: aws_credentials
          displayName: 'Download AWS Credentials'
          inputs:
            secureFile: 'credentials'
        - task: PowerShell@2
          displayName: 'New directory to keep the aws credentials'
          inputs:
            targetType: 'inline'
            script: |
              New-item -Path "~/" -Name ".aws" -ItemType "directory"
        - task: CopyFiles@2
          displayName: 'Copy aws credentials to home directory '
          inputs:
            SourceFolder: "$(Agent.TempDirectory)"
            contents: 'credentials'
            targetFolder: '~/.aws/'
        - task: PowerShell@2
          inputs:
            targetType: 'inline'
            script: 'terraform init -reconfigure -backend-config="dev-backend.conf"'
            workingDirectory: '$(Build.Repository.LocalPath)/saas_app'
          displayName: Terraform init for plan
        - task: PowerShell@2
          inputs:
            targetType: 'inline'
            script: 'terraform plan --var-file="dev-app.tfvars" -var createdBy="dev-pipeline" -var tags="asd" -out main.tfplan'
            workingDirectory: '$(Build.Repository.LocalPath)/saas_app'
          displayName: "Terraform Plan"
      - job: approve
        dependsOn: plan
        condition: and (succeeded(), ne(variables['Build.Reason'], 'PullRequest'))
        pool: server
        displayName: Wait for approval
        steps:
        - task: ManualValidation@0
          timeoutInMinutes: 15 # task times out in 15 mins
          inputs:
            notifyUsers: [e-mail]
            instructions: 'Please validate the build configuration and resume'
            onTimeout: 'reject'
      - job: apply
        dependsOn: approve
        pool:
          vmImage: ubuntu-latest
        displayName:  Terraform Dev Init and Apply
        steps:
        - task: DownloadSecureFile@1
          name: aws_credentials
          displayName: 'Download AWS Credentials'
          inputs:
            secureFile: 'credentials'
        - task: PowerShell@2
          displayName: 'New directory to keep the aws credentials'
          inputs:
            targetType: 'inline'
            script: |
              New-item -Path "~/" -Name ".aws" -ItemType "directory"
        - task: CopyFiles@2
          displayName: 'Copy aws credentials to home directory '
          inputs:
            SourceFolder: "$(Agent.TempDirectory)"
            contents: 'credentials'
            targetFolder: '~/.aws/'
        - task: PowerShell@2
          inputs:
            targetType: 'inline'
            script: 'terraform init -reconfigure -backend-config="dev-backend.conf"'
            workingDirectory: '$(Build.Repository.LocalPath)/saas_app'
          displayName: Terraform init for apply
        - task: PowerShell@2
          inputs:
            targetType: 'inline'
            script: 'terraform apply --var-file="dev-app.tfvars" -var createdBy="dev-pipeline" -var tags="asd" --auto-approve'
            workingDirectory: '$(Build.Repository.LocalPath)/saas_app'
          displayName: "Terraform Apply"
  - stage: Prod
    displayName: Prod
    jobs:
      - job: plan
        pool:
          vmImage: ubuntu-latest
        displayName: Terraform Prod Init and Apply
        steps:
        - task: DownloadSecureFile@1
          name: aws_credentials
          displayName: 'Download AWS Credentials'
          inputs:
            secureFile: 'credentials'
        - task: PowerShell@2
          displayName: 'New directory to keep the aws credentials'
          inputs:
            targetType: 'inline'
            script: |
              New-item -Path "~/" -Name ".aws" -ItemType "directory"
        - task: CopyFiles@2
          displayName: 'Copy aws credentials to home directory '
          inputs:
            SourceFolder: "$(Agent.TempDirectory)"
            contents: 'credentials'
            targetFolder: '~/.aws/'
        - task: PowerShell@2
          inputs:
            targetType: 'inline'
            script: 'terraform init -reconfigure -backend-config="prod-backend.conf" -no-color'
            workingDirectory: '$(Build.Repository.LocalPath)/saas_app'
          displayName: Terraform init for plan
        - task: PowerShell@2
          inputs:
            targetType: 'inline'
            script: 'terraform plan --var-file="prod-app.tfvars" -var createdBy="prod-pipeline" -var tags="asd"  -out main_prod.tfplan -no-color '
            workingDirectory: '$(Build.Repository.LocalPath)/saas_app'
          displayName: "Terraform Plan"
      - job: approve
        dependsOn: plan
        condition: and (succeeded(), ne(variables['Build.Reason'], 'PullRequest'))
        pool: server
        displayName: Wait for approval
        steps:
        - task: ManualValidation@0
          timeoutInMinutes: 15 # task times out in 15 mins
          inputs:
            notifyUsers: [e-mail]
            instructions: 'Please validate the build configuration and resume'
            onTimeout: 'reject'
      - job: apply
        dependsOn: approve
        pool:
          vmImage: ubuntu-latest
        displayName: Terraform Prod Apply
        steps:
        - task: DownloadSecureFile@1
          name: aws_credentials
          displayName: 'Download AWS Credentials'
          inputs:
            secureFile: 'credentials'
        - task: PowerShell@2
          displayName: 'New directory to keep the aws credentials'
          inputs:
            targetType: 'inline'
            script: |
              New-item -Path "~/" -Name ".aws" -ItemType "directory"
        - task: CopyFiles@2
          displayName: 'Copy aws credentials to home directory '
          inputs:
            SourceFolder: "$(Agent.TempDirectory)"
            contents: 'credentials'
            targetFolder: '~/.aws/'
        - task: PowerShell@2
          inputs:
            targetType: 'inline'
            script: 'terraform init -reconfigure -backend-config="prod-backend.conf" -no-color'
            workingDirectory: '$(Build.Repository.LocalPath)/saas_app'
          displayName: Terraform init for apply
        - task: PowerShell@2
          inputs:
            targetType: 'inline'
            script: 'terraform apply --var-file="prod-app.tfvars" -var createdBy="prod-pipeline" -var tags="asd" --auto-approve -no-color'
            workingDirectory: '$(Build.Repository.LocalPath)/saas_app'
          displayName: "Terraform Apply"

Workflow 4 and 5
Let's walk through the stages in the pipeline.

Dev stage job tasks

1. Download a secure file

This task will download the secure file into the agent's temporary path

1. Copy secure file

This task will copy the file secure file from the temporary path to the agent home directory because we referred to this path in terraform provider section.

terraform providers.tf file content

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
    }
  }
  backend "s3" {}
}

provider "aws" {
  region                   = var.aws_region
  shared_credentials_files = ["~/.aws/credentials"]
  profile                  = "vscode"
  assume_role {
    role_arn = var.aws_assume_role_arn 
  }
}

3. Terraform init

For terraform initialization, we will use the S3 bucket as the backend state.

Pre-requisites:

• Create an S3 bucket for Development and Production AWS account to maintain the terraform state for those accounts

• Create dynamodb for Development and Production AWS account to maintain the terraform lock for those accounts

• Role name should be present in backend configuration

dev-backend.conf file content

bucket         = "example-tf-dev-states"
key            = "example-dev-state/terraform.tfstate"
region         = "ap-southeast-2"
dynamodb_table = "example-tf-dev-locks"
encrypt        = true
profile        = "vscode"
role_arn = "arn:aws:iam::<AWS Development env account number >:role/ManagementInDEVAdminRole"

prod-backend.conf file content

bucket         = "example-tf-prod-states"
key            = "example-prod-state/terraform.tfstate"
region         = "ap-southeast-2"
dynamodb_table = "example-tf-prod-locks"
encrypt        = true
profile        = "vscode"
role_arn = "arn:aws:iam::<AWS Development env account number >:role/ManagementInProdAdminRole"

Terraform init command:
terraform init -reconfigure -backend-config="dev-backend.conf"

Here we are using -reconfigure with the backend config, this will correctly set the terraform state
s3 bucket for the corresponding AWS account based on the environment configuration file.

4. Terraform plan
Terraform plan is used to compare the state of the infrastructure and the current desired state from the code changes, It presents the changes that are going to happen in our infrastructure.

Terraform plan command:
terraform plan --var-file="dev-app.tfvars" -var createdBy="dev-pipeline" -var tags="asd" -out main.tfplan

Here we need to pass Development environment-specific values

dev-app.tfvars file content

# Common
project = "example-project"

# General 
aws_region  = "ap-southeast-2"
profile     = "vscode"
environment = "dev"
aws_assume_role_arn = "arn:aws:iam::<Development AWS account number>:role/ManagementInDEVAdminRole"

# Terraform Backend State
backend_s3_bucket      = "example-tf-dev-states"
backend_dynamodb_table = "example-tf-dev-locks"
backend_s3_key = "example-dev-state/terraform.tfstate"

# VPC
create_vpc           = true
cidr                 = "10.123.0.0/16"
instance_tenancy     = "default"
enable_dns_hostnames = true
enable_dns_support   = true
create_igw           = true

# Subnet
create_subnet         = true
subnet_cidr_block = "10.123.1.0/24"
subnet_map_public_ip_on_launch = true
subnet_availability_zone = "ap-southeast-2a"

Once the plan is completed, we can see the result.

4. Azure DevOps Terraform plan review and approve or reject
This approval gate helps us to review the terraform plan and understand the changes we are going to make in our infrastructure.

5. Terraform apply
In Azure DevOps pipeline, we need to define the download, copy a secure file, and terraform init task because we are using a new job definition.

Terraform apply command
terraform apply --var-file="dev-app.tfvars" -var createdBy="dev-pipeline" -var tags="asd" --auto-approve

Here we need to pass auto-approve because we already reviewed the terraform plan.

Once the terraform apply is completed, we can see the AzureDevOps pipeline status.

NOTE: For the Prod stage as well, we will be using the same concept and we need to pass different environment variable files.

Comments

[Anurag Vishwakarma]

Hi, Guys currently I am pursuing my b.tech in CSE(last year). In starting of my first year I have so many ideas and an interest in coding but now I don't like coding. I don't know what should I do. Now, I started figuring out NO CODE TOOLS & PLATFORMS like Webflow & WordPress. I left learning coding after 2-3 months. Now Placement in college has started. Some of my friends got 7LPA Package. I don't know and I am confused. Have financial issues too, my dad not living with us and the source of income is my mom's work in a garment factory. I have also applied for an education loan for my 3rd/4th Year. Every night I sleep with a lot of overthinking & thoughts, depression, and anxiety in my mind. Please help me to find a path.
@sathish86