In this blog post, we’ll demonstrate how to leverage GitHub Actions (GHA) and Google Workload Identity Federation (WIF) to securely authenticate and create resources on Google Cloud Platform (GCP) using Terraform.
We’ll use two GitHub repositories:
Main Repo: Contains the Terraform code for provisioning GCP resources.
Workflow Repo: Hosts the GitHub Actions workflow, triggered by events in the Main Repo, to execute Terraform commands.
We will use two Google Cloud Projects.
Host Project as a landing zone to which GitHub Actions will authenticate using WIF.
Target Google Project where the required resources will be created.
Let’s dive into the setup!
Step 1: Setting Up Google Workload Identity Federation
1.1 - On your host project create a Google Cloud Service Account
Navigate to the Google Cloud Console.
Go to IAM & Admin > Service Accounts.
Create a new service account with the necessary roles for managing your GCP resources.
1.2 - Configure Workload Identity Federation
Go to IAM & Admin > Workload Identity Federation.
Create a Workload Identity Pool and a Provider linked to your GitHub repository.
Follow Google’s official WIF setup guide for detailed instructions. More in a separate Blog Post
Step 2: Permissions
2.1 - Ensure that the tragte gcp project has a terraform state bucket (ex: project_id-tfstate)
2.2 - Assign relevant roles(roles/editor &
roles/storage.admin) to the Service Account of the Host Project to perform relevant actions on the Target Google Project
gcloud projects add-iam-policy-binding gcp_project_name \
--member="serviceAccount:service-acct-name@host_gcp_project.iam.gserviceaccount.com" \
--role="roles/editor"
gcloud projects add-iam-policy-binding gcp_project_name \
--member="serviceAccount:service-acct-name@host_gcp_project.iam.gserviceaccount.com" \
--role="roles/storage.admin"
Step 3: Configure the Main Repo
This repository (e.g., org_name/google_cloud) will contain your Terraform code to manage GCP resources.
Example main.tf:
provider "google" {
project = var.project
region = var.region
}
resource "google_storage_bucket" "bucket" {
name = "${var.project}-bucket"
location = var.region
}
Step 4: Add a Workflow to the Main Repo
Create a GitHub Actions workflow (e.g., .github/workflows/dispatch.yml) that triggers events in the Workflow Repo. Example triggers include push, pull request, or tag creation.
Step 5: Configure the Workflow Repo
In the Workflow Repo, create a Terraform-specific GitHub Actions workflow(lets call this Terraform Workflow) (e.g., .github/workflows/terraform_plan_apply_gcp.yml) to perform Terraform operations such as plan and apply.
Example Trigger:
This workflow listens to repository_dispatch events:
repository_dispatch:
types: [terraform_plan, terraform_apply]
How the Main Repo Workflow Operates
Scenario 1: Push with a Tag
A developer pushes Terraform code to a feature branch and adds a tag (e.g., gcp_project_TFPLAN_01).
The dispatch.yml workflow triggers when a tag matching a specific pattern is pushed (e.g., '[a-z]+-[a-z]+TFPLAN[0-9]+').
The workflow determines the Terraform action (e.g., plan) and triggers the Workflow Repo's Terraform workflow via a GitHub API repository_dispatch event.
Scenario 2: Pull Request Creation
When a pull request is opened from a feature branch to develop, the workflow sends a repository_dispatch event with details such as:
event_type: terraform_plan
GCP project name & PR metadata (e.g., PR number, status=opened, merged=false).
Scenario 3: Pull Request Merge
When a pull request is merged, the workflow sends a repository_dispatch event with:
event_type: terraform_apply
GCP project name & PR metadata (e.g., PR number, status=closed, merged=true).
# This GitHub Actions workflow is designed to trigger a Terraform workflow based on specific events.
name: Trigger Terraform Workflow
on:
push:
tags:
- '[a-z]+-[a-z]+_PLAN_[0-9]+'
# branches:
# - 'feature/*'
pull_request:
types: [opened, synchronize, closed]
branches:
- develop
jobs:
trigger:
runs-on: ubuntu-latest
if: >
github.event_name == 'push' ||
(github.event_name == 'pull_request' &&
(github.event.action == 'opened' ||
github.event.action == 'synchronize' ||
(github.event.action == 'closed' && github.event.pull_request.merged == true)))
steps:
- name: Checkout repository
uses: actions/checkout@v2
# Debug step to print the GITHUB_REF
- name: Retrieve GitHub Data
if: github.event_name == 'push' || github.event_name == 'pull_request'
run: |
echo "GITHUB_REF=${GITHUB_REF}"
TAG_NAME=${GITHUB_REF#refs/tags/}
echo "TAG_NAME=$TAG_NAME" >> $GITHUB_ENV
- name: Print PR Information
if: github.event_name == 'pull_request'
run: |
# Get the latest TAG Name from the source branch
git fetch --tags
# Get latest tag
TAG_NAME=$(git tag --sort=-creatordate | head -n 1)
# Output the tag
if [ -z "$TAG_NAME" ]; then
echo "No tags found on the source branch: $SOURCE_BRANCH"
else
echo "The latest tag on the source branch ($SOURCE_BRANCH) is: $TAG_NAME"
echo "TAG_NAME=$TAG_NAME" >> $GITHUB_ENV
fi
echo "PR Number: ${{ github.event.pull_request.number }}"
echo "PR Action: ${{ github.event.action }}"
echo "PR Merged: ${{ github.event.pull_request.merged }}"
- name: Extract Information from Tag or PR
id: extract_info
run: |
GCP_PROJECT=$(echo $TAG_NAME | cut -d'_' -f1)
echo "GCP_PROJECT=$GCP_PROJECT" >> $GITHUB_ENV
echo "The Tag Name is: $TAG_NAME"
echo "The Target GCP Project is: $GCP_PROJECT"
if [[ "${{ github.event_name }}" == "push" ]]; then
ACTION="plan"
elif [[ "${{ github.event_name }}" == "pull_request" ]]; then
if [[ "${{ github.event.action }}" == "closed" && "${{ github.event.pull_request.merged }}" == "true" ]]; then
ACTION="apply"
else
ACTION="plan"
fi
fi
echo "ACTION=$ACTION" >> $GITHUB_ENV
echo "The Terraform Action is: $ACTION"
- name: Trigger Terraform Workflow for Push Commits
if: github.event_name == 'push'
run: |
echo "This was triggered as a result of the Event: ${{ github.event_name }} commits"
curl -X POST \
-H "Accept: application/vnd.github.v3+json" \
-H "Authorization: Bearer ${{ secrets.GH_DISPATCH_PAT }}" \
https://api.github.com/repos/${{ github.repository }}/dispatches \
-d "{\"event_type\":\"terraform_${{ env.ACTION }}\", \"client_payload\": {\"repository\": \"${{ github.repository }}\", \"project_name\": \"${{ env.GCP_PROJECT }}\", \"tag_name\": \"${{ env.TAG_NAME }}\"}}"
- name: Trigger Terraform Workflow for PR
if: github.event_name == 'pull_request'
run: |
echo "This was triggered as a result of PR Number: ${{ github.event.pull_request.number }} being ${{ github.event.action }}"
curl -X POST \
-H "Accept: application/vnd.github.v3+json" \
-H "Authorization: Bearer ${{ secrets.GH_DISPATCH_PAT }}" \
https://api.github.com/repos/${{ github.repository }}/dispatches \
-d "{\"event_type\":\"terraform_${{ env.ACTION }}\", \"client_payload\": {\"repository\": \"${{ github.repository }}\", \"pr_number\": \"${{ github.event.pull_request.number }}\", \"pr_event\": \"${{ github.event.action }}\", \"pr_merged\": \"${{ github.event.pull_request.merged }}\", \"project_name\": \"${{ env.GCP_PROJECT }}\", \"action\": \"${{ env.ACTION }}\"}}"
How the Terraform Workflow Operates
This workflow is hosted in the Workflow Repo and is triggered by repository dispatch events. Below, we break down each step with detailed explanations and corresponding code.
Step 1: Define Workflow Triggers
The workflow listens for certain event types:
repository_dispatch: Triggered by the Main Repo for Terraform plan and apply actions. More on repository dispatch can be found on the Official GitHub Documentation.
name: Terraform CI/CD
on:
workflow_dispatch:
repository_dispatch:
types: [terraform_plan, terraform_apply]
Step 2: Set Up the Terraform Job
Define a Terraform job that runs on ubuntu-latest with relevant permissions for GitHub Actions to interact with the repository and pull requests.
jobs:
terraform:
runs-on: ubuntu-latest
permissions:
contents: read
id-token: write
pull-requests: write
repository-projects: write
Step 3: Set Environment Variables
Capture the payload data from the triggering event(the main workflow) and set them as environment variables for subsequent steps.
- name: Set ENV variables
run: |
echo "TARGET_GCP_PROJECT=${{ github.event.client_payload.project_name }}" >> $GITHUB_ENV
echo "CLOUDSDK_CORE_PROJECT=${{ github.event.client_payload.project_name }}" >> $GITHUB_ENV
echo "GH_REPOSITORY=${{ github.event.client_payload.repository }}" >> $GITHUB_ENV
GH_REPO_NAME=$(echo "${{ github.event.client_payload.repository }}" | cut -d'/' -f2)
echo "GH_REPO_NAME=${GH_REPO_NAME}" >> $GITHUB_ENV
echo "GH_PR_NUMBER=${{ github.event.client_payload.pr_number }}" >> $GITHUB_ENV
echo "GH_PR_EVENT=${{ github.event.client_payload.pr_event }}" >> $GITHUB_ENV
echo "GH_PR_MERGED=${{ github.event.client_payload.pr_merged }}" >> $GITHUB_ENV
Step 4: Checkout the Code
Clone the Main Repo containing the Terraform code.
- name: Checkout code
uses: actions/checkout@v2
with:
repository: ${{ env.GH_REPOSITORY }}
token: ${{ secrets.GITHUB_TOKEN }}
Step 5: Set Up Terraform
Set up Terraform to run commands such as init, plan, and apply.
- name: Set up Terraform
uses: hashicorp/setup-terraform@v3
Step 6: Authenticate to Google Cloud
Use Workload Identity Federation to securely authenticate with Google Cloud.
- name: Authenticate to Google Cloud
id: authenticate
uses: google-github-actions/auth@v2
with:
create_credentials_file: true
workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }}
service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}
Step 7: Initialize Terraform
Set the GCP project and initialize Terraform with the remote backend configuration.
- name: Terraform Init
id: init
run: terraform init -backend-config="bucket=$TARGET_GCP_PROJECT-tfstate"
working-directory: ${{ env.TF_WORKING_DIR }}
Step 8: Validate Terraform Code
Ensure the Terraform configuration is correctly formatted and syntactically valid.
- name: Terraform Format
id: fmt
run: terraform fmt
working-directory: ${{ env.TF_WORKING_DIR }}
- name: Terraform Validate
id: validate
run: terraform validate
working-directory: ${{ env.TF_WORKING_DIR }}
Step 9: Generate Terraform Plan
Generate a plan and output the details for review.
- name: Terraform Plan
id: plan
run: terraform plan -var-file="$TARGET_GCP_PROJECT.tfvars" -out=tfplan
working-directory: ${{ env.TF_WORKING_DIR }}
- run: terraform show -no-color tfplan
id: show
working-directory: ${{ env.TF_WORKING_DIR }}
## We will use the output of terraform show to write the plan as a comment to the pull request
Step 10: Comment on Pull Requests
If triggered by a pull request, post the plan as a comment for review.
- name: PR Comment
uses: actions/github-script@v7
if: github.event.action == 'terraform_plan' && ( env.GH_PR_EVENT == 'opened' || env.GH_PR_EVENT == 'synchronize' )
env:
PLAN: "terraform\n${{ steps.show.outputs.stdout }}"
with:
github-token: ${{ secrets.GH_PAT }}
script: |
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: process.env.GH_REPO_NAME,
issue_number: process.env.GH_PR_NUMBER,
})
const botComment = comments.find(comment => comment.body.includes('Terraform Format and Style'))
const output = `#### Terraform Plan\n\`\`\`\n${process.env.PLAN}\n\`\`\``
if (botComment) {
github.rest.issues.updateComment({
owner: context.repo.owner,
repo: process.env.GH_REPO_NAME,
comment_id: botComment.id,
body: output
})
} else {
github.rest.issues.createComment({
owner: context.repo.owner,
repo: process.env.GH_REPO_NAME,
issue_number: process.env.GH_PR_NUMBER,
body: output
})
}
Step 11: Apply the Terraform Plan
If the event is terraform_apply, apply the plan to create resources.
- name: Terraform Apply
if: github.event.action == 'terraform_apply'
id: apply
run: terraform apply -auto-approve tfplan
working-directory: ${{ env.TF_WORKING_DIR }}
Conclusion
By integrating GitHub Actions and Google Workload Identity Federation, you can establish a secure, automated CI/CD pipeline for managing GCP resources using Terraform. This approach ensures that Terraform plans are reviewed, validated, and applied only after thorough approval, enhancing both security and operational efficiency.
Top comments (0)