DEV Community: Rishav Dhar

Enhance Terraform/Tofu Automation with GitHub Action

Rishav Dhar — Mon, 21 Oct 2024 21:12:13 +0000

Provisioning infrastructure-as-code (IaC) in a GitOps framework can feel like walking a tightrope: balancing pipeline security while trying to communicate changes clearly. This blog explores a GitHub Action I maintain—DevSecTop/TF-via-PR—which addresses common pitfalls to plan and apply IaC, including:

Summarize plan changes (with diff)
Reuse plan file (with encryption)
Apply on PR merge (before OR after)

Tip
Throughout this blog, 'TF' is used to reference both Terraform and OpenTofu interchangeably due to first-class support for both tools.

Summarize plan changes (with diff)

While the plan should be transparent, reviewing 1000s of lines of planned changes is simply not feasible. On the other hand, a brief 1-liner like "Plan: 2 to add, 2 to change, 2 to destroy" fails to convey the scope of impact. So why not visualize the summary of changes the same way Git does—with diff syntax highlighting.

Figure: PR comment of TF plan with "Diff of changes" section expanded.

Right after the color-coded diff, there’s another collapsible section with the trimmed plan output for a more detailed view. Below that, we’ll find a direct link to the full workflow log—handy for when the output exceeds a PR comment’s character limit. Speaking of workflows, the same output is attached to the job summary, even when used in matrix strategy.

Figure: Workflow log with command-specific job summary.

This is achieved with the following few lines of GitHub Action.

jobs:
  provision:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout repository
        uses: actions/checkout@4

      - name: Setup TF
        uses: hashicorp/setup-terraform@v3

      - name: Provision TF
        uses: devsectop/tf-via-pr@v12
        with:
          command: plan
          arg-lock: false
          arg-var-file: env/dev.tfvars
          working-directory: stacks

By the way, you may have noticed the oddly-named "terraform[...]tfplan" artifact in the previous screenshot and wondered what it's all about.

Reuse plan file (with encryption)

Too often, when a plan is approved for merge, the IaC pipeline is rerun to apply changes with auto-approve enabled. This can lead to unpredictable results, especially if configuration drift occurs due to changes made outside the workflow. Since both the code and pipeline are hosted on GitHub, we can take advantage of workflow artifacts to store and reuse the plan file between runs.

To ensure the triggered workflow picks up the correct plan file artifact, it needs a uniquely identifiable name which accounts for variables, such as:

Tool: either terraform or tofu.
PR number: so multiple PR branches can plan simultaneously without over-writing each other.
CLI arguments: workspace, working directory, backend-config, var-file, and destroy.

Additionally, we want to avoid the risk of exposing sensitive data by uploading the plan file as-is. Instead, the file should be encrypted with a secret string before upload. Here's how we can add this to the GitHub Action workflow step from before.

- name: Provision TF
  uses: devsectop/tf-via-pr@v12
  with:
    command: plan
    arg-lock: ${{ github.event_name == 'push' }}
    arg-var-file: env/dev.tfvars
    working-directory: stacks
    plan-encrypt: ${{ secrets.PASSPHRASE }}

Tip
For a deeper dive into securing cloud provisioning pipelines, check this blog out.

Secure cloud provisioning pipeline with GitHub automation

Rishav Dhar ・ Oct 20 '24

#infrastructureascode #terraform #githubactions #devops

Speaking of reuse, it’s common for a PR to accumulate several commits before it’s ready to merge. In this case, plan updates can be rendered in one of two ways.

Update (default): the existing PR comment is updated in place, complete with a revision history to track changes over time.
Recreate: the existing PR comment is deleted and replaced with a new one after each commit.

Figure: PR comment revision history comparing plan and apply outputs.

By now, you’ll have noticed that we’re applying changes with the same GitHub Action—the final piece of the ~~puzzle~~ pipeline.

Apply on PR merge (before OR after)

Whether you decide to apply IaC changes before or after merging, the workflow adapts to fit your needs. By using unique identifiers, we can retrieve the relevant plan file even if the PR branch has been pushed to the default branch, outside of 'pull_request' context. Here’s a complete workflow example to illustrate.

on:
  pull_request:
  push:
    branches: [main]

jobs:
  provision:
    runs-on: ubuntu-latest

    permissions:
      actions: read        # Required to identify workflow run.
      checks: write        # Required to add status summary.
      contents: read       # Required to checkout repository.
      pull-requests: write # Required to add comment and label.

    steps:
      - name: Checkout repository
        uses: actions/checkout@4

      - name: Setup TF
        uses: hashicorp/setup-terraform@v3

      # Only plan by default, or apply with lock on merge.
      - name: Provision TF
        uses: devsectop/tf-via-pr@v12
        with:
          command: ${{ github.event_name == 'push' && 'apply' || 'plan' }}
          arg-lock: ${{ github.event_name == 'push' }}
          arg-var-file: env/dev.tfvars
          working-directory: stacks
          plan-encrypt: ${{ secrets.PASSPHRASE }}

We're not limited to just these workflow triggers; it's compatible with 'merge_group' using a merge queue to filter out failed apply attempts. Other triggers, like 'cron' (for scheduled drift checks) and 'workflow_dispatch' (for manual checks), are also supported, with their plan file shared between workflows.

Bonus Extras

While this blog has primarily focused on planning and applying changes, we can also perform 'fmt' and 'validate' checks, along with 'workspace' selection. In fact, we can pass the full range of TF options and flags using the 'arg-' prefix, such as 'arg-auto-approve: true', 'arg-destroy: true', 'arg-workspace: dev', and 'arg-parallelism: 20'.

For more complex workflows, detailed 'exitcode' and 'identifier' outputs are provided to help with decision chains or to integrate with linting and security scans. You can find the full list of parameters documented in the Readme.

OP5dev / TF-via-PR

Plan and apply Terraform/OpenTofu via PR automation, using best practices for secure and scalable IaC workflows.

* *

Terraform/OpenTofu via Pull Request (TF-via-PR)

What does it do?	Who is it for?
Plan and apply changes with CLI arguments and encrypted plan file to avoid configuration drift. Outline diff within up-to-date PR comment and matrix-friendly workflow summary, complete with log.	DevOps and Platform engineers wanting to empower their teams to self-service scalably. Maintainers looking to secure their pipeline without the overhead of containers or VMs.

View: Usage Examples · Inputs · Outputs · Security · Changelog · License

Usage Examples

How to get started?

on
  pull_request:
  push:
    branches: [main]

jobs:
  provision:
    runs-on: ubuntu-latest

    permissions:
      actions: read        # Required to identify workflow run.
      checks: write        # Required to add status summary.
      contents: read       # Required to checkout repository.
      pull-requests: write # Required to add PR comment.

    steps:
      - uses: actions/checkout@v4

      - uses: hashicorp/setup-terraform@v3

…

View on GitHub

All forms of contribution are welcome and appreciated for fostering open-source projects. Please feel free to open a discussion to share your ideas, or become a stargazer if you find this project useful.

Whether it's interpolating dynamic backends, bulk-provisioning environments simultaneously, or triggering actions through labels and comments, this workflow offers plenty of flexibility. Is there a specific setup you'd like us to dive into next?

Secure cloud provisioning pipeline with GitHub automation

Rishav Dhar — Sun, 20 Oct 2024 00:54:11 +0000

As a member of the Platforms engineering team, we understand that security is a shared responsibility throughout the DevSecOps lifecycle for provisioning infrastructure. As a result, we set about championing best practices across the organization, with a focus on:

Configuring short-lived credentials
Automating cloud-provisioning pipelines
Comparing infrastructure-as-code tooling
Securing deployments from code-to-delivery

Figure: How to provision infrastructure-as-code.

Short-lived credentials

GitHub Actions form the basis of our continuous integration/continuous deployment (CI/CD) pipeline as it integrates seamlessly with GitOps: the framework by which we ship peer-reviewed code early and often. It enables us to extend our workflow with Actions from verified creators, such as aws-actions (other cloud computing platforms are available).

This streamlines the process of configuring AWS credentials via OpenID Connect (OIDC): our favorite “keyless” authentication method. OIDC only permits access from federated providers using short-lived credentials, which pairs perfectly with GitHub Actions’ ephemeral runners. What’s more, we can grant least-privilege provisioning permissions to the assumed-role (and include both thumbprints). Let’s add it to our workflow.

Tip
While the following pseudo-code is for legibility, we will build up to the complete workflow by the end of this post.

jobs:
  provision:
    runs-on: ubuntu

    steps:
      - name: Authenticate AWS
        uses: aws-actions/configure-aws-credentials

Figure: Security hardening GitHub Action workflow with OpenID Connect (source).

Automated provisioning pipeline

With authentication sorted and access to infrastructure secured, we turned to Test Double for inspiration on a “roll-your-own” deployment workflow.

It goes on to link OP5dev/TF-via-PR: another open-source project which combines the outlined concepts into a reusable Action. In support of trunk-based Pull Request (PR) automation, this workflow:

Runs init > workspace > plan with optional arguments when a PR is opened, returning the plan output in a PR comment and storing the encrypted plan file as an artifact.
Reruns plan when the PR is updated with new commits, returning the latest output and replacing the old PR comment to reduce noise.
Runs init > workspace > apply with the previous plan file artifact, along with optional arguments when the PR is approved and merged.

Tip
Passing in the previously-generated plan file when applying prevents any configuration drift from stale plans.

By default, each PR and associated workflow runs hold a complete log of infrastructure change/proposals for ease of collaboration and audit compliance within a set retention period. The use of GitHub Actions also removes the overhead of maintaining dedicated containers or self-hosted compute instances: lending itself to empower development teams to self-service scalably. Time to add another step to our workflow.

on:
  pull_request:

jobs:
  provision:
    runs-on: ubuntu

    steps:
      - name: Authenticate AWS
        uses: aws-actions/configure-aws-credentials

      - name: Checkout repository
        uses: actions/checkout

      - name: Provision infrastructure
        uses: op5dev/tf-via-pr

Figure: Flow chart of provisioning pipeline steps.

Infrastructure-as-code tooling

Although plan file encryption is one of the options provided by the Action, it’s also a feature native to OpenTofu: a project backed by The Linux Foundation as an open-source alternative to Terraform.

More recently, OpenTofu released support for early static evaluation of variables/locals, enabling the likes of module versions and backend inputs to be loosely coupled with don’t-repeat-yourself (DRY) configuration. While a number of teams have successfully experimented by replacing terraform with tofu, many others are holding out for dynamically configured provider support as well as Dependabot automation before making the switch.

Tip
For anyone else experimenting between the two, whole-heartedly recommend tofuutils/tenv as the go-to package manager of choice with broad compatibility.

Secured deployments workflow

With the GitHub Action dependencies accounted for, all that’s left is to bring them together in a workflow that is reactive to the provisioning result. In other words, if apply fails for any reason, then reject the PR from merge and return the error output in a PR comment for follow-up. This happens to be the ideal use-case for a merge queue of size and concurrency one.

Now that we have all the pieces to form the complete workflow below, can you spot what else has been done to shore up security? Answers below!

on:
  pull_request:
    branches: [main]
  merge_group:
    types: [checks_requested]

jobs:
  provision:
    runs-on: ubuntu-24.04

    permissions:
      actions: read        # Required to download artifact.
      checks: write        # Required to add status summary.
      contents: read       # Required to checkout repository.
      id-token: write      # Required to authenticate via OIDC.
      pull-requests: write # Required to add PR comment and label.

    env:
      AWS_ACCOUNT: platforms-dev
      AWS_REGION: us-east-1
      AWS_ROLE: arn:aws:iam::123456789012:role/provision-cicd

    environment: ${{ github.event_name == 'merge_group' && env.AWS_ACCOUNT || '' }}

    steps:
      - name: Authenticate AWS
        uses: aws-actions/configure-aws-credentials@e3dd6a429d7300a6a4c196c26e071d42e0343502 # v4.0.2
        with:
          aws-region: ${{ env.AWS_REGION }}
          role-to-assume: ${{ env.AWS_ROLE }}

     - name: Checkout repository
       uses: actions/checkout@d632683dd7b4114ad314bca15554477dd762a938 # v4.2.0
        with:
          persist-credentials: false

      - name: Setup TF
        uses: hashicorp/setup-terraform@b9cd54a3c349d3f38e8881555d616ced269862dd # v3.1.2
        with:
          terraform_version: "1.8.5"

      - name: Provision infrastructure
        uses: op5dev/tf-via-pr@f455e10e6e83d28f53f505a2d8242433a17536f3 # v13.0.0
        with:
          command: ${{ github.event_name == 'merge_group' && 'apply' || 'plan' }}
          arg-lock: ${{ github.event_name == 'merge_group' }}
          working-directory: path/to/${{ env.AWS_ACCOUNT }}
          plan-encrypt: ${{ secrets.TF_ENCRYPTION }}

Make Actions immutable by pinning to a commit SHA to reduce third-party supply-chain risks (and automate updates with Dependabot).
Use environment: to enforce deployment protection rules when attempting to apply changes to infrastructure.
Use persist-credentials: false with checkout to prevent leaking the GitHub token to subsequent steps of the workflow.

Tip
For a deeper dive into TF-via-PR GitHub Action, check this blog out.

Enhance Terraform/Tofu Automation with GitHub Action

Rishav Dhar ・ Oct 21 '24

#devops #terraform #infrastructureascode #githubactions

Next steps

There’s plenty of potential with this workflow: from interpolating workspaces with dynamic backends to bulk-provisioning multiple accounts in matrix strategy concurrently. Is there any particular setup or tooling you’d like us to explore next?

DEV Community: Rishav Dhar

Enhance Terraform/Tofu Automation with GitHub Action

Summarize plan changes (with diff)

Reuse plan file (with encryption)

Secure cloud provisioning pipeline with GitHub automation

Rishav Dhar ・ Oct 20 '24

Apply on PR merge (before OR after)

Bonus Extras

OP5dev / TF-via-PR

Plan and apply Terraform/OpenTofu via PR automation, using best practices for secure and scalable IaC workflows.

Terraform/OpenTofu via Pull Request (TF-via-PR)

What does it do?

Who is it for?

View: Usage Examples · Inputs · Outputs · Security · Changelog · License

Usage Examples

How to get started?

Secure cloud provisioning pipeline with GitHub automation

Short-lived credentials

Automated provisioning pipeline

Infrastructure-as-code tooling

Secured deployments workflow

Enhance Terraform/Tofu Automation with GitHub Action

Rishav Dhar ・ Oct 21 '24

Next steps