AWS CloudFormation and Github Action

Overview

The focus of this article is to introduce the integration of AWS CloudFormation and Github Action.

Introduction

Github Action is a CI/CD service provided by Github and it has free tier. With Github Action, you can choose to run CI/CD actions on your self-hosted server or Github docker. It's user friendly and there are multiple plugins available in Github Action Marketplace which reduce your time of development and avoid RTW (Reinvent the Wheel).

AWS CloudFormation is a service that gives developers an easy way to automate the infrastructure provisioning using Infrastructure as Code (IaC). What's so special about Infrastrucure as Code is that we are able to run tests because these infrastructure components are written in code.

In this demo, we will be integrating AWS CloudFormation with Github Action and do the following:

• Any branch other than main:
  1. Linting: Check the CloudFormation linting format using cfn-lint
  2. Security check: Check the security of each components in CloudFormation using cfn-nag to validate if they are adhered to security best practices from AWS.
• Main branch (protected):
  1. Deploy: Deploy the CloudFormation template to S3.

Prerequisite

You may need the following to run on your projects:

1. AWS credentials with programmatic access, so you need your AWS Access Key, AWS Secret Key
2. AWS S3 bucket with bucket policy such that a put action is allowed and ACL is allowed, you need the AWS S3 bucket name and make sure it is globally unique.
3. Github account

Project Structure

.
├── .github
│   └── workflows
│       ├── master.yml
│       └── test.yml
└── cloudformation
    ├── conformance-pack.yaml
    └── example.yaml

For this project demo, I write two CloudFormation templates:

1. conformance-pack.yaml to deploy a set of config rules and remediation actions.
2. example.yaml to deploy a simple S3 bucket with SNS trigger

Nevertheless, the main focus is on the Github Actions. Let's take a look on what does the Github Action do.

Github Actions

As you can see from the project structure, there are mainly two templates, master.yaml and test.yaml under the .github/workflow directory. GitHub Actions uses YAML syntax to define the events, jobs, and steps and these files are stored in a directory called .github/workflows. In order for you to run the Github Actions, you need to create this directory so that whenever you push a commit into your repository, Github Actions will run automatically.

In this directory, create two files, master.yaml and test.yaml.

# test.yaml

name: 'Testing Github Actions'


on: [push]

jobs:
  Cloudformation-checker:
    name: Check linting and security concerns
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: cfn-lint-action
        uses: ScottBrenner/cfn-lint-action@1.6.1
        with:
          args: "cloudformation/**/*.yaml"

      - name: cfn-nag-action
        uses: minchao/cfn-nag-action@v0.1
        with:
          args: '--input-path cloudformation/

This Github Action will be triggered if a commit is pushed into this repository for all the branches. It will validate the template syntax using cfn-lint-action and run security check using cfn-nag-action if the commit passed the cfn-lint-action. What's good abou this Github Action is that all of these steps are ready-made plugin from Github Action Marketplace and they have been created and used by the Github communities. For this project, I'm using the plugins from ScottBrenner/cfn-lint-action@1.6.1 for cfn-lint-action and
minchao/cfn-nag-action@v0.1 for cfn-nag-action. There are several similar plugins in the Marketplace, please makes sure you read their README.md in order to enable the actions.

name: S3Deploy

on: 
  push:
    branches:
      - main
  pull_request:
    branches:
      - main

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - name: "Checkout"
        uses: actions/checkout@v2
      - name: S3 Sync
        uses: jakejarvis/s3-sync-action@v0.5.1
        with:
          args: --acl bucket-owner-read --follow-symlinks
        env:
          AWS_ACCESS_KEY_ID: ${{secrets.AWS_ACCESS_KEY}}
          AWS_SECRET_ACCESS_KEY: ${{secrets.AWS_SECRET_KEY}}
          AWS_S3_BUCKET: ${{secrets.AWS_BUCKET}}
          SOURCE_DIR: './cloudformation'

Lastly, this Github Action will copy the files in the cloudformation directory to S3 bucket if a commit or a pull request is made into the main branch. As you can see from the env section, there are several secrets, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_S3_BUCKET. Please refer to this documentation site to learn how to input secrets into your Github Actions.

What's good about this is that you don't have to manually push the templates into S3 bucket by running multiple command line or browsing into the AWS management console.

Conclusion

Throughout this simple project, you can learn how to run a simple CI/CD pipeline to run simple template syntax check and security check on the CloudFormation templates and then push them into a dedicated S3 bucket. Of course, there are some improvements, for example, auto provision the infrastructures into different environments after a file is pushed to S3 bucket. Feel free to extend from here and hope to see more on CI/CD for Infrastructure as Code.

Reference

• Github Action documentaions
• Github Action secrets
• Github Action Marketplace
• CloudFormation