How do you prove that your infrastructure is compliant

When you are building your cloud infrastructure. And you have to meet certain compliance standards. You can achieve this with preventive and detective controls.

In this blog post I want to focus on the preventive control. How you can stop the deployment pipeline when the infrastructure is non-compliant. And how you can visualize this using CloudWatch Reports.
Let’s say that our company has the following rule:

S3 buckets must be KMS encrypted

How can you block a deployment that would deploy a S3 bucket without KMS encryption? The answer is using CloudFormation Guard.

With CloudFormation Guard you can write preventive compliance rules. You can then use to check your CloudFormation templates.
Create a template.yaml file with the following content:

Resources:
  MyBucket:
    Type: AWS::S3::Bucket

Install CloudFormation Guard using the provided instructions.

Then you need to create a s3.guard file with the following content:

# Select all resources of the type AWS::S3::Bucket
let buckets = Resources.*[ Type == 'AWS::S3::Bucket' ]

# Only when there are reources of the type AWS::S3::Bucket
rule S3_bucket_encryption_at_rest when %buckets !empty {
  # Check that each resource has the SSEAlgorithm set to aws:kms
  %buckets.Properties {
    BucketEncryption.ServerSideEncryptionConfiguration[*] {
      ServerSideEncryptionByDefault.SSEAlgorithm == 'aws:kms' <<S3 Bucket must use KMS encryption.>>
    }
  }
}

For more information on the the rules and the format. I recommend reading Introducing AWS CloudFormation Guard 2.0.

Alright, now we can validate the template with the following command:

cfn-guard validate --rules s3.guard --data template.yml

If you now change the template.yml to the following:

Resources:
  MyBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketEncryption:
        ServerSideEncryptionConfiguration:
          - ServerSideEncryptionByDefault:
              SSEAlgorithm: aws:kms

And you execute the command again it will now pass.

If you execute this step in your deployment pipeline. It will stop and fail your CodeBuild project. Preventing the non-compliant infrastructure to be deployed.

But when you have a deployment that is blocked. You don’t want to look up the CloudWatch logs of the build. And you might have many guard files that you want to check.
To make it easier for yourself you can use CodeBuild Reports. With these reports you get a visual overview of what rule passed and what role failed.

But by default cfn-guard does not generate compatible reports. And when you execute it, and has failures it will exit with an exit code of 1.

I have written a conversion tool to help you with this called report2junit.
It merges and coverts the JSON reports into the JUnit format used by CodeBuild Reports. By using the || true postfix the CodeBuild execution will continue.
Instead, the report2junit tool will return an exit code of 1 after it generated the report. And because the report is there when it fails you can use that to see what rule failed.

This removes the need to look at the CodeBuild Logs.

version: 0.2

phases:
  install:
    runtime-versions:
      python: 3.8
    commands:
      - pip install -Ur requirements.txt
      - mkdir -p reports
  build:
    commands:
      - cfn-guard validate --rules s3.guard --data template.yml --output-format json --show-summary none > reports/s3-guard.json || true
      - cfn-guard validate --rules dynamodb.guard --data template.yml --output-format json --show-summary none > reports/dynamodb-guard.json || true
  post_build:
    commands:
      - report2junit reports/s3-guard.json reports/dynamodb-guard.json --destination-file reports/cfn-guard.xml

reports:
  Tests:
    base-directory: ./reports
    file-format: JUNITXML
    files:
      - cfn-guard.xml

Conclusion

When you use CloudFormation Guard in combination with CodeBuild Reports it makes it easier to see what rules have failed and keeps a history.
When you have a solid set of compliance rules. It gives you a report that you can use to prove that the build of the infrastructure was compliant. You are also able to prevent non-compliant code rollout in production.