Achieving organisation-scoped AWS Config compliance using Cloudformation Lambda Hooks

#aws #cloudformation #devops

Overview

AWS Cloudformation Hooks is an existing AWS feature that helps ensure compliance of AWS resources when being created or updated in accounts as Infrastructure as Code (IaC) via Cloudformation or CDK , against an organization’s defined standards. Examples of such checks can include ensuring that RDS, S3 or other data storage resources are configured with at-rest encryption, that log groups have retention policies, EC2 instances are not publicly-exposed etc. These rules typically strongly-align with and promote AWS Well-Architected best practices.

AWS first introduced the ability to create custom Cloudformation hooks in 2022, using Java or Python code to allow organizations to also define their own rules. Implementation required the following steps:
i. Initialising an AWS Cloudformation Hooks project using the Cloudformation CLI.
ii. Implementing the hook’s handler logic to evaluate compliance of resources included within Cloudformation stacks.
iii. Packaging and registering the hook in Cloudformation.

The creation of these hook projects unfortunately are often not simple, where much boilerplate code is often required, and packaging and registering of hooks also requires further effort.

AWS however have recently (since November 2024) made available 2 new hook types which make these configurations easier, being:

Guard Hooks: These allow the specification of AWS Cloudformation Guard rules to set requirements for resources within Cloudformation templates. Pros: Uses an open-source Domain Specific Language (DSL) to define rules for simplicity, rather than needing to compile and package code to implement. They have no associated execution charges. Cons: Its own DSL may lack flexibility of logic, and require a learning curve in-contrast to a more-general language where in-house skills may already exist
Lambda Hooks: Custom logic for interrogating resources within a Cloudformation changeset or stack can also be implemented as lambda functions. Lambda hooks may then simply be configured to reference the lambda implementations. Pros: Provide further flexibility for implementing compliance logic in contrast to Guard Hooks, while also being simpler to implement and configure than Custom Hooks. Rule behaviour can be easily amended within the lambda function, without requiring reconfiguration of the hook in Cloudformation. Hook rules may also be implemented in any language that lambda supports. Cons: AWS does not charge for the hook, but standard lambda execution changes do apply.

While considering how Cloudformation hooks may assist with ensuring standards compliance with the previously-introduced “The Better Store” sample project, I have chosen to explore lambda Cloudformation hooks further here, and how they may be integrated with AWS Organizations/Stack Sets to automatically validate Cloudformation templates for all AWS accounts/regions within defined Organization Units (OU’s). This setup and its results are described next.

Solution Prerequisites
The following are required for implementing the organisation-level hooks for the example:

AWS Organizations has been configured, with the following being set on the Organizations Master Account:
- Within AWS Organizations/Services section of the AWS web UI, the service “AWS Account Management” is enabled.
- A separate ‘Tools’ account has been provisioned within the Organization, which will be used for performing organization-scoped Cloudformation deployments via StackSets.
- The “Tools” account has been configured as a ‘Delegated Administrator’ for Cloudformation stacksets within the AWS Organizations master account (via the Cloudformation Stacksets web UI).
- Target accounts which are to be automatically configured with Cloudformation hooks are provisioned within target Organization Units in AWS Organizations (e.g. ‘non-prod’ for all non-production workload accounts).
A deployment bucket is created within the tools account, to store built lambda artefacts and Cloudformation templates which deploy them. A bucket policy is defined which provides GetObject access to its objects, to all accounts defined within the AWS Organization.
A deployment user is configured in the “Tools” account, which has the following permissions:
- Write access to the above S3 deployment bucket.
- Cloudformation permissions
- AWS Organizations query permissions A user access key is also securely created and used for this to perform deployments, and an AWS user profile ‘thebetterstore-tools’ is configured to use these. NB it is expected a productionized process would perform deployments within a DevOps pipeline, and that IAM roles may instead be used, which are preferred over user access keys.
The AWS CLI is installed (examples assume we are using a Linux environment, though Windows can also be used with minor tweaks). N5. odeJS v20+ (the example will be implemented in Javascript, to target the NodeJS 20.x runtime)

Implementation

The following illustrations describe my target deployment architecture for lambda Cloudformation hooks in an organization, and the composition of components used:

Figure 1. High-level Lambda Hook Deployment via AWS Organizations / StackSets. Deployment of organization-specific Cloudformation StackSets has been delegated to a separate Tools account, which is also responsible for building the lambda hooks project and publishing these to the deployment bucket. A Hooks stackset automatically-deploys changes to the lambda hooks stack (tbs-devops-cfnhooks) across existing and/or new AWS accounts within specified target OU’s.

Figure 2: Illustrating the composition of deployed CfnHooks stacks, hook configuration scope, and the relationship of the hooks to target resources

where main components are implemented with the following solution scaffolding:

bin
— deploy-setup.sh
— deploy-stackset.sh
deploy-setup/
— template-setup.yaml
— template-stackset.yaml
tbs-devops-cfnhooks/
— cfnhooks-lambda/
— — app.js
— — package.json
— cfnhooks-loggroup/
— — app.js
— — package.json
— template.yaml

Further details of these are as follows:

A. Lambda Hooks Template (template.yaml)

This defines the CloudFormation hook resources, their corresponding lambda functions and required IAM roles to be implemented in target AWS accounts, as below:

AWSTemplateFormatVersion: '2010-09-09'
Description: >
  tbs-devops-cfnhooks

Resources:
  LambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      Path: '/'
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - 'lambda.amazonaws.com'
            Action:
              - sts:AssumeRole
      ManagedPolicyArns:
        - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"

  LambdaHookExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      Path: '/'
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - 'hooks.cloudformation.amazonaws.com'
            Action:
              - sts:AssumeRole
            Condition:
              StringEquals:
                "aws:SourceAccount": !Sub ${AWS::AccountId}
              ArnLike:
                "aws:SourceArn": !Sub "arn:aws:cloudformation:${AWS::Region}:${AWS::AccountId}:type/hook/*"
      Policies:
        - PolicyName: !Sub ${AWS::StackName}-LambdaHookExecutionRole
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: Allow
                Action: lambda:InvokeFunction
                Resource:
                  - !GetAtt LambdaCfnHookFunction.Arn

  LambdaHook:
    Type: AWS::CloudFormation::LambdaHook
    Properties:
      Alias: Tbs::Devops::LambdaCfnHooks
      ExecutionRole: !GetAtt LambdaHookExecutionRole.Arn
      FailureMode: FAIL
      HookStatus: ENABLED
      LambdaFunction: !GetAtt LambdaCfnHookFunction.Arn
      TargetFilters:
        TargetNames:
          - AWS::Lambda::Function
        Actions:
          - CREATE
          - UPDATE
        InvocationPoints:
          - PRE_PROVISION
      TargetOperations:
        - STACK
        - RESOURCE
        - CHANGE_SET
        - CLOUD_CONTROL

  LogGroupHook:
    Type: AWS::CloudFormation::LambdaHook
    Properties:
      Alias: Tbs::Devops::LogGroupCfnHooks
      ExecutionRole: !GetAtt LambdaHookExecutionRole.Arn
      FailureMode: FAIL
      HookStatus: ENABLED
      LambdaFunction: !GetAtt LogGroupCfnHookFunction.Arn
      TargetFilters:
        TargetNames:
          - AWS::Logs::LogGroup
        Actions:
          - CREATE
          - UPDATE
        InvocationPoints:
          - PRE_PROVISION
      TargetOperations:
        - STACK
        - RESOURCE
        - CHANGE_SET
        - CLOUD_CONTROL

  LambdaCfnHookFunction:
    Type: AWS::Lambda::Function
    Properties:
      Architectures:
        - x86_64
      Code: cfnhooks-lambda/
      Handler: app.handler
      MemorySize: 256
      Role: !GetAtt LambdaExecutionRole.Arn
      Runtime: nodejs20.x
      ReservedConcurrentExecutions: 10
      Timeout: 10

  LogGroupCfnHookFunction:
    Type: AWS::Lambda::Function
    Properties:
      Architectures:
        - x86_64
      Code: cfnhooks-loggroup/
      Handler: app.handler
      MemorySize: 256
      Role: !GetAtt LambdaExecutionRole.Arn
      Runtime: nodejs20.x
      ReservedConcurrentExecutions: 10
      Timeout: 10

B. Sample Lambda code(cfnhooks-loggroup)
The following code illustrates implementation for a LogGroup inspection lambda hook, which checks to ensure that log groups are encrypted with a KmsKey, and that a retention period has been set:

export const handler = async (event, context) => {
  var targetModel = event?.requestData?.targetModel;
  var targetName = event?.requestData?.targetName;

  var response = {
    "hookStatus": "SUCCESS",
    "message": "LogGroup is correctly configured.",
    "clientRequestToken": event.clientRequestToken
  };

  if (targetName == "AWS::Logs::LogGroup") {
    let retentionInDays = targetModel?.resourceProperties?.RetentionInDays;
    let kmsKeyId = targetModel?.resourceProperties?.KmsKeyId;

    let errorMessage = ""
    if (!retentionInDays) {
      errorMessage += "LogGroup RetentionInDays must be present.\n "
    }
    if (!kmsKeyId) {
      errorMessage += "LogGroup KmsKeyId must be present.\n "
    }

    if(errorMessage) {
      response.hookStatus = "FAILED";
      response.errorCode = "NonCompliant";
      response.message = errorMessage;
    }
  }
  return response;
};

C. StackSet Template (deploy-setup/template-stackset.yaml):

AWSTemplateFormatVersion: '2010-09-09'
Description: >
  This template creates an S3 bucket for storing built lambda artefacts for devops deployments, which may be accessed
  by other accounts within this AWS Organization

Parameters:
  TargetOrgUnitIds:
    Description: Target organization units for deploying solution
    Type: CommaDelimitedList

  StackSetName:
    Type: String
    Default: tbs-devops-cfnhooks-stackset

  TemplateUrl:
    Description: S3 URL of CF template which defined our SAM solution for lambda hooks
    Type: String

Resources:
  CfnLambdaHookStackset:
    Type: AWS::CloudFormation::StackSet
    Properties:
      AutoDeployment:
        Enabled: true
        RetainStacksOnAccountRemoval: false
      CallAs: DELEGATED_ADMIN # Required when deploying to delegated admin accounts for an org
      Capabilities: [CAPABILITY_IAM, CAPABILITY_NAMED_IAM]
      PermissionModel: SERVICE_MANAGED
      StackInstancesGroup:
        - Regions:
            - ap-southeast-2
          DeploymentTargets:
            OrganizationalUnitIds: !Ref TargetOrgUnitIds
      StackSetName: !Ref StackSetName
      TemplateURL: !Ref TemplateUrl

D. StackSet Deployment Bash Script

#!/bin/bash

stackName="tbs-devops-cfnhooks-stackset"
region="ap-southeast-2"
toolsAccountId="1234" # To replace with real value
targetOrgUnitIds="ou-234,ou-34534" # To replace with real values
deployBucket="lambdacfnhooks-${toolsAccountId}-deploybucket"
currentTime=$(date +"%Y%m%d%H%M%S")

cd ../tbs-devops-cfnhooks

# First we package our cfn-hooks lambda solution
aws cloudformation package --template-file template.yaml \
--s3-bucket $deployBucket --s3-prefix $stackName --region $region \
--output-template-file generated-template.yaml \
--profile thebetterstore-devopsaccount

# Next export our generated template to S3
aws s3 cp ./generated-template.yaml s3://$deployBucket/generated-template.yaml \
--profile thebetterstore-tools


# Next deploy the stackset, defined in IaC (Cfn) to our tools account, which will then manage deployment of the hooks to accounts
# within specified target OU's. NB use lastupdatedtime tag to ensure deployment/that changes are detected
cd ../deploy-setup
aws cloudformation deploy --template template-stackset.yaml --stack-name $stackName \
--capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM --region $region \
--parameter-overrides TemplateUrl="https://$deployBucket.s3.ap-southeast-2.amazonaws.com/generated-template.yaml" \
TargetOrgUnitIds=${targetOrgUnitIds} \
--tags lastupdatedtime="$currentTime" \
--profile thebetterstore-tools

The end result from deployment should be the observation of the following in the AWS Cloudformation UI of target accounts:

Finally we can check our hook functionality by attempting to deploy a stack containing non-compliant resources. For this exercise, I attempted to deploy a new stack called tbs-devops-cfnhooktest in an AWS account belonging to a target Organization Unit, which was comprised of:

Resources:
  LogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: ANonCompliantLogGroup

i.e. containing a log group missing the required retention period and KmsKeyId properties.

Attempting to deploy this resulted in the following errors being thrown by our new Cloudformation LogGroup hook:

Conclusion

Traditional methods for helping to ensure compliance of automatically-deployed resources using Cloudformation/CDK in AWS have generally included developer training, code reviews, ‘shift-left’ guards in deployment pipelines, and detective controls using AWS Config and/or SecurityHub. It is however still very easy for non-compliant configurations to be missed and creep in, particularly in cases where guards or other controls may not be available for an organization’s internal standards such as naming conventions. Security, reliability, performance and maintainability of deployed solutions can all suffer as a result of standards not being met.

The use of Custom Lambda Hooks as preventative controls however look very promising to tackle this challenge; which offer great flexibility for implementing checks or naming conventions that may be required, and generation of custom and meaningful errors when there are issues. Furthermore their implementation and deployment across accounts within AWS Organizations proved fairly straight-forward in this exercise. Targeting Lambda Hooks deployments against Organization Units within AWS Organizations, using ‘Self-Managed’ StackSets also means that any new accounts created within these will also be automatically configured.

It may be that development OU’s should be targeted initially to ensure that compliance issues of stacks are caught during their maintence, and not for example during an emergency hotfix against production! However with the gradual implementation of the hooks across environments, hopefully AWS Config non-compliance alerts may become a thing of the past!

Code used in this post is also available via my public GitHub repository at: https://github.com/TheBetterStore/tbs-devpops-cfnhooks.