Introduction:
In this blog post, we delve into creating CloudFormation nested stacks and automating their deployment using AWS CodePipeline. This discussion builds on the GitOps deployment strategy previously explored, where we detailed modeling GitOps environments and promoting releases across them using an “environment-per-folder” approach. While my earlier blog post (link here) focused on updating standalone CloudFormation stacks with the CloudFormation Git Sync feature, here we expand our scope to include nested stacks, offering a more structured solution for managing complex infrastructure.
About the Project:
This project is structured into two main folders: codepipeline and infrastructure. The codepipeline folder contains the codepipeline_pipeline.yaml template, which establishes all required configurations for orchestrating the deployment of nested stacks.
Key resources set up by this template include:
CodeStar Connection: Facilitates integration with remote Git repository.
IAM Roles and Policies: Three roles are defined to securely manage resources during the pipeline operations — creating resources via CloudFormation, executing CodeBuild projects, and running CodePipeline workflows.
S3 Bucket and S3 Bucket Policy: Provides storage for nested stack templates and CodePipeline artifacts.
CodeBuild Project: Executes cfn-lint to ensure the nested stack templates adhere to best practices before deployment.
The infrastructure folder has environment-specific folders containing simple CloudFormation templates for S3 bucket creation and associated policies. These serve as examples to illustrate the deployment process within a GitOps framework.
Pipeline Overview:
The CodePipeline setup includes several stages crucial for streamlined operations:
Source: Monitors the Git repository for changes and triggers the pipeline.
CFN-Lint: Validates the CloudFormation templates against common errors and best practices.
Copy-to-S3: Processes and stores files from the Source stage to the S3 bucket.
Deploy-CFN-stacks: Handles the creation and updates of CloudFormation nested stacks for each environment.
The project structure:
├── codepipeline
│ └── codepipeline_pipeline.yaml
└── infrastructure
├── development
│ ├── root.yaml
│ ├── s3_bucket.yaml
│ └── s3_bucket_policy.yaml
├── production
│ ├── root.yaml
│ ├── s3_bucket.yaml
│ └── s3_bucket_policy.yaml
└── staging
├── root.yaml
├── s3_bucket.yaml
└── s3_bucket_policy.yaml
The CodePipeline pipeline configuration from codepipeline_pipeline.yaml:
CreateCfnStackFromRepo:
Type: 'AWS::CodePipeline::Pipeline'
Properties:
Name: !Ref CodePipelineName
RoleArn: !GetAtt CodePipelineRole.Arn
ArtifactStore:
Type: S3
Location: !Ref S3BucketName
Stages:
- Name: Source
Actions:
- Name: Source
ActionTypeId:
Category: Source
Owner: AWS
Provider: CodeStarSourceConnection
Version: '1'
RunOrder: 1
Configuration:
BranchName: !Ref BranchName
ConnectionArn: !GetAtt GitLabConnection.ConnectionArn
DetectChanges: 'true'
FullRepositoryId: !Ref FullRepositoryId
OutputArtifactFormat: CODE_ZIP
OutputArtifacts:
- Name: SourceArtifact
Namespace: SourceVariables
- Name: CFN-Lint
Actions:
- Name: Run-CFN-Lint
ActionTypeId:
Category: Build
Owner: AWS
Provider: CodeBuild
Version: '1'
Configuration:
ProjectName: !Ref CfnlintCodeBuildProject
InputArtifacts:
- Name: SourceArtifact
OutputArtifacts:
- Name: CflintArtifact
RunOrder: 1
- Name: Copy-to-S3
Actions:
- Name: Copy-to-S3
ActionTypeId:
Category: Deploy
Owner: AWS
Provider: S3
Version: '1'
RunOrder: 1
Configuration:
BucketName: !Ref S3BucketName
Extract: 'true'
InputArtifacts:
- Name: SourceArtifact
- Name: Deploy-CFN-stacks
Actions:
- Name: DeployDevelopmentStack
ActionTypeId:
Category: Deploy
Owner: AWS
Provider: CloudFormation
Version: '1'
Configuration:
ActionMode: CREATE_UPDATE
Capabilities: 'CAPABILITY_NAMED_IAM,CAPABILITY_AUTO_EXPAND'
StackName: !Sub '${CodePipelineName}-development'
TemplatePath: SourceArtifact::infrastructure/development/root.yaml
RoleArn: !GetAtt CloudFormationExecutionRole.Arn
ParameterOverrides: |
{
"Environment": "development"
}
InputArtifacts:
- Name: SourceArtifact
RunOrder: 1
- Name: DeployStagingStack
ActionTypeId:
Category: Deploy
Owner: AWS
Provider: CloudFormation
Version: '1'
Configuration:
ActionMode: CREATE_UPDATE
Capabilities: 'CAPABILITY_NAMED_IAM,CAPABILITY_AUTO_EXPAND'
StackName: !Sub '${CodePipelineName}-staging'
TemplatePath: SourceArtifact::infrastructure/staging/root.yaml
RoleArn: !GetAtt CloudFormationExecutionRole.Arn
ParameterOverrides: |
{
"Environment": "staging"
}
InputArtifacts:
- Name: SourceArtifact
RunOrder: 1
- Name: DeployProductionStack
ActionTypeId:
Category: Deploy
Owner: AWS
Provider: CloudFormation
Version: '1'
Configuration:
ActionMode: CREATE_UPDATE
Capabilities: 'CAPABILITY_NAMED_IAM,CAPABILITY_AUTO_EXPAND'
StackName: !Sub '${CodePipelineName}-production'
TemplatePath: SourceArtifact::infrastructure/production/root.yaml
RoleArn: !GetAtt CloudFormationExecutionRole.Arn
ParameterOverrides: |
{
"Environment": "production"
}
InputArtifacts:
- Name: SourceArtifact
RunOrder: 1
CodePipeline pipeline stages schema:
Prerequisites:
Before you start, make sure the following requirements are met:
- An AWS account with permissions to create resources.
- AWS CLI installed on your local machine.
- Account in any Git provider for remote Git repository.
Deployment:
- Clone the repository to get started.
git clone https://gitlab.com/Andr1500/cloudformation_stack_from_codepipeline.git
- Set up the required AWS and GitLab integration:
a. Deploy a CloudFormation stack with necessary services and roles:
aws cloudformation create-stack \
--stack-name codepipeline-pipeline-cfn \
--template-body file://codepipeline/codepipeline_pipeline.yaml \
--capabilities CAPABILITY_NAMED_IAM --disable-rollback
b. Authorise the CodeStar connection via the AWS Console under CodePipeline settings to link to your Git repository.
Create CloudFormation nested stacks. Update the templates in the infrastructure folders as needed, then commit and push these changes. The pipeline will handle creating the stacks automatically.
Update nested stacks. Always use Git to make changes to your templates and push them to apply. Avoid using the AWS Management Console for updates. To move updates from one environment (like development) to another (like staging), copy the files, commit, and push:
cp -rf infrastructure/development/* infrastructure/staging/
- Delete nested Stacks. If you no longer need the nested stacks, they can be deleted either through the AWS Management Console, using the AWS CLI, or by configuring the CodePipeline to perform the deletion. To enable deletion via the pipeline, modify the Deploy-CFN-stacks stage in your pipeline configuration by changing the ActionMode from CREATE_UPDATE to DELETE_ONLY. This adjustment will instruct the pipeline to remove the stacks rather than update or create new ones.
Conclusion:
AWS CodePipeline is a powerful tool that can be customized for complex deployment tasks like managing CloudFormation nested stacks. For enhanced oversight, consider setting up a Notification rule linked to an SNS topic to monitor pipeline activities. You can find the setup for this in my other project here.
If you found this post helpful and interesting, please click the reaction button below to show your support for the author. Feel free to use and share this post! 🙂
Top comments (0)