Delighted to share my first post as AWS Community Builder, do read the full and share your thoughts.
🔰 Beginners new to AWS CDK, please do look at my previous articles one by one in my previous series on AWS CDK 101 which gives exposure to working from beginner to intermediate level.
If in case missed my previous article, do find it with the below links.
🔁 Original previous series post at 🔗 Dev Post
🔁 Reposted the previous series post at 🔗 dev to @aravindvcyber
In this new series article, we will be talking about new advanced serverless projects series built with AWS CDK using typescript.
This will be also my first series as AWS Community Builder in the developer's tools category.
Without my ado, let me start briefing about this first project I just put it into the public domain for this awesome aws community folks as an opportunity to collaborate and refine this tool which was just started to learn by playing in the cloud solutions.
Expectations 🏗️
This solution is one which I feel could not only help the developer but also their peers along with Leads, Ops, and architects to track the deployment of Cloud formation solutions in AWS cloud and monitor and track any mutations continuously using slack channels and yes in the future drive actions and discussions from the posts to have a seamless developer experience across multi-region and cross-account deployments.
This could very much help the developer/team monitor the deployments of both personal and organizational accounts with less footprint cost-effectively.
Starting with Cloudformation events to post Slack notifications effortless from multi-region and even cross accounts to never miss your/peers/ci initiated aws cloudformation deployments on stacks and resources besides that it could also notify drift events.
Architecture in short 🏗️
The backbone of this stack is based on the new feature released for the availability of cloud formation events in the default bus in the AWS Eventbridge.
These events are of three types.
- Stack level Events
- Resource level Events
- Drift detection Events
This project stack has three components as elaborated below.
-
Event Forwarder Data StackThis has the dynamodb schema and pre-created indexes -
Remote Event Router Stackwhich is deployed into one or many regions across accounts forwarding specific eventbridge events (Cloudformation Events specifically) into the specific target default EventBus (in the below stack) from the current default EventBus (source) by making use of event bridge rules. -
Event Forwarder Stackwhich lives in a single region for event ingestion and transformation to various delivery channels (Slack is the first channel) from the default EventBus (target).
Prerequisites 🏗️
1) At least single region where you could cdk deploy with necessary privileges to spin up the resources such as lambda, sqs, eventbridge rules, and targets with access to cloudwatch and xray
Make use of AWS free tier benefits for personal use or request access sandbox account from your org to try this out.
1) At least one Custom Slack App with incoming webhook generated and configured to post to necessary slack channels
Want to experiment create a free slack workspace (Recommended), creating a slack app with an incoming webhook, or generating an incoming webhook in a slack app from your organization admin? There are multiple tutorials to get this or DM me for clarifications.
Project repo in GitHub 🎢
I have fully shared the code base that I have built into this public repository which now available for your review and learning or even implement this into your account to catch up with the deployments happening across the interested stacks direct into your Organizational or Private Slack workspace. Private Slack workspaces are free to set up and use like me and you could create a simple slack app with an incoming webhook and direct the messages to your channel of choice.
Use cases which emit Cloudformation Events 🎢
- CDK deployment via terminal or even CDK destroy these are visible for the developer but may not be saved anywhere except in the AWS console view. Your peers are also not aware of these deployments happening until they check the console. Sending this to slack channels will drive greater involvements from the team.
- Direct cloudformation console level changes/deletes
Resource level mutations can also be tracked with drift detection but only one person knows what happened and could not easily correlate
Someone edits the Cloud formation template using designer and updates the stack with/without changeset
Or yourself/team can have other parallel IaC framework like terraform making some changes via terminals
Even your CI/CD pipeline does deployments to AWS, you need not check them to understand if the deployments are completed rather slack post will reach your channel in seconds consistently across the included environments
Forbidden environments where you don't have access to checkout the events and resources created and may know if it had failed or does not know the resource names and you could not check this in the console.
Possibilities 🎢
Quite frankly this is my second weekend on this project, I believe that this could be very much improved when I receive feedback from the community and fine-tune and extend the solution in future.
Also I have not restricted the cfn events to any specific stack, since data generation was key to continuously build this solution. Make use of the event bridge rule filtering pattern and matchers to get the interest events only to avoid too much noise in your channels by simply adding more refined patterns.
const stackEventsRule = new Rule(this, "stack-events-rule", {
eventPattern: {
detail: {
"stack-id": [{ exists: true }],
},
source: ["aws.cloudformation"],
},
eventBus,
});
Extracted Utils 🎢
I have made used most of the generic parts of this solution are refracted into separated util functions which could help to simply the solution and may be reuse them in your own project work.
- Slack Utils
- Console Utils
- Dynamodb Utils
- Data model to interact with dynamodb
- Cfn events definition and may be more soon.
Checkout the config folder 🎢
This project is designed to have the various developer expectations and limits we may have so I have made myself aware of those and included most of the configurations below as config.
The default.json and test.json are from GitHub with dummy fields make sure you create local.json and production.json overriding the necessary fields of choice.
{
"account": "123", //Primary account to host the solution
"region": "ap-south-1", //Primary region to host the solution
"slackhook": "", //You need to update your slack app incoming webhook as the primary delivery channel for your users
"errorslackhook": "",//You need to update your slack app incoming webhook as the error delivery channel to notify the stack maintainers in your environment to get the errors as and when they happen and need to dive into cloudwatch logs to figure out most of the issues.
"eventStore": "eventStores9",//This is some new dynamodb table name of your choice
"remoteRegions":"us-east-1,us-east-2", //Your secondary regions to monitor
"remoteAccounts":"123", //Your secondary accounts to monitor
"logLevel":"WARN", //Use WARN for less verbose and use INFO for the verbose cloudwatch logs for the main processor handler
"perPostEventCount": "10",//This is used to limit max N events in a single slack post, this is mainly to make sure we don't hit limit on a single slack post
"dynamodbQueryPagingLimit": "15",//This is used to have limited the read units on a single API call to not throttle your dynamodb if you are using provisioned RCU.
"logRetentionDays": "14",//Retention period for the primary handler cloudwatch logs
"deleteNotified": "false", //Setting this to true will automatically delete data from dynamoDb once the event are notified. But I highly recommend you to have this false to understand the data which gets generated, which could help you with other integrations or possibilities in this monitoring
"timeZone": "Asia/Kolkata", //custom time zone for time
"locale": "en-US" //necessary locale
}
Slack Posts Results
Never miss anything happening to your cloudformation stacks since you will be always notified in your respective slack channel.
Drift detection notifications
Stack deletion notifications
Stack/Resource creation/update notifications
Even exception are sent to slack
This can be directly reported to the maintainers without waiting to dig into the cloudwatch logs to identify and track the issue.
And all these cool things about this slack post it is having a rich format that could be customized at the source and it also include a deep link to the resources provisioned besides the stack-level links to various console screens.
Note these console links will only work if you have already logged into your respective AWS account in the browser where you open them and if you have the necessary privileges to make sure that security even for production environments is honored when you share this across various members of your teams for any follow ups in any slack threads.
Dynamodb to store data
Here in this article, we choose to use dynamodb not only as an ad-hoc data store. I believe the data generated will trigger further insights and expand the possibilities of this solution. Also I have used provisioned RCUs and WCUs to make use of my free tier benefits and as well set throttling, it is also recommend try with on-demand mode and pay as your usage.
Indexes created for critical data lookups
Sample DB item
{
"stackId": "arn:aws:cloudformation:ap-south-1:575066707855:stack/EventForwarderStack/279c3120-1f91-11ed-a6ef-022d5cdd6444",
"time": 1661700649000,
"account": "575066707855",
"clientRequestToken": "",
"detail": "{\"stack-id\":\"arn:aws:cloudformation:ap-south-1:575066707855:stack/EventForwarderStack/279c3120-1f91-11ed-a6ef-022d5cdd6444\",\"stack-drift-detection-id\":\"651434b0-26e6-11ed-817d-06bb5d2f96a6\",\"status-details\":{\"stack-drift-status\":\"\",\"detection-status\":\"DETECTION_IN_PROGRESS\"},\"drift-detection-details\":{\"drifted-stack-resource-count\":-1}}",
"detectionStatus": "DETECTION_IN_PROGRESS",
"driftDetectionDetails": "{\"drifted-stack-resource-count\":-1}",
"eventId": "2892c708-d38d-9a9a-115e-c981c1d3d62f",
"logicalResourceId": "",
"notified": "false",
"notifiedTime": 1661700651498,
"physicalResourceId": "",
"region": "ap-south-1",
"resources": [
"arn:aws:cloudformation:ap-south-1:575066707855:stack/EventForwarderStack/279c3120-1f91-11ed-a6ef-022d5cdd6444"
],
"resourceType": "",
"status": "DETECTION_IN_PROGRESS",
"statusReason": "",
"type": "CloudFormation Drift Detection Status Change"
}
Engineering Aspects 🎢
I would like to discuss this further in the upcoming articles and we refine this stack and refine. There is always opportunities to improve and perfect.
Besides this, I made this repo public now, and I am sure our creative community will like to request features/issues, help patch and contribute and refractor this solution further to feed our discussions in our engineering articles soon beside me.
This project is open to your generous contributions if you feel it will help other developers and also you could also solve the issues or bring new features with a Pull request.
Github Workflow for CI and testing 💥
Added Github Workflow to validate the new changes.
⏭ We have our next article in serverless, do check out
If in case missed my previous article, do find it with the below links.
🔁 Original previous series post at 🔗 Dev Post
🔁 Reposted the previous series post at 🔗 dev to @aravindvcyber
🎉 Thanks for supporting! 🙏 and do follow and share this series for more such articles.
Would be great if you like to ☕ Buy Me a Coffee, to help boost my efforts 😍.
🔁 Original post at 🔗 Dev Post
🔁 Reposted at 🔗 dev to @aravindvcyber
Top comments (0)