DevSecOps with AWS – ChatOps with AWS and AWS Developer Tools – Part 1

Level 200

Table of Contents

• Hands On
  • Requirements
  • AWS Services
  • Solution Overview
  • Step by Step

The “boom” of AI is transforming the industry sure thing you are listening to in all social media comments and many applications based on ChatGPT API, an application programming interface that allows users to experience an AI-based chatbot. It is one of the most innovative AI models ever created. Imagine if you could create and respond to all IT operations task through chat channels centralizing the management of infrastructure and applications, as well as to automate and streamline your workflows (ChatOps) just move single pane of glass to your familiar chat interface to get the job done, for example, Microsoft teams or slack.
Today, base on previous post, you can learn how use AWS Chatbot for integrate Microsoft Teams and take actions like:

• Approve pipelines.
• Start, Stop pipelines.
• Get alarms.
• Describe resources.
• Receive predefined CloudWatch dashboards interactively.
• Retrieve Logs Insights logs to troubleshoot issues directly from the chat thread. ...

There aren't limits!

Hands On

Requirements

• cdk >= 2.73.0
• AWS CLI >= 2.7.0
• Python >= 3.10.4
• Pytest >= 7.1.3
• cdk-nag >=2.18.44
• checkov >= 2.1.229

AWS Services

• AWS Cloud Development Kit (CDK): is an open-source software development framework to define your cloud application resources using familiar programming languages.
• AWS Identity and Access Management (IAM): Securely manage identities and access to AWS services and resources.
• AWS IAM Identity Center (Successor to AWS Single Sign-On): helps you securely create or connect your workforce identities and manage their access centrally across AWS accounts and applications.
• AWS CodeCommit: secure, highly scalable, managed source control service that hosts private Git repositories.
• AWS CodeBuild: fully managed continuous integration service that compiles source code, runs tests, and produces software packages that are ready to deploy.
• AWS CodePipeline: fully managed continuous delivery service that helps you automate your release pipelines for fast and reliable application and infrastructure updates.
• AWS Key Management Service (AWS KMS): lets you create, manage, and control cryptographic keys across your applications and more than 100 AWS services.
• AWS CloudFormation: Speed up cloud provisioning with infrastructure as code as code
• AWS Lambda: A serverless compute service that lets you run code without provisioning or managing servers, build workload-based cluster scaling logic, maintain event integrations, or manage runtimes.
• AWS Chatbot: Monitor, operate, and troubleshoot your AWS resources with interactive ChatOps

Solution Overview

Figure 1- Solution Overview

The Figure 1 depicts the general architecture for this demonstration:

1. The pipeline starts or change.
2. The pipeline events are captured and trigger a lambda service
3. Lambda service loads the secret webhook url from secrets manager and send the notification card to Microsoft teams Channel.
4. There is an alternative for sending notifications easy and simple, using codestar notifications or SNS with email subscription.
5. Finally the manual steps as approve transition is invoked from chat using AWS Chatbot Service.

Step by Step

1. Create webhook URL in Microsoft Teams -> Create an Incoming Webhook - Teams | Microsoft Learn

2. Create a custom lambda for sending messages to Microsoft Teams.

The following is a fragment code, according to Monitoring CodePipeline events
Code example- Here the library pymsteams is used to construct a message card with the events information.

import os
import pymsteams
from get_secret import get_secret


def set_state(message, state, connection_card, activity_text):
    if message['detail']['state'] == 'STARTED':
        state.text(message['detail']['state'])
        state.activityText(activity_text)
        state.activityImage(
            "https://support.content.office.net/en-us/media/6b8c0bff-7ddc-4bff-9101-8360f8c8a727.png")
    elif message['detail']['state'] == 'SUCCEEDED':
        state.activityText(activity_text)
        state.activityImage(
            "https://support.content.office.net/en-us/media/773afccb-4687-4b9f-8a89-8b32f640b27d.png")
    elif message['detail']['state'] == 'FAILED':
        state.activityText(activity_text)
        state.activityImage(
            "https://support.content.office.net/en-us/media/c9ed80c9-a24c-40b0-ba59-a6af25dc56fb.png")

    else:
        state.activityImage(
            "https://support.content.office.net/en-us/media/47588200-0bf0-46e9-977e-e668978f459c.png")

        state.text(message['detail']['state'])
    connection_card.addSection(state)


def lambda_handler(event, context):
    print(event)
    #
    # Get event source
    #

    region = os.environ['AWS_REGION']
    # message = event['Records'][0]['Sns']['Message']
    message = event

    webhook_secret_url = get_secret()
    # message = json.loads(message)

    print(message)
    if message['source'] == "aws.codepipeline":
        team_message = pymsteams.connectorcard(webhook_secret_url)

        # create the section action
        state = pymsteams.cardsection()
        pipeline = message['detail']['pipeline']
        # Section Title
        state.title("State")

        if message["detail-type"] == "CodePipeline Action Execution State Change":

            team_message.title(
                f"Action Execution State Change - {message['detail-type']} in {message['detail']['pipeline']} Pipeline")
            team_message.color("7b9683")

            # Add text to the message.
            team_message.text(
                f"Action Execution State Change in pipeline {message['detail']['pipeline']} in account {message['account']} in region {message['region']} \n "

            )

            # create the section stage

            stage = pymsteams.cardsection()
            # create the section action
            action = pymsteams.cardsection()
            # Section Title
            action.title("Action")
            action.text(message['detail']['action'])

            team_message.summary("Approval deployment")
            team_message.addSection(stage)

            set_state(message=message, state=state, connection_card=team_message,
                      activity_text='Action Execution State Change! -Review if you need take action')
            team_message.addSection(action)

            if 'execution-result' in message['detail']:
                # create the section result
                result = pymsteams.cardsection()
                # Section Title
                result.title("Result")
                summary = message['detail']['execution-result'].get('external-execution-summary', '')
                result_summ = message['detail']['execution-result'].get('external-execution-state', '')
                result_url = message['detail']['execution-result'].get('external-execution-url', '')
                result.text(f"{summary} - {result_summ}")
                if result_url != '':
                    team_message.addLinkButton("Go to the console",
                                               result_url)

                team_message.addSection(result)

        elif message["detail-type"] == "CodePipeline Pipeline Execution State Change":
            print("State change ... \n")
            team_message.title(
                f"State Change in {message['detail']['pipeline']} Pipeline")
            team_message.color("7b9683")
            # Add text to the message.
            team_message.text(
                f"Stage Change in pipeline {message['detail']['pipeline']} in account {message['account']} in region {message['region']} \n "

            )

            set_state(message=message, state=state, connection_card=team_message, activity_text='')

        elif message["detail-type"] == "CodePipeline Stage Execution State Change":

            print("Stage change ... \n")
            team_message.title(
                f"Stage Change in {message['detail']['pipeline']} Pipeline")
            team_message.color("7b9683")

            # Add text to the message.
            team_message.text(
                f"Stage Change in pipeline {message['detail']['pipeline']} in account {message['account']} in region {message['region']} \n "

            )
            pipeline = message['detail']['pipeline']
            # create the section stage

            stage = pymsteams.cardsection()
            # create the section action
            # Section Title
            stage.title(f"Stage {message['detail']['stage']}")
            set_state(message=message, state=state, connection_card=team_message,
                      activity_text="")

            team_message.summary("Stage Change")
            team_message.addSection(stage)

            team_message.addLinkButton("Go to the console",
                                       f"https://{region}.console.aws.amazon.com/codesuite/codepipeline/pipelines/{pipeline}/view?region={region}")

        team_message.send()
...

1. Create event rules for any codepipeline event.

Finally, in the construct L3 the lambda and events are created:

from aws_cdk import (
    SecretValue,
    aws_secretsmanager as secretsmanager,
    aws_lambda as _lambda,
    aws_codedeploy as codedeploy,
    aws_codepipeline as pipeline,
    aws_events as events,
    aws_events_targets as targets,
    Duration
)
import os
from constructs import Construct


class LambdaNotification(Construct):

    def __init__(self, scope: Construct, construct_id: str,
                 props: dict = None,
                 pipeline: pipeline.IPipeline = None,
                 emails: list = None,

                 **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        dirname = os.path.dirname(__file__)

        # lambda definitions for events
        lambda_notification = _lambda.Function(self, f"lambdanotification_{props['project_name']}",
                                               runtime= _lambda.Runtime.PYTHON_3_9,
                                               code=_lambda.Code.from_asset(
                                                   os.path.join(dirname, "lambda_notifications/lambda_function")),
                                               handler='app.lambda_handler',
                                               function_name=f"lambdanotification_{props['project_name']}",
                                               timeout=Duration.seconds(70),
                                               environment={
                                                   "webhook_secret_name": f"webhook_{props['project_name']}_channel"
                                               }
                                               )

        secret = secretsmanager.Secret(self, f"webhook_{props['project_name']}_channel",
                                       description="WebHook Url for notifications channel in microsoft teams",
                                       secret_name=f"webhook_{props['project_name']}_channel",
                                       secret_string_value=SecretValue.unsafe_plain_text(props["webhook_ops_channel"])
                                       )
        secret.grant_read(lambda_notification)
        alias = _lambda.Alias(self, f"{props['project_name']}-LambdaAlias",
                              alias_name="Prod_2", version=lambda_notification.current_version)

        codedeploy.LambdaDeploymentGroup(self, f"{props['project_name']}-lambda-DeploymentGroup",
                                         alias=alias,
                                         deployment_config=codedeploy.LambdaDeploymentConfig.ALL_AT_ONCE
                                         )
        pipe_states = pipeline.on_event(id=f"{props['project_name']}-pipelinevent",
                                        description="Event for activate lambda when app_pipeline",
                                        event_pattern=events.EventPattern(
                                            detail={
                                                "state": [
                                                    "SUCCEEDED",
                                                    "INPROGRESS",
                                                    "FAILED",
                                                    "SUPERSEDED",
                                                    "STOPPED",
                                                    "STOPPING",
                                                    "CANCELED",
                                                    "STARTED",
                                                    "RESUMED",
                                                    "ABANDONED"

                                                ]
                                            }
                                        )
                                        )

        # Add subscription event lambda
         pipe_states.add_target(targets.LambdaFunction(lambda_notification))

Here, the event bridge service has one rule that listen the events from CodePipeline.

Secrets Manager saves the webhook URL for Microsoft teams channel.

1. Add notifications object into the pipeline.

Finally, the notifications are added to the main pipeline construct:

...
# Enable notifications
        if enable_notifications == "true":
            LambdaNotification(self, "Notifications", props= props,pipeline= pipeline.pipeline)
        # Define Outputs
...

In the environment options file include the enable_notifications option:

# Multi Environment setup
devsecops_account: "123456789012"
devsecops_region: "us-east-2"
enable_notifications: "true"

Push your changes or redeploy the pipeline stack.

Now, you can get the notifications on teams channel.

Figure 2- Teams Notifications.

you can watch the notifications into your channel.

In the next blog you can learn how configure and integrate AWS chatbot.

You can find the code here ⬇️ ⬇️

velez94 / cdkv2_pipeline_multienvironment

Repository for cdk pipelines for multiaccount environment

• CDK Pipelines Multi Environment Devployment
  • Solution Overview
• How to
  • Code structure
  • Parametrize the project
  • Useful commands

CDK Pipelines Multi Environment Devployment

This is a project for CDK development with Python for creating multi AWS account deployment.

Solution Overview

Solution Overview – Enhanced CDK Pipeline

The Figure shows the steps to accomplish this task. Also, it shows a cross account pipeline using AWS CodePipeline, AWS CodeCommit, AWS Codebuild and AWS CloudFormation. But, how can you construct this pipeline secure and with the minimum effort? The answer [CDK Pipelines](https://docs.aws.amazon.com/cdk/v2/guide/cdk_pipeline.html).

This pipeline is a default pipeline composed by next steps 1.The changes are detected and activate de pipeline. For this demo the branch master is the default branch. 2.The CDK project is synthesized if is aligned with AWS Security Best practices. 3.The pipeline run self-update action. 4.The unit test runs, and its report is published in codebuild reports group. 5.The SAST…

View on GitHub

In the next blog you can learn how configure and integrate AWS Chatbot.
Thanks for reading and sharing 😃