[Part 1]What is Amazon EventBridge and How to provision Logging S3 bucket, and bucket policy using AwsCDK

As mentioned in last part

• We already have a S3 bucket which is Amplify set up for us
• Now when a file uploaded to that bucket, we need to notify a lambda function, which we will use other Aws service inside it

What is Amazon EventBridge?

Amazon EventBridge is a serverless event bus service that makes it easy to connect your applications with data from a variety of sources. EventBridge was formerly called Amazon CloudWatch Events. (From aws docs)

Based on my understanding:

• Anything happen in your resource, will emit an events
• events is actually a log files in json format produce by CloudTrail.
• EventBridge is the watcher for this events
• You tell EventBrigde about what events you want to watch(In our case, when a file uploaded to our S3 bucket)
• You tell EventBridge when that events happen what you want to do(In our case, invoke a Lambda Function)

What we need TODO:

• Define S3 bucket to store the event (log files produce by CloudTrail)
• Define the bucket policy for that S3 bucket (Bucket policy)
• Tell EventBridge what pattern of event you want to watch (EventRule)
• Tell EventBridge if that pattern of event happened, what you want it to do. (EventTarget)

Ok, how we do that?
In this blog post, we will provisioned all the resource defined above using AWS CDK.

Why we use CDK?
Think about this, AWS have 160+ services. Every time when we need something, we have to define it 1 by 1 in the AWS management console. Very time consuming, need to find service by service in the console.

No, we don't do that. We write some code using AWS CDK. Then we run a short command cdk deploy, then everything get set up in the cloud.

Nice.

Let's get started

First set up the cdk project using Python:

Make a directory

$ mkdir mycdkapp && cd mycdkapp

Set up a CDK app

$ cdk init app --language python

Activate the virtual environment

$ .env\Scripts\activate.bat

Install all the dependencies needed

$ pip install -r requirements.txt

Cool, so we done setting up a CDK app for our project.

Now you will see mycdkapp_stack.py in your directory. This is where we define all the resource need here.

Refer to the existing S3 bucket and create a new Logging bucket for CloudTrail

Install the library needed:

$ pip install aws_cdk.aws_s3

This will install the L1 construct which represent a single type of aws service. All this create by aws, so every time you need a particular service, you can find the library here

In this case, we install the aws_s3 library.

Now paste the code below into mycdkapp_stack.py

from aws_cdk import (
    aws_s3 as s3,
    core
)


class MycdkappStack(core.Stack):

    def __init__(self, scope: core.Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        # refer the existing s3 bucket, substitute <your-existing-s3-bucket-arn> with your bucket arn
        existing_bucket=s3.Bucket.from_bucket_arn(self, "BucketByArn", "arn:aws:s3:::<your-existing-s3-bucket-arn>")

        # this create a new s3 bucket with name s3LoggingBucket
        logging_bucket = s3.Bucket(self,"s3LoggingBucket", bucket_name="s3loggingbucket")

With just 2 lines of code, we already get the existing_bucket and created a new bucket named s3LoggingBucket.

Now run this

cdk synth > template.yaml

You will see a template.yaml in your directory.

Next we need to set up Bucket policy for this Logging bucket.

1. Make a new directory name contruct inside mycdkapp
2. Create a new file named logging_policy.py
3. pip install aws_cdk.aws_iam
4. Paste this below code into logging_policy.py

from aws_cdk import (
    aws_s3 as s3,
    aws_iam as iam,
    core,
)

class LoggingBucketPolicy(core.Construct):

    def __init__(self,scope: core.Construct, id: str, _bucket: s3.IBucket, **kwargs):
        super().__init__(scope,id, **kwargs)

        cloudtrail_service_principal = iam.ServicePrincipal('cloudtrail.amazonaws.com')
        get_bucket_policy = iam.PolicyStatement(
            actions=["s3:GetBucketAcl"],
            resources=[_bucket.bucket_arn],
            principals=[cloudtrail_service_principal]
            )

        put_object_policy = iam.PolicyStatement(
                actions=['s3:PutObject'],
                resources=[f"arn:aws:s3:::{_bucket.bucket_name}/AWSLogs/{core.Environment.account}/*"],
                principals=[cloudtrail_service_principal]
            )

        _bucket.add_to_resource_policy(get_bucket_policy)
        _bucket.add_to_resource_policy(put_object_policy)

What is this code above do?

• We define a Construct by inherit core.Construct
• This let us group together all the component which doing the same thing(In this case, this construct will define policy to a bucket)
• We will insert s3.IBucket into this construct with variable _bucket
• Define 2 iam policy get_object_policy and put_object_policy
• Allow both policy can use by cloudtrail.amazonaws.com
• Then the policy to the _bucket

Now in our mycdkapp_stack.py add this line:

LoggingBucketPolicy(self, 's3Policy', logging_bucket)

Then run

cdk synth > template.yaml

Now check template.yaml, you will see something like this:

s3LoggingBucketPolicyEF0E9CBA:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket:
        Ref: s3LoggingBucket06C9F6F3
      PolicyDocument:
        Statement:
          - Action: s3:GetBucketAcl
            Effect: Allow
            Principal:
              Service: cloudtrail.amazonaws.com
            Resource:

Done.

*Now you successfully: *

• Understand what is Amazon Eventbridge and why we use it.
• Understand what is Aws CDK and why we use it.
• Refer the existing bucket into our newly created cdk app.
• Create a new Logging bucket for CloudTrail using CDK
• Apply all bucket policy to the new bucket using CDK

*Next we will talking about, *

• How to define a Event rule/pattern for Amazon Eventbrige?
• When the certain pattern in Eventbridge occurred, what should do next?

All using CDK.

Stay tuned.

[Intro] How to trigger a Lambda function when a file is uploaded to an existing S3 bucket using Amazon EventBridge and AWS CDK

[Part 1]What is Amazon EventBridge and How to provision Logging S3 bucket, and bucket policy using AwsCDK

Coming soon
[Part 2] How to define EventRule, EventTarget for Amazon Eventbridge using CDK.

---

Before you go, if you like this series or find this useful consider to buy me a coffee 😊🤞 for 5 USD or more.
I will prepare a GitHub repo for this whole tutorial series and arrange into separate commit for each part.
This will only available for my supporter cause I spent a lot of time to prepare this. Anyway, I appreciate you here. Have a good day.

Follow me on Twitter: @upupkenchoong

My upcoming product(If you interested): @sarah_assistant