My Project: Implementing Amazon Macie with EventBridge and SNS

Project Introduction

In this project we are implementing the functionality of Amazon Macie. Amazon Macie is a machine learning service which can automatically evaluate any data stored in s3, identify and take action based on sensitive data.
• We will be creating a discovery job to identify findings within Macie.
• Findings will be identified using Managed data identifiers and/or Custom data identifier.
• A SNS topic is created that will have the configuration of valid publishers and subscribers to the topic.
• A subscription is created for the above SNS topic with email endpoint.
• A ‘pattern form event’ is created in EventBridge with event source as AWS Macie and Target as SNS topic.
Steps & Workflow

1. Uploading sensitive data to s3
2. Create a discovery job in Macie using Managed Data Identifier
3. Create SNS Topic for notifications
4. Create an EventBridge rule whenever Macie has any findings
5. Create Custom Data Identifier Demo
6. Uploading sensitive data to s3 Let’s begin by creating a new bucket in s3 and upload the data

Upload the text files which contains sensitive data

accesscredentials.txt
AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
AWS_SESSION_TOKEN=AQoDYXdzEPT//////////wEXAMPLEtc764bNrC9SAPBSM22wDOk4x4HIZ8j4FZTwdQWLWsKWHGBuFqwAeMicRXmxfpSPfIeoIYRqTflfKD8YUuwthAx7mSEI/qkPpKPi/kMcGdQrmGdeehM4IC1NtBmUpp2wUE8phUZampKsburEDy0KPkyQDYwT7WZ0wq5VSXDvp75YU9HFvlRd8Tx6q6fE8YQcHNVXAkiY9q6d+xo0rKwT38xVqr7ZD0u0iPPkUL64lIZbqBAz+scqKmlzm8FDrypNC9Yjc8fPOLn9FX9KSYvKTr4rvx3iSIlTJabIQwj2ICCR/oLxBA==
github_key: c8a2f31d8daeb219f623f484f1d0fa73ae6b4b5a
github_api_key: c8a2f31d8daeb219f623f484f1d0fa73ae6b4b5a
github_secret: c8a2f31d8daeb219f623f484f1d0fa73ae6b4b5a
creditcards.txt
American Express
5135725008183484 09/26
CVE: 550

American Express
347965534580275 05/24
CCV: 4758

Mastercard
5105105105105100
Exp: 01/27
Security code: 912
customdata.txt (Australian license plates)

Victoria

1BE8BE
ABC123
DEF-456

New South Wales

AO31BE
AO-15-EB
BU-60-UB

Queensland

123ABC
000ZZZ
987-YXW
employeedata.txt
74323 Julie Field
Lake Joshuamouth, OR 30055-3905
1-196-191-4438x974
53001 Paul Union
New John, HI 94740
Amanda Wells

354-70-6172
242 George Plaza
East Lawrencefurt, VA 37287-7620
GB73WAUS0628038988364
587 Silva Village
Pearsonburgh, NM 11616-7231
LDNM1948227117807
Brett Garza

1. Create a discovery job in Macie using Managed Data Identifier We are now creating a job to analyze the data in s3 bucket.

Select Managed data identifier as ‘Recommended’ i.e. All managed data identifiers that AWS recommends.

At this stage we are not adding any custom data identifiers. We will be doing that later in the project

The discovery job is now created

Click on the job → Show results → Show findings.

We will be able to see the type of sensitive data that has been identified by Macie. In our case, it is personal, credentials and financials.
Notice that license plates information is not flagged. We need to create custom data identifier for Macie to flag that content. We will be doing that in step 5. In the next step we are setting up a notification so that we get notified whenever Macie has identified a data as sensitive.

1. Create SNS Topic for notifications Navigate to SNS console and create a SNS topic

Next, we need to create a subscription for this SNS topic.

The subscription is now created:

1. Create an EventBridge rule whenever Macie has any findings

Select the target for the event. In our case it is SNS topic. Whenever Macie identifies any sensitive data, this event bridge rule will be triggered and it will send that event through SNS and we will be notified (since we subscribed to the SNS topic).

The rule is now enabled

1. Create Custom Data Identifier Now lets create a custom data identifier to identify the below data. If we notice the below data is not flagged earlier by Macie. customdata.txt (Australian license plates) # Victoria 1BE8BE ABC123 DEF-456 # New South Wales AO31BE AO-15-EB BU-60-UB # Queensland 123ABC 000ZZZ 987-YXW Navigate to Macie → Custom Data Identifier

In the regular expression type the below expression. This is the expression that will identify Australian license plates
([0-9][a-zA-Z][a-zA-Z]-?[0-9][a-zA-Z][a-zA-Z])|([a-zA-Z][a-zA-Z][a-zA-Z]-?[0-9][0-9][0-9])|([a-zA-Z][a-zA-Z]-?[0-9][0-9]-?[a-zA-Z][a-zA-Z])|([0-9][0-9][0-9]-?[a-zA-Z][a-zA-Z][a-zA-Z])|([0-9][0-9][0-9]-?[0-9][a-zA-Z][a-zA-Z])
Now, lets configure a new job and select the created custom data identifier for Macie to identify Australian License plates

The job is now created

If we navigate to show findings we can notice that the license plates data is now flagged by Macie

We also received an email notification via EventBridge

Thanks for reading the article.