AWS CloudWatch: Implementing Data Protection Policy for Sensitive Log Data!

Are you storing sensitive data in your application logs and relying solely on cloud security to protect it? It’s time to think again and revisit your data protection policies!

In this article, I will demonstrate how to implement a data protection policy for sensitive log data in a CloudWatch log group. Logging is crucial for application monitoring; however, it is equally important to ensure that sensitive data is not published in the logs. If such data is unintentionally published due to a bug in the code, a robust data protection policy must be in place to safeguard it. This ensures that associates without the appropriate credentials cannot access sensitive information from the logs.

Examples of sensitive data include SSNs, account numbers, phone numbers, names, addresses, or any other information classified as sensitive by your organization's information security or business policies.

Let's look at the architecture diagram

Why Data Protection Policy?

The AWS Shared Responsibility Model is a foundational concept for understanding security and compliance in the cloud. AWS takes responsibility for securing the infrastructure that runs all AWS services, including hardware, software, networking, and facilities. However, customers are responsible for securing the data and workloads they deploy in AWS.

This means that while AWS provides robust tools, services, and policies to safeguard data, implementing these measures and adhering to best practices rests with the customer. For example, AWS offers features like encryption, access control, and monitoring, but customers must configure and use these tools to protect their applications and data effectively.

With this shared responsibility in mind, this article explores how to implement Data Protection Policy for the CloudWatch Logs, ensuring compliance with security best practices while leveraging AWS's powerful features.

In production environments, logging is essential for monitoring and troubleshooting. However, despite best practices to avoid logging sensitive data, there may be bugs or missed checks where such data inadvertently gets logged. Once exposed, this data could potentially be accessed by other applications or users, creating a security risk.
By implementing AWS CloudWatch Data Protection Policy, we can protect sensitive information in logs. This feature masks sensitive data in log groups and can route the data to Amazon S3, Kinesis Firehose, or another log group for auditing and alerting.

I will use another log group to route the data for auditing and alerting.

Ways to create Data Protection Policy

AWS provides several different ways to configure the Data Protection Policy.

Data Protection Policies can be configured through:

• AWS Management Console
• AWS CLI
• AWS CloudFormation

In this example, I will use AWS Management Console to configure the Data Protection Policy.

I will use name and address data elements as an example data to protect in the published logs.

Select or Create a CloudWatch Log Group

To create a new CloudWatch log group, in the AWS Management Console, navigate to CloudWatch and create a new log group.

To Select an existing CloudWatch log group, in the AWS Management Console, navigate to CloudWatch and select an existing log group.
In this example, I will proceed with an existing log group.

Create a Data Protection Policy

To create a Data Protection Policy, in the AWS Management Console, navigate to CloudWatch and select the log group.

• Once log group is selected, navigate to the Data Protection tab in this log group.
• Select Create Policy

Configure Data Element for masking/protection

CloudWatch offers over 100 predefined data elements to mask, including sensitive information like credit card numbers, email addresses, and names.

Choose the data elements you wish to mask.

• For this example, I’ll select Address and Name as the data elements.

After selecting the elements, click Activate Data Protection.

Configure a destination log group for audit results

AWS allows various destinations like another cloud watch group, S3 or Kinesis Data Firehose to store the audit log results.

For this example, I will select another CloudWatch Log Group as the audit destination.

I’ll use a log group named PIIAuditLogsGB to store audit findings.

Validate the Data Protection Policy by invoking a Lambda function

Now that I have the cloud watch group and data protection policy configured, I need to create logs and validate that data protection policy is working as designed.

I will use a lambda function to log sample data in the log group.
In this demonstration, I will use a GenAI app I created to invoke the lambda function in the backend. Once invoked, this lambda function will create logs in the log group.

Review the results in the audit log

Let's refresh the log group in the CloudWatch.

Open the log stream and look for the entries where name and address elements are masked per the configured Data Protection Policy.

Next, let's review that audit logs are in the target log group.
In our example, this log group is PIIAuditLogsGB.

Validate that the log entries with sensitive information are stored here for auditing.

Cleanup - Delete Data Protection Policy, Delete Log Group

If you don't need the log group, make sure to delete it and also delete the data protection policy.

Conclusion

In this article, I have demonstrated how to implement a CloudWatch Data Protection Policy and mask sensitive data according to the configured elements. I also explained how to store audit records in the designated log group. This setup ensures that any logging of sensitive information is automatically protected, reducing the risk of data exposure.

Logging and monitoring are essential components of observability, and mastering different techniques can greatly enhance your skills in this area.

Thank you for reading!

Click here to watch the YouTube video for this solution:

https://www.youtube.com/watch?v=T1_IiwA3M1I

𝒢𝒾𝓇𝒾𝓈𝒽 ℬ𝒽𝒶𝓉𝒾𝒶
𝘈𝘞𝘚 𝘊𝘦𝘳𝘵𝘪𝘧𝘪𝘦𝘥 𝘚𝘰𝘭𝘶𝘵𝘪𝘰𝘯 𝘈𝘳𝘤𝘩𝘪𝘵𝘦𝘤𝘵 & 𝘋𝘦𝘷𝘦𝘭𝘰𝘱𝘦𝘳 𝘈𝘴𝘴𝘰𝘤𝘪𝘢𝘵𝘦
𝘊𝘭𝘰𝘶𝘥 𝘛𝘦𝘤𝘩𝘯𝘰𝘭𝘰𝘨𝘺 𝘌𝘯𝘵𝘩𝘶𝘴𝘪𝘢𝘴𝘵