Automating detection and alerts of VPC endpoint changes using AWS
CloudTrail, EventBridge, Lambda, SNS, to Email.
This demo demonstrates the automated monitoring of VPC
endpoint events. The goal is to quickly detect when VPC endpoints are created, modified, or deleted and to alert the operations team in real time.
Business Rationale:
In production environments, immediate detection of changes can help prevent configuration drift, reduce security risks, and ensure compliance.
Scope:
The PoC covers event capture using CloudTrail, event filtering with EventBridge, message formatting with Lambda, and alert distribution via SNS to email (with future integration to Microsoft Teams).
AWS CloudTrail:
Captures API calls made in your AWS account. All VPC endpoint events (CreateVpcEndpoint, ModifyVpcEndpoint, DeleteVpcEndpoints) are logged.
Amazon EventBridge:
Filters CloudTrail events using a specific event pattern. Only relevant VPC endpoint events trigger the next step.
AWS Lambda:
A Lambda function is triggered by EventBridge. This function formats the raw CloudTrail event data into a clear alert message.
Amazon SNS:
The formatted alert is published to an SNS topic, which then distributes notifications to subscribed email addresses.
**_Step-by-Step Implementation_**
Step 1: Create the Sample VPC and Network Components
- Create a VPC: • Navigate to VPC Console → Create VPC • Enter details (e.g., CIDR 12.0.0.0/16) and name it “endpoint-vpc.”
- Create Subnets: • Create at least two subnets in different Availability Zones.
- Create a Security Group: • Define necessary inbound/outbound rules.
- Create a Route Table: • Associate with the VPC for gateway-type endpoints.
- Added an inbound rule (TCP 443) to the security group and attached it to the VPC endpoint for secure HTTPS communication.
**_CloudTrail Setup_**
Step 1: Create a New Trail
• Navigated to AWS CloudTrail service.
• Clicked on “Create trail”. • Provided a Trail name (e.g., VpcEndpointMonitoringTrail).
• Selected “Create new S3 bucket” as the storage location.
• Entered an appropriate bucket name (e.g., vpc-endpoint-logs-kranthi).
• Enabled Log file validation for added integrity checks.
Step 2: Enable CloudWatch Logs Integration
• Under CloudWatch Logs, enabled “Send to CloudWatch Logs”. • Selected “Create new IAM role”.
• Assigned a recognizable role name (CloudTrailDeliveryRoleVpcEndpoint).
• This allows CloudTrail to send logs to CloudWatch for real-time
monitoring and alerting.
• Reviewed all settings, “Create trail”.
• Confirmation message appeared stating the trail was created
successfully.
Step 4: Verify S3 Bucket and Log Delivery
• Navigated to the S3 console.
• Confirmed that the bucket (vpc-endpoint-logs-kranthi) was successfully created.
• Verified that log files were being delivered to the bucket under the
specified prefix structure ( AWSLogs//CloudTrail/).
Amazon SNS Topic for Alerts setup
Create Amazon SNS Topic for Alerts
• Navigated to the Amazon SNS console.
• Chose to create a new topic.
1.Selected Standard type.
• Standard topics offer high throughput and best-effort ordering.
• In contrast, FIFO topics provide strict ordering and exactly-once message delivery, but with lower throughput.
2.Provided a name: vpc-endpoint-alert
3.Create SNS topic.
4.Create SNS Subscription
• Clicked on the newly created topic to create a subscription.
• Set the protocol as Email.
• Entered the email address as the endpoint (alerts@yourdomain.com).
• Used the Topic ARN of the previously created SNS topic.
• Subscription created successfully.
• Verified that a confirmation email was sent to the specified email address.
• Opened the subscription email and clicked the confirmation link to approve the SNS subscription.
• The email endpoint was successfully confirmed to receive notifications.
Steps to Create Amazon EventBridge Rule
1.Navigate to EventBridge Console
• Open the AWS Management Console and go to Amazon EventBridge.
2.Create Rule
• Click on "Create rule" and proceed to define the rule details.
3.Define Rule Details
• Enter rule name: vpcendpointchangerule
• Select Event Source as "Event Source with an event pattern"
3.Configure Event Pattern
• Choose "Build event pattern" > Other
• Select Custom pattern (JSON editor)
{
"source": ["aws.ec2"],
"detail-type": ["AWS API Call via CloudTrail"],
"detail": {
"eventSource": ["ec2.amazonaws.com"],
"eventName": [
"CreateVpcEndpoint",
"ModifyVpcEndpoint",
"DeleteVpcEndpoints"
]
}}
• Write custom event pattern that listens to VpcEndpoints actions.
4.Add Target
• Select Target 1 as AWS Service, Choose SNS topic.
• Select the previously created topic: vpc-endpoint-alerts.
• Choose to Create a new execution role.
• Define a recognizable role name for easy identification.
5.Review and Create
• The final image confirms that the EventBridge rule vpcendpointchangerule was successfully created and is now active.
**_About VPC Endpoints_**
What is a VPC Endpoint?
A VPC endpoint enables private connectivity between your VPC and supported AWS services or VPC endpoint services, powered by PrivateLink, without requiring internet access, NAT gateway, or VPN.
Use Cases
• Access Amazon S3 or DynamoDB from private subnets.
• Enhance security by keeping traffic within the AWS network.
• Reduce latency and data transfer costs. • Meet compliance and audit requirements for isolated networks.
**_VPC Endpoint Creation & Email Notification Testing_**
- Navigate to VPC Console • Open the AWS Management Console and go to the VPC service. • In the Endpoints section, observe that no endpoints currently exist.
- Create a New VPC Endpoint • Click “Create endpoint”. • For Service category, select AWS Services. • In Service name, choose any supported AWS service.
3. Configure VPC Settings
• Choose the VPC you created in the initial step of the POC.
• Select appropriate subnets and route tables associated with your VPC.
• Leave the default options or customize Policy as needed.
• Click “Create endpoint”.
4. Trigger and Validate Email Notification
• After creating the endpoint, the EventBridge rule triggers an SNS notification.
• Check the configured email inbox and observe that anotification was received.
• The notification, however, appears unstructured and raw, making it difficult to read or parse.
Lambda Integration for Structured Notifications
Why Use AWS Lambda?
AWS Lambda lets you run backend code without provisioning or managing servers. It's event-driven and integrates well with other AWS services.
Common Use Cases for Lambda:
• Format and forward notifications.
• Filter or enrich incoming events.
• Automatically remediate certain changes.
• Connect multiple AWS services efficiently.
Purpose in This POC
In this POC, Lambda will:
• Process the raw EventBridge message.
• Convert it into a structured and readable format.
• Send the refined output to the SNS topic, which emails it to recipients.
**_Lambda Setup for Structuring Notifications_**
1. Create Lambda Function
• Navigate to the AWS Lambda Console and click “Create function.”
• Select Author from scratch.
• Enter the function name: formatvpcendpoint.
• Choose Python 3.13 as the runtime.
• Under Permissions, select Create a new role with basic Lambda
permissions.
2. Add Notification Formatting Code
• Paste the Python code that:
Parses the incoming EventBridge JSON -->Formats key fields (action, time, VPC ID) -->Publishes the structured message to the SNS topic --> Save your changes.
import os
import boto3
sns = boto3.client('sns')
def lambda_handler(event, context):
topic_arn = os.environ.get('SNS_TOPIC_ARN')
if not topic_arn:
print(" SNS_TOPIC_ARN environment variable is not set.")
return
# build your message as you did
message = (
" *VPC Endpoint Alert*\n"
f" Time: {event['time']}\n"
f" Region: {event['region']}\n"
f" User: {event['detail']['userIdentity'].get('userName', 'N/A')}\n"
f" Action: {event['detail']['eventName']}\n"
f" VPC ID: {event['detail']['requestParameters'].get('VpcId', 'N/A')}\n"
f" Service: {event['detail']['requestParameters'].get('ServiceName', 'N/A')}\n"
f"Check AWS Console for more details."
)
print("Formatted message:")
print(message)
sns.publish(
TopicArn=topic_arn,
Message=message,
Subject=" VPC Endpoint Action Alert"
)
3. Deploy and Test the Function
• Click “Deploy” to publish your function changes.
• Use the “Test” feature to run the function with sample EventBridge event data and verify the output is formatted as expected.
- Configure Environment Variables • Go to the Configuration tab → Environment variables. • Add a key-value pair: o Key: SNS_TOPIC_ARN o Value: (Paste the ARN of the SNS topic created earlier, arn:aws:sns:ap-southeast-1:xxxxxxx:vpc-endpoint-alert)
Update EventBridge Rule Target
• Go to Amazon EventBridge → Rules → Select your rule vpcendpointchangerule.
• Edit the rule’s target:
o Change the target from SNS to Lambda function.
o Select the Lambda function formatvpcendpoint.
• Save the updated rule configuration.
Important Troubleshooting Step – Resolving Lambda Permission Issue
I successfully created a VPC endpoint, but did not receive any email notification.
This happened because the newly created Lambda function role lacked authorization to publish to the SNS topic.
Handling Lambda Permission Error for SNS
Step 1: Identifying the Issue
• After deploying the Lambda function and triggering the EventBridge rule, CloudWatch Logs showed an error indicating sns:Publish permission was denied.
• This was an expected error, as the Lambda execution role was created with only basic Lambda permissions.
**_2: Resolving the Permission Issue: _**
• Created the following inline policy to allow the Lambda function to
publish messages to the SNS topic:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"sns:Publish"
],
"Resource": [
"arn:aws:sns:ap-southeast-1:<AWS account number>:vpc-endpoint-alerts"
]
}
]}
• Attached this policy to the Lambda execution role.
Step 3: Verification
• Triggered a new VPC Endpoint creation to test the fix.
• Checked CloudWatch Logs – the authorization error was resolved.
• A structured email notification was received as expected, confirming end-to-end functionality.
Top comments (0)