How to Get Custom Email Notification for EC2 State Changes Using EventBridge & Lambda

Introduction:

In any sort of architecture, monitoring is an important part without which an architecture cannot be treated as well architect. So, Cloudwatch is there to provide monitoring service, but aoart from this, any kind of state change must be tracked so closely so that operations team can be aware of this change and work on accordingly. Amazon EventBridge provides that level of triggering service which gets triggered in different sort of activities and SNS topic would be there to get people notified. One of those use cases is EC2 state change.

Suppose someone did patching activity and rebooted post patching or someone launched some new instances or due to some network glitches server got rebooted, but you're not notified as there is no alerting system configured. Definitely there is a ap which you think off to get those fixed, otherwise servers would keep starting/rebooting/stopping and team won't be aware.

To sort this issue, EventBridge comes into the picture with integration of SNS topic. So, you might be thinking that SNS will send an alert to recipients. YES, that's correct, it will send an alert for sure, but that notification email seems to be clumsy to you as you'll see so many information which seems not to be needed. Customers prefect neat and clear and "to-the-point" information to have better clarification. Here is a problem as sns notification email cannot be modified according to customers wish. That clearly tells that scripting is needed for sure to get this implemented.

Pattern:

Solutions Overview:

This blog consists of following steps:

1.Create an EventBridge rule as below.

So, here basically EventBridge rule is checking below API calls from CloudTrail -

• RunInstances
• StopInstances
• StartInstances
• RebootInstances

1. Associate this rule with Lambda function. Below is the python code which takes events details in JSON as an input and fetches required details from JSON content.

import os
import json
import boto3

def lambda_handler(event, context):

    EventID = event['detail']['eventID']
    Account = event['account']
    Timestamp = event['time']
    Region = event['region']
    InstanceID = event['detail']['requestParameters']['instancesSet']['items']
    EventName = event['detail']['eventName']
    SourceIP = event['detail']['sourceIPAddress']
    InitiatedBy = event['detail']['userIdentity']['arn']

    msg_status = {'StopInstances': 'stopped', 'StartInstances': 'started', 'TerminateInstances': 'terminated', 'RebootInstances': 'rebooted'}
    instance_id = []

    if len(InstanceID) > 1:
        for id in range(0, len(event['detail']['requestParameters']['instancesSet']['items'])):
            v = event['detail']['requestParameters']['instancesSet']['items'][id]['instanceId']
            instance_id.append(v)
        body = f'Hi Team, \n\nPlease be informed that multiple instances got {msg_status[EventName]} simultaneously.\n\nEventID = {EventID}, \nAccount = {Account}, \nTimestamp = {Timestamp}, \nRegion = {Region}, \nInstanceID = {instance_id}, \nEventName = {EventName}, \nSourceIP = {SourceIP}, \nInitiatedBy = {InitiatedBy} \n\nRegards,\nCloud Team'
        sns_client = boto3.client('sns')
        snsarn = os.environ['snsarn']
        res = sns_client.publish(
            TopicArn = snsarn,
            Subject = f'Alert-Multiple Instances are {msg_status[EventName]}',
            Message = str(body)
            )
    elif len(InstanceID) == 1:
        instance_id = InstanceID[0]['instanceId']
        body = f'Hi Team, \n\nThis is to inform you that EC2 instance with {instance_id} is {msg_status[EventName]}.Please find below information. \n\nEventID = {EventID}, \nAccount = {Account}, \nTimestamp = {Timestamp}, \nRegion = {Region}, \nInstanceID = {instance_id}, \nEventName = {EventName}, \nSourceIP = {SourceIP}, \nInitiatedBy = {InitiatedBy} \n\nRegards,\nCloud Team'

        sns_client = boto3.client('sns')
        snsarn = os.environ['snsarn']
        res = sns_client.publish(
            TopicArn = snsarn,
            Subject = f'Alert - {instance_id} is {msg_status[EventName]}',
            Message = str(body)
            )

1. Here, SNS topic name is stored in Environment Variable section in lambda.

Note: Here, we didn't add SNS topic in Destination section because there will be below unwanted stuffs occurred if added in destination section.

• SNS will send email as per python code with customizations you did. (Expected Behavior)
• SNS will also send email with complete lambda execution details like EventID, Event Timestamps, Payload etc. which is already part of customized email, no need to get separate email. Hence adding sns topic as destination has been omitted.

Summary:

This post will guide you to configure customize email notification settings using EventBridge and Lambda with the help of python scripting. This same approach can be used in other cases as well like getting notified when ec2 metrics(cpu/mem/disk) is in ALARM state, so basically EventBridge will be triggered once alarm status will be switched from OK to ALARM.

From configurational changes perspective this will help a lot to identify details in deep with AWS Config service. For more details, please checkout

docs.aws.amazon.com

Please follow me for more contents.

Thanks for Reading!!

Let's connect https://www.linkedin.com/in/anirban-das-507816169/