Automated Alerts for High CPU Utilization: Real-Time Email Notifications with Instance Details

piyush desai — Mon, 02 Dec 2024 06:03:01 +0000

Create Instance:
Create an Instance.
Step 1: Create Lambda

Go to the AWS Lambda console.
Create a new function:
Click Create function.
Choose Author from scratch.
Provide a name (e.g., SendAlarmNotification).
Select a runtime (e.g., Python 3.x).
Set the permissions (create a new role with basic Lambda permissions).
Click on Create Function.
Add code to send a notification: Here’s an example code snippet that you can use in your Lambda function:

import json
import boto3
import datetime

# Initialize the Boto3 clients
cloudwatch_client = boto3.client('cloudwatch', region_name='ap-south-1')
sns_client = boto3.client('sns', region_name='ap-south-1')

# Constants
INSTANCE_ID = 'i-06b205998588d9c99'  # Your EC2 instance ID
SNS_TOPIC_ARN = 'arn:aws:sns:ap-south-1:533267114782:AlarmNotificationTopic'  # Your SNS topic ARN

def lambda_handler(event, context):
    # Get CPU utilization
    cpu_utilization = get_cpu_utilization(INSTANCE_ID)

    # Construct the subject and message body
    subject = f"Your Server's CPU is {cpu_utilization}%. Please check"
    message = f"""
    Hi,

    The server CPU usage is high. Please check.

    Instance ID: {INSTANCE_ID}
    CPU Utilization: {cpu_utilization}%
    """

    # Publish the message to SNS
    response = sns_client.publish(
        TopicArn=SNS_TOPIC_ARN,
        Message=message.strip(),
        Subject=subject
    )

    return {
        'statusCode': 200,
        'body': json.dumps('CPU Utilization notification sent to SNS!')
    }

def get_cpu_utilization(instance_id):
    # Get the CPU utilization metric from CloudWatch
    response = cloudwatch_client.get_metric_statistics(
        Namespace='AWS/EC2',
        MetricName='CPUUtilization',
        Dimensions=[{
            'Name': 'InstanceId',
            'Value': instance_id
        }],
        StartTime=datetime.datetime.utcnow() - datetime.timedelta(minutes=10),
        EndTime=datetime.datetime.utcnow(),
        Period=300,
        Statistics=['Average']
    )

    # Get the average CPU utilization and convert to integer
    if len(response['Datapoints']) > 0:
        return int(round(response['Datapoints'][0]['Average']))  # Round and convert to int
    return 0  # Return 0 if no data points are available

Step 2: Create SNS and Subscribe it

Go to the Amazon SNS Console:
Navigate to the Amazon SNS Console.
Create a Topic:
Click on Topics in the left sidebar.
Click on Create topic.
Select Standard as the topic type.
Enter a name for the topic (e.g., CpuUtilizationAlerts).
Click Create topic.
Note the Topic ARN for later use.
Subscribe Your Email to the SNS Topic
Open the Topic:
Click on the topic you just created (e.g., CpuUtilizationAlerts).
Create a Subscription:
Click on Create subscription.
In the Protocol dropdown, select Email.
In the Endpoint field, enter the email address where you want to receive notifications.
Click Create subscription.
Confirm the subscription.
Check your email for a subscription confirmation from Amazon SNS and confirm the subscription by clicking the link provided in the email.

Step 3: Add Cloud watch Alarm:

Go to the CloudWatch Console:
Navigate to the Amazon CloudWatch Console.
Create an Alarm:
Click on Alarms in the left sidebar.
Click Create Alarm.
Choose Select metric.
Navigate to EC2 > Per-Instance Metrics, and select CPUUtilization for your instance.
Click Select metric.
Set the Alarm Conditions:
Set the threshold (e.g., Greater than 50% for 1 consecutive period of 5 minutes).
Click Next.
Configure Actions:
Under Configure actions, select Lambda function.
Choose your Lambda function (CPUMonitorFunction) from the dropdown.
Review and Create the Alarm:
Click Next, review your settings, and click Create alarm.

Step 4: Lambda Permission Settings

In the AWS Lambda Console, navigate to your function’s configuration.
Under the Permissions section, find Resource-based policy statements.
Select Add permissions and choose AWS service.
Select Other from the dropdown list.
Add a unique statement ID for tracking.
For Principal, add lambda.alarms.cloudwatch.amazonaws.com.
Under Source ARN, add the ARN of your CloudWatch alarm.
In the Action dropdown, select lambda:InvokeFunction.
Click Save to apply the changes.

Step 5: Increase the CUP utilization in Instance using Python script

In AWS instance, Create a python file : cpu-utilization.py
Add the below code to cpu-utilization.py

import multiprocessing
import time

def cpu_stress():
    # This function performs an infinite loop to stress the CPU
    while True:
        pass  # Infinite loop to simulate CPU load

if __name__ == "__main__":
    # Number of CPU cores to stress
    num_cores = multiprocessing.cpu_count()

    print(f"Starting CPU stress on {num_cores} cores...")

    # Create one process per CPU core
    processes = []
    for i in range(num_cores):
        p = multiprocessing.Process(target=cpu_stress)
        processes.append(p)
        p.start()

    # Let the stress run for a certain amount of time
    time_to_run = 300  # Run for 5 minutes (300 seconds)
    time.sleep(time_to_run)

    # Stop all processes after the specified time
    for p in processes:
        p.terminate()

    print("CPU stress test completed.")

Now run the File using python3 cpu-utilization.py and you will receive the notification of Ec2 cpu utilization and ram at the email.

You may further edit the subject of email using Lamda.

Amazon S3 Increases Bucket Limit to 10,000 — Scalability Just Got Easier!

piyush desai — Fri, 15 Nov 2024 05:09:19 +0000

Amazon Web Services (AWS) recently introduced a significant enhancement to Amazon S3: the default bucket quota per AWS account has increased from 100 to 10,000. This substantial change allows organizations to use unique buckets for individual datasets, streamlining management and providing access to advanced storage features. For those managing even more data, AWS now offers the option to scale up to 1 million buckets by requesting a quota increase through Service Quotas.

What This New Default Bucket Quota Means for S3 Storage Architecture

This update is particularly valuable for companies aiming to scale their data storage strategies. Let’s explore the key benefits and best practices that come with this new bucket limit.

Key Benefits of Amazon S3’s New Default Bucket Limit

1. Better Data Organization and Management
• Enhanced Organization: Create dedicated buckets for various projects, departments, or teams.
• Granular Storage Configurations: Enable further customization for storage access and organization.

2. Improved Security with Default Encryption and Policies
• Enhanced Data Security: Use separate buckets to apply default encryption and customize security policies for individual datasets.
• Compliance with Data Privacy Regulations: Tailored security settings help meet regulatory requirements.

3. Streamlined Data Replication
• Easier S3 Replication: Simplify disaster recovery and data residency compliance with dedicated buckets for specific datasets.
• Reduced Configuration Complexity: Replicate individual datasets without the need for complex configurations.

4. No Action Needed for Default Quota Increase
• Automatic Upgrade: The 10,000-bucket limit is automatically applied to all AWS accounts.
• Immediate Access: AWS users can benefit from this increase without taking any additional steps.

5. Scalable to Meet Future Needs
• Flexibility for Growth: Request an increase through AWS Service Quotas to
• Support for Massive Datasets: Enhanced scalability makes S3 a powerful option for large-scale data storage.

6. Cost Efficiency for Big Data Needs
• Cost-Free for Up to 2,000 Buckets: Accounts with fewer than 2,000 buckets incur no additional charge.
• Small Monthly Fee Beyond 2,000 Buckets: A minimal fee applies for accounts with more than 2,000 buckets, keeping large-scale storage affordable.

DEV Community: piyush desai

Automated Alerts for High CPU Utilization: Real-Time Email Notifications with Instance Details

Amazon S3 Increases Bucket Limit to 10,000 — Scalability Just Got Easier!