Tanushree Aggarwal for AWS Community Builders

Posted on Sep 13, 2023

Scheduling EC2 start and stop using Eventbridge and Lambda (part 1 of 2)

#aws #serverless #lambda #cloud

Introduction:

By now we understand very well that:
Cloud == Pay for what you use
One of the biggest factor that attracts users to the cloud, is the ability to provision resources within minutes. You are able to setup a virtual server in the Cloud, with just a few clicks, in a matter of minutes.

Use Case:

Your organization's development team works from 08:00AM to 05:00PM Monday to Friday. The development team's Cloud Virtual Machines (Amazon EC2 instances) are running idle in their out of office hours. It is your responsibility to ensure that the dev environment stops at 05:30PM everyday, and resumes the next morning. The environment should remain down on the weekends. Automate this task.

Services :

Amazon Eventbridge (formerly called Amazon CloudWatch Events):

EventBridge is a serverless service that uses events to connect application components together, making it easier for you to build scalable event-driven applications. Event-driven architecture is a style of building loosely-coupled software systems that work together by emitting and responding to events. Event-driven architecture can help you boost agility and build reliable, scalable applications.

AWS Lambda:

AWS Lambda is a serverless, event-driven compute service that lets you run code for virtually any type of application or backend service without provisioning or managing servers. You can trigger Lambda from over 200 AWS services and software as a service (SaaS) applications, and only pay for what you use.

AWS IAM:

AWS Identity and Access Management (IAM), allows us to specify who or what can access services and resources in AWS, centrally manage fine-grained permissions, and analyze access to refine permissions across AWS.

Amazon EC2:

Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides secure, resizable compute capacity in the cloud.

Goal:

Setup a scheduler using the Amazon Eventbridge service which will trigger a Lambda function.
Lambda function will temporarily assume the defined IAM role. This IAM role will have an IAM policy associated with it. The IAM Policy is what grants the Role permissions to access our EC2 instances.
The Lambda function will stop/start our Amazon EC2 instances as per the schedule we define in Eventbridge.

This is part-1 of the blog, where we will :
i. Create the IAM Policy and associate it with an IAM Role.
ii. Create our Lambda functions: one to stop the EC2 instances and another to start the EC2 instance.
iii. Test the Lambda functions by triggering them manually, to check if they have the desired effect on our EC2 instances

Part-2 of the blog will tie everything together, where we will be configuring the Amazon Eventbridge rules and defining the Cron entry that will trigger our Lambda functions to stop/start our EC2 instances.

Pre-requisites :

AWS IAM account (do not use root account) having:
admin privileges
access to AWS Management Console
Understanding of AWS Identity based policies In case you are not familiar with IAM Policies or need a quick refresher, please refer my blog
Atleast one EC2 instance created in the account (this is required for testing purpose)

Cost :

None (if you have an AWS free-tier eligible account)

Implementation:

Let's begin! Login to the AWS Management Console as IAM admin user

1. Create IAM policy and IAM role

1.1 From the AWS Management Console navigate to IAM

1.2 From the left navigation panel, select Policies and then click the Create Policy button

1.3 Select JSON

1.4 Paste the below policy into the policy editor (delete all the pre-filled content) and click Next

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:*:*:*",
      "Condition": {
                "StringEquals": {
                    "aws:RequestedRegion": "eu-central-1"
                }
            }
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeInstances",
        "ec2:DescribeRegions",
        "ec2:StartInstances",
        "ec2:StopInstances"
      ],
      "Resource": "*",
      "Condition": {
                "StringEquals": {
                    "aws:RequestedRegion": "eu-central-1"
                }
            }
    }
  ]
}

1.5 Give the policy a meaningful name and description. Review the permissions and then click Click Policy

1.6 From the left navigation panel, select Roles and then click the Create Role button

1.7 Select AWS Service as the Trusted entity type

1.8 From the Service or use case dropdown list, select Lambda

1.9 Click Next to move to the next screen

1.10 From the policy dropdown list, select the policy we created in the previous step and click Next

1.11 Give the role a name and description

1.12 Verify the trusted entity (should be Lambda) and the associated customer managed IAM Policy. Click Create Role

Note the maximum duration this role can be assumed. Since we did not specify a duration, the default duration of 1 hour was assigned

2. Create Lambda Function to stop EC2 instances

2.1 From the AWS Management Console navigate to Lambda

Make sure you are in the desired AWS region. Change the region in case not.

2.2 Click Create Function

2.3 Select Author from scratch

2.4 Give the function a name and select the desired runtime.

In this demo we will be using Python 3.11 as the runtime.

2.5 In the Permissions tab, change the default execution role to Use an existing role.
From the dropdown list, select the lambda assume role we created in the previous step.

2.6 Click Create Function

Good job keeping up so far! You are making good progress!

2.7 Scroll down a bit to reach the Code section. You will notice a pre-filled Code source section.

2.8 Delete the pre-filled contents and paste the below code in its place:

import boto3

region = 'eu-central-1'


def lambda_handler(event, context):
    ec2_resource = boto3.resource("ec2", region_name=region)

    # Apply filter to check for EC2 instances in 'running' state
    instances = ec2_resource.instances.filter(Filters=[{'Name': 'instance-state-name', 'Values': ['running']}])

    print("EC2 region is: ", region)
    print("instances= ", instances)

    # Count running instances
    count = len(list(instances))
    print(f"{count} instances running")

    # Stopping EC2 instances        
    for instance in instances:
        instance.stop()
        print(instance, ": Stopping!!! .")

2.9 Click Deploy

2.10 In the Configuration tab, select General configuration and click Edit

2.11 Change the Timeout value to 10 seconds and click Save

Note: The timeout value may differ for each use case, so choose one according to your requirement

2.12 Click Test to test if our lambda function is working as expected
Configure the test event.

Just an event name will suffice. Our Lambda does not require any user input.

Click Save

Click Test to execute the newly created Lambda function.
Review the output logged by the stop EC2 Lambda.

Important: For this to work you need to have atleast one EC2 instance running in eu-central-1 region

Two EC2 instances were found to be in running state in the region defined within our Lambda function eu-central-1
Our Lambda attempts stopping each of these one-by-one (as coded in our for loop)

Notice the time duration of our Lambda execution - it is roughly 4 seconds. If we left the timeout value as default 3 seconds, our Lambda function would most likely have timed-out before completing the execution.

2.13 Go to the EC2 dashboard in the AWS Management Console (in the same region defined in your Lambda code) and check if the instances really stopped.

It worked!

3. Create the Start EC2 instance Lambda Function

3.1 Repeat steps to create another Lambda function and this time use the below Python code, which will start our EC2 instances.

import boto3

region = 'eu-central-1'


def lambda_handler(event, context):
    ec2_resource = boto3.resource("ec2", region_name=region)

    # Apply filter to check for EC2 instances in 'stopped' state
    instances = ec2_resource.instances.filter(Filters=[{'Name': 'instance-state-name', 'Values': ['stopped']}])

    print("EC2 region is: ", region)
    print("instances= ", instances)

    # Count running instances
    count = len(list(instances))
    print(f"{count} instances in stopped state")

    # Starting EC2 instances        
    for instance in instances:
        instance.start()
        print(instance, ": Starting!!! .")

3.2 Test the Lambda

Review the execution result of the Lambda function.

2 EC2 instances were found in running state in the defined region eu-central-1
Both are stopped one after the other, with their respective EC2 instance IDs logged.
The execution took ~4 seconds.

3.3 Verify if the EC2 instances started in the EC2 dashboard

Pheewww!!
That was a lot of hand's on work!
I hope this gave you good exposure on how to navigate the AWS Management Console and helped grow your understanding on how multiple services can be used together to build a simple solution !

Conclusion:

In this blog we discussed our use case and associated architecture. We created an IAM execution role which allows our Lambda functions to access our EC2 instances. We even created the EC2 stop and EC2 start Lambda functions and tested them in isolation.

Stay tuned for the part-2 of this blog, where we complete our architecture by configuring Eventbridge to trigger our Lambdas.

Bonus:

Now that you have understood how to create the Lambda functions, feel free to play around with these and modify as per your desired use case. In our demo we restricted the EC2 task to a single region, this can easily be extended to include multiple AWS regions.
You can even enhance the instance filters on the available EC2 instance states!
Additionally, you make the code more dynamic, by filtering the EC2 instances based on defined Tags!

Filter based on instance state and defined tags:

 instances = ec2_resource.instances.filter(Filters=[{'Name': 'instance-state-name', 'Values': ['running']},{'Name': 'tag:environment','Values':['qa']}])