Challenge: Display EC2 instances from all regions using Python and Boto3

This was an idea from the AWS Certified Global Community to create challenges by breaking a certain part of code, so the users can learn by fixing the code.

I have written this second challenge for the AWS Certified Global Community and have also used it to mentor some of my mentees.
I thought that it might be a good idea to share this with the broader community as well.

Boto3 is the name of the Python SDK for AWS. It allows you to directly create, update, and delete AWS resources from your Python scripts.

I agree that we have EC2 Global View, which shows what we are trying to achieve. However this is more of a programming challenge on how to code using Python and Boto3.

Let's start the challenge!

Pre-Requisites:

• Lambda with Python 3.9
• Lambda Execution role with access to EC2 and CloudWatch access (IAM)
• It is preferable to have at least one EC2 instance in any of the regions (not mandatory though)

---

Step 1. Create an IAM Policy:

You could either let Lambda create an execution role and add the AmazonEC2ReadOnlyAccess policy or you could create the following IAM policy and a role.

Note: Replace [Region], [LambdaFunctionName] and [AccountID] in the Policy

IAM Policy with the following access:

AWSLambdaBasicExecution

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "logs:CreateLogGroup",
            "Resource": "arn:aws:logs:[Region]:[AccountID]:*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "*"
        }
    ]
}

AmazonEC2ReadOnlyAccess

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "ec2:Describe*",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "elasticloadbalancing:Describe*",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "cloudwatch:ListMetrics",
                "cloudwatch:GetMetricStatistics",
                "cloudwatch:Describe*"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "autoscaling:Describe*",
            "Resource": "*"
        }
    ]
}

---

Step 2. Create a role for Lambda and attach the policy created in Step 1

Trusted entities:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "lambda.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

---

Step 3. Broken Code: Create a Lambda function with the following code and execute the function.

Note: Please increase the Lambda Function Timeout under General Configuration from 3 sec to 2 min or more as required.

import json

ec2list = []

def lambda_handler(event, context):

    # Get list of regions
    ec2 = boto3.client('ec2')
    regions = ec2.describe_regions().get('Regions',[] )

    # Iterate over regions
    for region in regions:

        print ("* Checking region  --   %s " % region['RegionName'])
        reg=region['RegionName']

        client = boto3.client('ec2', region_name=reg)
        response = client.describe_instances()

        for reservation in response["Instances"]:
            for instance in reservation["Instances"]:
                print ("  ---- Instance %s in %s" % (instance['InstanceId'], region['RegionName']))
                ec2list.append(instance['InstanceId'])

    return {
        "statusCode": 200,
        "body": ec2list
    }   

---

Note: DO NOT forget to deploy the Lambda function every time you make a change.

First Error: We get the following error when we execute the code❓

Response
{
  "errorMessage": "name 'boto3' is not defined",
  "errorType": "NameError",
  "requestId": "6926f216-e0e8-4f25-bb16-761ada972e3f",
  "stackTrace": [
    "  File \"/var/task/lambda_function.py\", line 8, in lambda_handler\n    ec2 = boto3.client('ec2')\n"
  ]
}

Hint 1:
The first one is easy to solve by executing the code in Lambda, the hint appears in the error message.

---

Second Error: We get the following error after fixing the first one❓

Response
{
  "errorMessage": "'Instances'",
  "errorType": "KeyError",
  "requestId": "c78a6211-1d73-4d72-b83a-4c51bfd4e181",
  "stackTrace": [
    "  File \"/var/task/lambda_function.py\", line 21, in lambda_handler\n    for reservation in response[\"Instances\"]:\n"
  ]
}

Hint 2:

Please refer to the Response Syntax of describe_instances(**kwargs) at Boto3 Documentation

---

Step 4. Final Code: ✅

import json
import boto3

ec2list = []

def lambda_handler(event, context):

    # Get list of regions
    ec2 = boto3.client('ec2')
    regions = ec2.describe_regions().get('Regions',[] )

    # Iterate over regions
    for region in regions:

        print ("* Checking region  --   %s " % region['RegionName'])
        reg=region['RegionName']

        client = boto3.client('ec2', region_name=reg)
        response = client.describe_instances()

        for reservation in response["Reservations"]:
            for instance in reservation["Instances"]:
                print ("  ---- Instance %s in %s" % (instance['InstanceId'], region['RegionName']))
                ec2list.append(instance['InstanceId'])

    return {
        "statusCode": 200,
        "body": ec2list
    }   

Output

{
  "statusCode": 200,
  "body": [
    "I-[instanceid]"
  ]
}

START RequestId: f6a1c381-51e2-43c9-8490-a62b8a6394dd Version: $LATEST
* Checking region  --   eu-north-1
* Checking region  --   ap-south-1
* Checking region  --   eu-west-3
* Checking region  --   eu-west-2
* Checking region  --   eu-west-1
* Checking region  --   ap-northeast-3
* Checking region  --   ap-northeast-2
* Checking region  --   ap-northeast-1
* Checking region  --   sa-east-1
* Checking region  --   ca-central-1
* Checking region  --   ap-southeast-1
* Checking region  --   ap-southeast-2
* Checking region  --   eu-central-1
* Checking region  --   us-east-1
---- Instance [instanceid] in us-east-1
* Checking region  --   us-east-2
* Checking region  --   us-west-1
* Checking region  --   us-west-2
END RequestId: f6a1c381-51e2-43c9-8490-a62b8a6394dd
REPORT RequestId: f6a1c381-51e2-43c9-8490-a62b8a6394dd Duration: 15667.85 ms Billed Duration: 15668 ms Memory Size: 128 MB Max Memory Used: 90 MB Init Duration: 246.67 ms

Note: The actual instance id is replaced with [instanceid] in the output.

So what have I done to break the code

• I removed import boto3 at line no 2
• Replaced response["Reservations"]: with response["Instances"]: at line no 21

Hope you liked the challenge! Please let me know your thoughts in the comments section.

Tip: Please do check out the reference link as you will learn something about global services too with Boto3. 👇

See you next time 👋

---

References:

I learnt this from stackoverflow when I asked a question about Retrieve list of buckets from all regions using boto3

Hope you have also learnt something about retrieving S3 buckets from all regions using Boto3.

---