AWS: Lambda - copy EC2 tags to its EBS, part 2 - create a Lambda function

AWS: Lambda — copy EC2 tags to its EBS, part 2 — create a Lambda function

let’s proceed in our journey of the AWS Lambda function, which will copy an EC2’s AWS Tags to all EBS volumes, attached to it.

In the first part, AWS: Lambda — copy EC2 tags to its EBS, part 1 — Python and boto3, we wrote a Python script that can get all EC2 instances in an AWS Region, then for every EC2 it grabs its EBS volumes, and then will copy all AWS Tags from the EC2 to all its EBS plus will add one additional.

In this part, we will create an AWS Lambda function that will be triggered with the AWS CloudWatch Events.

Contents

• Creating an AWS Lambda function
• Adding an Amazon EventBridge (CloudWatch Events) trigger
• AWS Lambda: UnauthorizedOperation
• Parsing a Lambda event

Creating an AWS Lambda function

Go to the AWS Lambda, create a new function from the ”Author from scratch”, with the Runtime set as Python 3.8:

In the Execution role leave the “Create a new role” — later, we will add additional permissions to it with an IAM Policy.

Paste the script’s code to the function, at the end of the code update the lambda_hanlder() execution and in its arguments instead of the "0, 0" from the previous post, set the event, context:

To check the content of the event object, which we will use later to get an EC2 ID, add its output to the function's log.

Add import json and print("CONTEXT: " + json.dumps(event)):

Also, for the ec2 = boto3.resource() call remove AWS keys, and leave the only type of the resource - "ec2":

Do not forget to press the Deploy button to apply your changes to the AWS Lambda.

Adding an Amazon EventBridge (CloudWatch Events) trigger

Next, we need to run this function each time when a new EC2 is launched in a region.

Here, we can use the Amazon EventBridge (former CloudWatch Events). Go to the CloudWatch Events > Rules, click on the Create rule:

In the Event Pattern choose the Service Name == EC2, in the Event Type choose the “EC2 Instance State-change Notification”, and in the Specific state(s) choose running:

On the right side in the Add target choose the AWS Lambda function that we’ve created above:

At the end of the page, in the Show sample event(s), we can see an example of the event object that will be passed to the function to get an EC2 ID:

Save the new rule:

Check if it was added to the Lambda function as a trigger:

And let’s check how this will work.

In the CloudWatch Rules open the Rule’s monitoring:

Spin up a new EC2, for example by triggering an AutoScale Group:

On the CloudWatch graph we can see, that the Rule was triggered:

Check the Lambda function’s logs:

AWS Lambda: UnauthorizedOperation

In the logs, we are interested in two records.

The first one  -  is the event content, that was saved to the log from the print("CONTEXT: " + json.dumps(event)):

And the second one is the “ClientError: An error occurred (UnauthorizedOperation) when calling the DescribeInstances operation: You are not authorized to perform this operation” error:

The issue here is obvious enough: our Lambda function now is using an IAM Role, that was created during the function’s creation, and it has no permissions for API calls for the EC2 actions:

Go to the AWS IAM, find the Role, click on the Attach policies:

Create a new policy:

Describe EC2 permissions here:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeInstances",
                "ec2:DeleteTags",
                "ec2:CreateTags",
                "ec2:DescribeVolumes"
            ],
            "Resource": "*"
        }
    ]
}

Save it:

Go back to the Lambda function’s IAM Role, and add a new Policy:

Repeat a new EC2 launch and check the function’s logs again:

Yay! “ It works! ” ©

Next, need to update the code to get the instance-id from the event object.

Parsing a Lambda event

We’ve already seen an example of the event when we've created the CloudWatch Rule, and we know that it will be passed to the lambda_handler() function as a python-dictionary with a set of keys:

{
    "version": "0",
    "id": "a99597e5-90a5-5ac3-3aba-da3ecd51452a",
    "detail-type": "EC2 Instance State-change Notification",
    "source": "aws.ec2",
    "account": "534***385",
    "time": "2021-10-11T09:02:09Z",
    "region": "eu-west-3",
    "resources": [
        "arn:aws:ec2:eu-west-3:534***385:instance/i-0cc24729109ba61e5"
    ],
    "detail": {
        "instance-id": "i-0cc24729109ba61e5",
        "state": "running"
    }
}

And from here we need to grab a value of the detail.instance-id element.

Add a new variable to the script, let’s call it instance_id, and it will keep a value from the event["detail"]["instance-id"], and then by using this ID, we are creating a new object of the ec2.Instance class, the ec2.Instance(instance_id):

Instead of running a new EC2 manually, we can use the Test for our Lambda by passing an event object to it.

Create a new test event:

Add a data similar to that we will get from the CloudWatch:

{
    "version": "0",
    "id": "a99597e5-90a5-5ac3-3aba-da3ecd51452a",
    "detail-type": "EC2 Instance State-change Notification",
    "source": "aws.ec2",
    "account": "534***385",
    "time": "2021-10-11T09:02:09Z",
    "region": "eu-west-3",
    "resources": [
        "arn:aws:ec2:eu-west-3:534***385:instance/i-0cc24729109ba61e5"
    ],
    "detail": {
        "instance-id": "i-0cc24729109ba61e5",
        "state": "running"
    }
}

Run the test:

“It works!” _ ©_

And now let’s spin up a common EC2 to check the whole scheme including CloudWatch Event, Lambda’s trigger, and its execution results.

Trigger an AutoScaling Group:

A new EC2 i-051f2e1f1bc8d332b was created, check Lambda’s logs:

Find an EBS of this EC2:

And check its Tags:

And the function’s code now is the next:

#!/usr/bin/env python

import os
import json
import boto3

def lambda_handler(event, context):

    ec2 = boto3.resource('ec2')
    instance_id = event["detail"]["instance-id"]
    instance = ec2.Instance(instance_id)

    print("[DEBUG] EC2\n\t\tID: " + str(instance))
    print("\tEBS")

    for vol in instance.volumes.all():

        vol_id = str(vol)
        print("VOLUME: " + str(vol))

        device_id = "ec2.vol.Device('" + str(vol.attachments[0]['Device']) + "')"
        print("\t\tID: " + vol_id + "\n\t\tDev: " + device_id)

        role_tag = vol.create_tags(Tags=set_role_tag(vol))
        ec2_tags = vol.create_tags(Tags=copy_ec2_tags(instance))
        print("\t\tTags set:\n\t\t\t" + str(role_tag) + "\n\t\t\t" + str(ec2_tags) + "\n")

def is_pvc(vol): 

    try:
        for tag in vol.tags:
            if tag['Key'] == 'kubernetes.io/created-for/pvc/name':
                return True
                break
    except TypeError:
            return False

def set_role_tag(vol):

    device = vol.attachments[0]['Device']
    tags_list = []
    values = {}

    if is_pvc(vol):
        values['Key'] = "Role"
        values['Value'] = "PvcDisk"
        tags_list.append(values)
    elif device == "/dev/xvda":
        values['Key'] = "Role"
        values['Value'] = "RootDisk"
        tags_list.append(values)
    else:   
        values['Key'] = "Role"
        values['Value'] = "DataDisk"
        tags_list.append(values)

    return tags_list


def copy_ec2_tags(instance):

    tags_list = []
    values = {} 

    for instance_tag in instance.tags:

        if instance_tag['Key'] == 'Env':
            tags_list.append(instance_tag)
        elif instance_tag['Key'] == 'Tier':
            tags_list.append(instance_tag)
        elif instance_tag['Key'] == 'DataClass':
            tags_list.append(instance_tag)
        elif instance_tag['Key'] == 'JiraTicket':
            tags_list.append(instance_tag)

    return tags_list

if __name__ == " __main__":
    lambda_handler(event, context)

Done.

Originally published at RTFM: Linux, DevOps, and system administration.

---