DEV Community


Configuring S3 bucket with lambda function, lambda function with SQS

A Cloud Computing/DevOps enthusiast who is eager to learn new things in DevOps and help society with it.
・4 min read

Hola Developers, Cloud Enthusiasts!!

The most important reason to write this is to help others, because this is not available anywhere if you find it.

In my first summer internship we are working on a Document management system designed using serverless architecture. In this project I was assigned this task of sending notifications of S3 bucket to SQS using a lambda function.

So let's get started.

Step1: First make a bucket no need to do anything else in that just simply make a bucket with standard storage.

Step2: Now go to Lambda function in services, and click on create a function. Following picture will make you understand.

Creating lambda function

Step 3: Create a lambda function by clicking on Create a function button.

Step 4: It will show you the following page

Alt Text

In this select Author from scratch than move down where you will see the following

Alt Text

Here give your function a name of your choice and then select a language under Runtime , preferably python.

Then under Permissions Choose or create an execution role. Here you will get 3 options as shown below.

Alt Text

Step 5: For that go to IAM dashboard shown in the following picture.

Alt Text

Here now got to Roles under Access management after that create a role as shown below.

Alt Text

In create role it will ask you to choose a use case in that select lambda.

Alt Text

Than click next which will for permissions.

Allow for the following policies for the services to be in a sync.

Alt Text

SQS Full access to write the message in the message queue.

S3 Full access to get the notification in lambda which will give to SQS.

CloudWatch Full access to watch the logs, by default logs of lambda are shown in CloudWatch but when these are integrated it is necessary to attach this policy.

AWSLambdaSQSQueueExecutionRole to send data from lambda to SQS.

After assigning these policies go next for tags they are optional, then go for review. Then give the name of this role as shown below.

Alt Text

After this click Create role and the role is created.

Step 6: Go back to the lambda dashboard and select Select an existing role which was made in IAM.

Alt Text

As shown below after selecting Selecting an existing role a dropdown box will appear and select the role which was created in IAM.

After that Create function.

Step 7 : It will show like this as shown below, function name under Configuration under Designer dropdown it will show these configuration.

Alt Text

Now add the trigger by clicking on the button on the left, can be shown in the above picture.

In the Add trigger give S3 and choose the bucket name which was created and under Event type dropdown select put, which means whenever an object is uploaded to the object it will trigger the lambda function.

Alt Text

Then Add these triggers to the lambda function. After adding these trigger to the function dashboard will look like this.

Alt Text

Step 8: Add the destination which is SQS.

For that click on Add destination, in which configure the destination.

Select source as Asynchronous invocation.

Select condition on which condition lambda will throw it to destination, so select On success.

After that select Destination type, under dropdown select SQS queue.

Alt Text

After that select destination, for that navigate to SQS queue dashboard.

Make a queue for receiving messages from lambda.

Click on Create New queue.

Alt Text

After that give the name of queue, and keep the queue type as Standard queue.

Alt Text

Scroll down and click on Quick Create-queue and the queue is ready for receiving messages.

Alt Text

Go back to lambda dashboard and in the Destination Selection select the name of the queue which was created. As shown below.

Alt Text

Now Save the destination and the lambda dashboard will look like these.

Alt Text

Now select the lambda function and scroll down where there will a Function code window.

Alt Text

Any changes can be made in the return part of the code.Now Save and Test the function.

Below it will show the execution result as shown below.

Alt Text

Step 9: Now add a file to the S3 bucket and check the SQS queue for messages.

For doing that upload a file to S3 bucket and navigate to the SQS dashboard and refresh it will show you the file uploaded and time stamp, file name, date of which the file was uploaded.

Go in Queue Actions dropdown and select View/Delete messages and start polling.

Alt Text

After that it will show you the message as shown below.

Alt Text

This is how you configure S3-Lambda-SQS.

Discussion (0)