Hola Developers, Cloud Enthusiasts!!
The most important reason to write this is to help others, because this is not available anywhere if you find it.
In my first summer internship we are working on a Document management system designed using serverless architecture. In this project I was assigned this task of sending notifications of S3 bucket to SQS using a lambda function.
So let's get started.
Step1: First make a bucket no need to do anything else in that just simply make a bucket with standard storage.
Step2: Now go to Lambda function in services, and click on create a function. Following picture will make you understand.
Step 3: Create a lambda function by clicking on Create a function button.
Step 4: It will show you the following page
In this select Author from scratch than move down where you will see the following
Here give your function a name of your choice and then select a language under Runtime , preferably python.
Then under Permissions Choose or create an execution role. Here you will get 3 options as shown below.
Step 5: For that go to IAM dashboard shown in the following picture.
Here now got to Roles under Access management after that create a role as shown below.
In create role it will ask you to choose a use case in that select lambda.
Than click next which will for permissions.
Allow for the following policies for the services to be in a sync.
SQS Full access to write the message in the message queue.
S3 Full access to get the notification in lambda which will give to SQS.
CloudWatch Full access to watch the logs, by default logs of lambda are shown in CloudWatch but when these are integrated it is necessary to attach this policy.
AWSLambdaSQSQueueExecutionRole to send data from lambda to SQS.
After assigning these policies go next for tags they are optional, then go for review. Then give the name of this role as shown below.
After this click Create role and the role is created.
Step 6: Go back to the lambda dashboard and select Select an existing role which was made in IAM.
As shown below after selecting Selecting an existing role a dropdown box will appear and select the role which was created in IAM.
After that Create function.
Step 7 : It will show like this as shown below, function name under Configuration under Designer dropdown it will show these configuration.
Now add the trigger by clicking on the button on the left, can be shown in the above picture.
In the Add trigger give S3 and choose the bucket name which was created and under Event type dropdown select put, which means whenever an object is uploaded to the object it will trigger the lambda function.
Then Add these triggers to the lambda function. After adding these trigger to the function dashboard will look like this.
Step 8: Add the destination which is SQS.
For that click on Add destination, in which configure the destination.
Select source as Asynchronous invocation.
Select condition on which condition lambda will throw it to destination, so select On success.
After that select Destination type, under dropdown select SQS queue.
After that select destination, for that navigate to SQS queue dashboard.
Make a queue for receiving messages from lambda.
Click on Create New queue.
After that give the name of queue, and keep the queue type as Standard queue.
Scroll down and click on Quick Create-queue and the queue is ready for receiving messages.
Go back to lambda dashboard and in the Destination Selection select the name of the queue which was created. As shown below.
Now Save the destination and the lambda dashboard will look like these.
Now select the lambda function and scroll down where there will a Function code window.
Any changes can be made in the return part of the code.Now Save and Test the function.
Below it will show the execution result as shown below.
Step 9: Now add a file to the S3 bucket and check the SQS queue for messages.
For doing that upload a file to S3 bucket and navigate to the SQS dashboard and refresh it will show you the file uploaded and time stamp, file name, date of which the file was uploaded.
Go in Queue Actions dropdown and select View/Delete messages and start polling.
After that it will show you the message as shown below.
This is how you configure S3-Lambda-SQS.
Top comments (0)