How to use SQS to communicate between NodeJs microservices

Amazon SQS message is a distributed message queuing service. We can use it to communicate between distributed services. We can send and receive messages from the queue. The producer sends the message to the queue, and the consumer read the message from the queue. I will show you how to use the Amazon SQS with nodejs. Amazon SQS has two queues, standard and FIFO. I used the standard queue.

SQS standard queue can send the same message multiple times. So there is the possibility of duplicate messages in the Amazon SQS. Before starting the SQS setup and coding, there are a few terminologies to understand.

• Visibility timeout is the amount of time after that another consumer receives the message. So if one consumer receives the message right now, then the next consumer receives the same message after the visibility time from the current time.

• Receive message wait time is the time that Amazon SQS waits for the message to become available after it is added to the queue.

• Delivery delay is the time SQS wait before adding the message to the queue.

• Retention period is the amount of time that Amazon SQS retain a message in the queue.

Let's start with creating an SQS queue in AWS. Open the AWS console and search SQS. I am using Mumbai(ap-south-1) region.

Click on create the queue.

SQS setup requires name, configuration, and access policy. These three are mandatory, and the rest are optional. Dead-letter queue helps to handle the failure, but it is out of the scope of this blog. We are going to use the standard queue. Keep the access policy basic.

Click on create queue button at the bottom of the screen.

On the next screen, you will get the queue URL in the details section that we will need later.

Once the queue setup is complete, we can move to the coding. We require a secret access key and access key of an IAM user that has the read, write, delete SQS message permission and permission to create log stream, log group and add log event. Log permission is required to save the logs in cloudwatch that we can use later for debugging. Install aws-sdk npm package.
npm install aws-sdk
Now we have to initialize sqs to send or receive the message. QueueUrl is the URL that we received after the queue creation.

const AWS = require('aws-sdk');
const sqs = new AWS.SQS({
  accessKeyId: 'IAM user access key',
  secretAccessKey: 'IAM user secret key',
  region: 'region where you created queue(like ap-south-1)',
  apiVersion: '2012-11-05',
});
//parameter to send message
const paramsSendMessage = {
  MessageBody: JSON.stringify({
    type: 'event-live-status',
  }),
  QueueUrl: 'url of queue we created',
};

//param to receive message
const paramsReceiveMessage = {
  QueueUrl: 'url of queue we created',
};

SQS sendMessage function uses to send the message on the queue.

  sqs.sendMessage(params, (err, data) => {
    if (err) {
      console.log('Error', err);
    } else {
      console.log('Successfully added message', data.MessageId);
    }
  });

AWS SQS use polling to read the message from the queue. If ReceiveMessageWaitTimeSeconds is set to 0 in the configuration or we have added the WaitTimeSeconds to 0, or there is no WaitTimeSeconds property in the parameter

//param to receive message
const paramsReceiveMessage = {
  QueueUrl: 'url of queue we created',
  WaitTimeSeconds: 0,
};

  sqs.receiveMessage(params, (err, data) => {
    if (err) {
      console.log(err);
    } else if (!data.Messages) {
      console.log('no message found');
    } else {
/*
we sent the message as json string so json parse the message body
*/
      const messageBody = JSON.parse(data.Messages[0].Body);

    }
  });

If the nodejs server is running then, it keep try to read the message from the queue. It will use either short polling or long polling based on configuration.

Note: Make sure the IAM user has the permission to write, read, delete message from SQS queue, queue url is correct. There is very high chance that we receive the duplicae message from the queue. If we delete the message immediately when we receive it then we can avoid the duplicate message.

We can also use AWS lambda function to read message from queue and do action based on message. I will include it in my next blog.

Comments

[mobisoftinfotech]

Great article on Differences between SQS, SNS, and Amazon MQ! Your explanation of how SQS decouples distributed systems and SNS’s pub-sub model really clarified the distinctions between the services.

While researching, I found this resource on AWS SQS and SNS integration with NestJS: mobisoftinfotech.com/resources/blo.... which dives into event-driven architecture with TypeScript

Thanks for the insightful guide! I found it really helpful for understanding SNS and SQS integration in scalable systems.