AWS SQS: Decode the Secret Power Of Short & Long Polling

Discover the dynamics of Short & Long Polling in AWS SQS. When using the ReceiveMessage API call, SQS can be configured to perform short or long Polling, which will retrieve data from distributed components. Short Polling fetches a subset of servers, while long Polling queries all servers for message retrieval. let's understand each Polling mechanism for seamless integration into your application.

What is Short & Long Polling in SQS?

While invoking the ReceiveMessage API call in SQS, the option to configure Short & Long Polling arises.

Background

SQS, a distributed queue, distributes messages across various components in different regions. Short & long polling capitalises on this architecture, enhancing accessibility to fetch data from these distributed components.

Short Polling: This polling method optimise the user experience by sending data seamlessly fast by extracting the message in the subset of the server which stores the message.

Long Polling: On the opposite to short polling, this method search all the servers to fetch the messages in the server, which will take time to fetch all the server even there is no messages in those server.

Consuming Message through Short Polling

We Will Invoke SQS in two scenarios: an empty queue and a queue with data. For short polling, set WaitTimeSeconds to 0.

Code
On the code level, to make the api call to short polling you need to specify the parameter WaitTimeSeconds to 0, and MaxNumberOfMessages to 3 (you can set as much you want).

const aws = require('aws-sdk')
const sqs = new aws.SQS({ region: 'us-east-1' });

const shortPolling = async () => {
    try {
        const params = {
            QueueUrl: 'Queue-URL',
            WaitTimeSeconds: 0,
            MaxNumberOfMessages: 3
        }
        const data = await sqs.receiveMessage(params).promise();
        console.log('[INFO] Received messages from SQS by short polling', JSON.stringify(data));
    } catch (error) {
        console.log('[ERROR] Error while short polling the sqs', error);
        throw error;
    }
};

Scenario1:
When the queue is empty, SQS quickly looks over the server and returns an empty message.

Scenario2:
let assume you have 5 message in your queue and make a query to get those message. SQS fetches a subset of the server and returns messages such as MSG0,3(message with number postfix of 0 and 3).

For subsequent calls, it returns Message 2 as it finds the third message in the server.

Consuming Message through Long Polling

Let's Invoke SQS in two scenarios: an empty queue and a queue with data. For long polling, set WaitTimeSeconds to 20 seconds.

code
below code will be used for both the scenario, here i have specified the WaitTimeSeconds to be 20 seconds and maxMessage to be 3, this will make long polling request.

const aws = require('aws-sdk')
const sqs = new aws.SQS({ region: 'us-east-1' });

const longPolling = async () => {
    try {
        const startTime = new Date().getTime();
        const params = {
            QueueUrl: 'https://sqs.us-east-1.amazonaws.com/693651664797/PollingQueue',
            WaitTimeSeconds: 20,
            MaxNumberOfMessages: 3
        }
        const data = await sqs.receiveMessage(params).promise();
        const endTime = new Date().getTime();
        console.log('[INFO] Time taken for long polling', `${endTime - startTime} ms`);
        console.log('[INFO] Received messages from SQS by long polling', JSON.stringify(data));
    } catch (error) {
        console.log('[ERROR] Error while long polling the sqs', error);
        throw error;  
    }
};

Scenario1:
When attempting to invoke the queue devoid of data on the server, SQS patiently awaits for a duration of 20 seconds, which is scanning through all distributed servers to fulfil the request, ultimately responding an empty array.

Scenario2:
For this polling it will take all 20 seconds to fetch all the messages in the server, and after 20 seconds it return the response.

Summary

Harness the potential of Short & Long Polling in AWS SQS. Grasp the intricacies of each polling mechanism, tailor them to your scenarios, and optimise message retrieval for a responsive and efficient application. Tailor your SQS interactions with precision, leveraging the strengths of short and long polling based on your use case.

Comments

[Dave Meehan]

Long polling does not guarantee that it will receive all available messages. It returns as soon as it finds any number of messages, only waiting the full WaitTimeSeconds if there are no messages available.

From my own observations, if there is a backlog of at least MaxNumberOfMessages waiting, then the response is likely to contain them all.

Also, any WaitTimeSeconds > 0 activates long polling, not only when it is equal to the maximum value of 20. In other words, short polling is only active when WaitTimeSeconds = 0 or undefined.