Overview ℹ️
Amazon SQS is a fully managed message queueing service offered by Amazon Web Services (AWS). Before cloud-based message queueing services existed, we often needed to create our own message queue solution. Amazon SQS helps to eliminate the overhead and the complexity of trying to create your own managed queueing solution. When using Amazon SQS, we can more easily decouple and scale our microservices and serverless applications. Amazon SQS allows you to send, store and receive any volume of messages between software components securely without worrying of losing any of the messages.
Amazon SQS provides two types of queues namely:
- Standard: It is fully managed, scales from 1 message per second to 10,000s per second. Default retention of messages: 4 days, maximum of 14 days and no limit to how many messages can be in the queue. It has low latency (<10 ms on publish and receive) and horizontal scaling in terms of number of consumers.
- FIFO: It is referred as First-In-First-Out delivery and exactly-once processing. The order in which messages are sent and received is strictly preserved and a message is delivered once and remains until consumer processes delete it. Duplication is not allowed in FIFO and it is limited to 300 transactions per second.
If your application is cloud-native, large-scale, or distributed, and does not include a messaging component, that’s probably a bug! Tim Bray – AWS Senior Principal Engineer
Benefits
Amazon SQS provides several benefits:
- Reduce administrative overheads – AWS manages all the ongoing operations and the underlying infrastructure that is required to provide the highly available and scalable messaging service. AWS has taken the need to install and configure any messaging software.
- Reliably deliver messages – We are able to transmit any volume of data and any level of throughput to our Amazon SQS queue without any worry of losing any messages. When using Amazon SQS, we can increase the fault tolerance of our system by decoupling our application components, which basically means creating components or services so that they run and fail independently. We can have a service which is unavailable and still processes messages.
- Messages are stored redundantly across multiple availability zones – which makes messages always available when we need them.
Common Usage of SQS
TV Voting System
Suppose you are watching a TV show that allows you to vote for a winner after a performance. At the end of the performance, 5 to 50 million viewers are all voting at the same time. How are you are going to handle a large spike of traffic in such a short space of time? One solution would be to build a significant web server tier and database back-end that could manage millions of messages per second but that would be costly to scale, as you would have to pre-provision for maximum expected workload and at the same time, it might not be fault-tolerant due a database failure or throttling. Consequently, this may cause a loss of votes.
As a solution to address this problem is to make use of a queueing mechanism to decouple the voting application system from the service where the vote queue will be highly scalable. This will allow to adequately absorb 10 messages/sec or 10 million messages/sec. Eventually, you would have a consumer tier which will be responsible to pull the messages from the queue as fast as possible to tally the votes. The queuing system will provide us with a more resilient system, and handle parts of our system being down.
Amazon SQS Architecture
Amazon states that there are three main parts in a distributed messaging system. First, there are the components or services which generally send or receive messages from our SQS queue. The second part is the SQS queue itself. The third part is the messages inside our queue.
An important concept to understand is where and how our messages are being stored. Whenever a message is sent to our SQS queue, there are redundantly stored across multiple Amazon SQS services. By distributing the same message across different SQS services, AWS can guarantee safe and successful storage of our messages. In our example, you can see messages A, B, C and D are stored in different AWS services to ensure that if anything goes wrong on AWS’s side, we are still guaranteed that we receive our messages that we have sent to our SQS queue.
The diagram below illustrates the architecture of Amazon SQS.
Wrapping Up 📢
In this article, we learnt that SQS can be used to efficiently manage messages that flow through queues, and how they are consumed. The architecture of Amazon SQS illustrated in this article explains the underlying mechanism of how messages are stored and processed. In a next article, I shall demonstrate how to create, configure Amazon SQS and consume a message.
Top comments (0)