Day 019 - 100DaysAWSIaCDevopsChallenge
Recently, I’ve come across questions in communities like Stack Overflow and Reddit, where users are asking how to enable multiple applications (acting as consumers) to process a single message from an Amazon SQS Queue. While it is technically possible to consume or read an SQS message multiple times, in this article, I will walk you through the best practices for handling situations where you have one queue serving multiple consumers.
Amazon SQS vs Amazon SNS
| Points of comparison | Simple Queue Service | Simple Notification Service |
|---|---|---|
| Communication model |
Queue based, SQS is used for Point-to-Point communication. The producer pushes a message to the queue, and only one consumer retrieves and processes it. Messages are garanteed to be processed only one time(idempotent). |
Pub/sub, SNS is used to broadcast messages to multiple subscribers simultaneously. The message is published on a topic and all subscribers receive it in parallel. |
| Destination | Only one destination per message. Once the message is consumed, it is desappeared/deleted from the queue. It is suitable for systems where a single service must process the message | Designed for multiple destinations. The message is sent to many consumers like applications (HTTP/S), SMS, SQS or Lambda Function. It is suitable for the broadcast at scale |
| Ordonnancing and duplication | There are two types of SQS: STANDARD and FIFO (First-In-First-Out). In the STANDARD version, messages can be delivered in any order and can be duplicated. In the FIFO version, messages are delivered exactly in the order they are sent and no duplication is allowed. |
The order is no guaranteed and no protection against duplication. Each consumer receives the message at the same time and in any order. |
| Message retention | Messages are retained inside the queue until they are consumed or until the retention period expires (the maximun retention period is 14 Days) | Message are not retained. They are immediately sent to the subscribers as soon as they published. ⚠️⚠️ If the subscriber is not available, the message is lost(only if he is attached to the SQS Queue) |
| Use Case | Buffering 📬 | Broadcasting 🛗 |
| Pricing | Based on the number of sent and receive requests and messages deletions | Based on publising and noticications sent |
There might be other elements of comparison that I haven't mentioned here. But for now we'll stop at the most important ones here. If you think of anything else, please leave a comment, it could also help me😇.
Which one to choose ?
Now that we've established the differences between SQS and SNS, the question arises: when should you use one over the other?
You can use SQS when:
- you wish to control the order of message processing using
SQS FIFO - you need to handle a high volume of messages or process messages in batches
- you have an asynchronous workflow where only one consumer should process each message.
You can use SNS when:
- you need to send the same message to multiple consumers simultanously.
- you're to implementing a Pub/Sub(Publish/Subscribe) model
- you require real-time proccessing, such as push notifications
- you want to alert multiple systems in real time (e.g. Email, SMS, monitoring, SQS Queue, etc.).
To summarize:
✅✅ SQS is suitable for systems where messages need to be processed by a single consumer.
✅✅ SNS is perfect for broadcasting notifications to multiple systems simultaneously.
In practice, vous can combine SNS and SQS, where SNS sends notifications to multiple SQS Queues, allowing each queue to have its dedicated consumer. This will be covered in the next article (How to combine SQS and SNS to implement multiple Consumers[soon]). stay turned!
Conclusion
The right choice between Amazon SQS and SNS can elevate your application’s messaging capabilities. If you have any experiences, questions, or insights to share, feel free to leave a comment below 💬.
Let’s learn and grow together!😎
Top comments (1)
Nice article