I noticed Helen released this good DEV.to post on SNS and SQS and I happen to have a video of the same nature lying around and thought I'd upload it to provide an explanation from a different perspective.
The 4 most important services you need to know when it comes to Application Integration are the following:
- SNS - PubSub (think pass along)
- SQS - Queueing (think batch)
- Kinesis - Real-time
- SWF - Queuing but with people
For the Solution Architect Associate and Certificated Cloud Practitioner you really need to know the difference between SNS and SQS.
Most companies in BigData who want to use real-time messaging systems tend to use Kafka. Luckily AWS has a managed service called MKS (Managed Kafka Service).
The use case for Kinesis has been really good for gaming where you need a simple streaming queue. There is a Solutions Architect Professional exam question which asks how you would ingest millions of events from IoT game devices to then insert into a database such as DynamoDB to update a scoreboard, and so Kinesis is what you would keep in mind.
At ExamPro we built our own real-time event tracking (think GoogleAnalytics) so we can accurately track and improve how people learn on our platform.
SNS is commonly used for sending simple internal emails. So for example when someone sign-ups on ExamPro I use the AWS SDK to send a message to an SNS topic which then sends me a plain text email.
Another way we use SNS is to trigger webhooks in your application. So for example, let us say you have a web-app which needs to extract data out of a PDF so you need to use a PDFTK which you have to run in a custom runtime in a Lambda function because of legacy reasons. When that lambda has processed that pdf you want to notify your web-app to tell the user it's done. You can have Lambda tell SNS which will then send an HTTP request to an api endpoint (webhook) to your web-app which will then update the user that the pdf has finished processing.
When we send out an email such as confirmation or reset password on ExamPro we use SQS to decouple email sending from our app so it doesn't cause the app to hang. ExamPro is a Rails app and so this gem called Shoryuken makes it really easy to start working with SQS will little knowledge.
Shoryuken sho-ryu-ken is a super-efficient Amazon SQS thread-based message processor.
- Rails Active Job
- Queue Load balancing
- Concurrency per queue
- Long Polling
- Batch processing
- Auto extend visibility timeout
- Exponential backoff
- Middleware support
- Amazon SQS CLI. See
shoryuken help sqs
Ruby 2.4 or greater.
Add this line to your application's Gemfile:
If you are using AWS SDK version 3, please also add this line:
The extra gem
aws-sdk-sqs is required in order to keep Shoryuken compatible with AWS SDK version 2 and 3.
And then execute:
Check the Getting Started page.
For more information check the wiki page.
- Fork it ( https://github.com/phstc/shoryuken/fork )
- Create your feature branch (
git checkout -b my-new-feature…
Nobody I know uses this service. It's used by AWS I believe to help coordinate a sequence of tasks with people and services which makes sense for Amazon when they need to have humans put things into boxes and then ship those boxes.
There isn't even a new AWS icon. I've never been able to figure out how to access this service. It shows up on exams.
If you like this kind of AWS content the way you can help me out is adding me to Linked In and endorsing my skill for AWS. You should add me anyway I love connecting with people on DEV.to
Posted on May 14 '19 by: