I noticed Helen released this good DEV.to post on SNS and SQS and I happen to have a video of the same nature lying around and thought I'd upload i...
For further actions, you may consider blocking this person and/or reporting abuse
@exampro ok to ask a question on setting up SES Email notifications with SNS? I can't seem to get it to work.
SNS to SES?
Yes, what I'm trying to do is send an email through SES. And, subscribe to notifications on bounce / complaint / delivered emails to hit an endpoint. Can't seem to receive the pingbacks, so Iwas wondering if this is something you've dealt with
So in SES you go to Domains > Click the Domain > Notifications and then you can set the SES notification.
Have you done this? I'm guessing you just want to get an SNS email on the bounce?
@andrewbrown thank you for the response :)
I've tried this.
I am using SES to send email and am trying to get notified of bounces, complaints and successful deliveries by:
Help with either (but especially the HTTP SNS POSTs) would be great. Note that the ngrok url / tunnel in the examples below was functional, pasting the url for discussion sake...I can resurrect it if you need me to.
My settings:
From us-west-2.console.aws.amazon.com/s...
Screenshot: prnt.sc/pquysj
SNS Topics and subscriptions:
Topic: us-west-2.console.aws.amazon.com/s...
Confirmed Subscription: us-west-2.console.aws.amazon.com/s...
Send email
From: support@keenbrain.com To: bounce@simulator.amazonses.com
FAIL: No bounce email notification received (at either support@keenbrain.com or kunal@keenbrain.com )
FAIL: No SNS HTTP(S) POST notification received at af84a2ab.ngrok.io/email/sns
Send email
From: support@keenbrain.com To: complaint@simulator.amazonses.com
PASS: Complaint email notification received (at support@keenbrain.com )
FAIL: No SNS HTTP(S) POST notification received at af84a2ab.ngrok.io/email/sns
Send email
From: support@keenbrain.com To: success@simulator.amazonses.com
FAIL: No "delivered" email notification received (at support@keenbrain.com or kunal@keenbrain.com )
FAIL: No SNS HTTP(S) POST notification received at af84a2ab.ngrok.io/email/sns
So, I'm trying to get all the failed scenarios to pass, but especially the ones involving the SNS HTTP(S) POST notifications.
Thank you for preparing that in good detail.
✔️So you have the SNS topics set correctly in SES
✔️So my next question was did you confirm your subscription which you did.
? The next thing I wasn't sure is what kind of subscription but since you are using ngrok and you mention HTTP(S) I am guessing you are trying to create an endpoint so that the SNS notification goes back into your laravel app so you can maybe store the bounces in your application's database.
So the question is where is the problem?
Is it SES, SNS or your Application?
So we need rule each one out, so now I ask if you have logging turned on about delivery status on the notification.
When this is turned on it will deliver that information to CloudWatch logs. If there are logs in there than we know it's your application, if you don't see anything in there then we know the problem is with SES.
If you already have this turned on its going to a bit tricky to debug and we'll have to think outside the box.
One thing to note is that I see a lot of people reach past SNS instead to SQS when they're looking for something "reliable", even when their consumer is going to be Lambda. I think this is generally a mistaken assumption for "serverless" pipelines, because a Lambda integration with SNS actually has capacity for retry and is an "at least once" reliable integration, just like SQS (you can also make SQS be exactly-once delivery, but it's not default).
9/10 times when a developer asks me to help them set up an SQS lambda integration, we end up settling on an SNS lambda integration instead.
I tend to think SQS is a bit outdated compared to SNS and Kinesis, but it definitely still has its place as an integration layer, especially with things that can't use Lambda extensively.
I agree. For lambdas. I don't see the need to use SQS and we use SNS just as you describe.
Both SQS and Kinesis are "good enough" solutions but there are certainly better streaming and queuing services so its easy to outgrow for find these messaging systems limited.
Interesting. Up until reading this I would have considered myself one of that lot.
When I started with Lambda I defaulted to SNS because that seemed more naturally event-driven than polling, but then a few times I ended up missing some notifications because I had some edge cases on a Lambda and had to take it down for whatever reason, then changed my heuristic: If I'm integrating with a 3rd party service which provides a reliable SLA I can use SNS, otherwise I'll default to SQS for peace of mind.
I was not aware of retry / "at least once" delivery on SNS, but it makes total sense. I'll keep it in mind next time I have to do it.
The retry is actually in lambda. All SNS + Lambda integrations will be resent until lambda receives the message, since this is an HTTP integration behind the scenes. You have to use caution once it gets to lambda to ensure you don't accidentally drop the message while working on the code.
Make sure you're not actually taking your lambda offline if you want to ensure delivery -- You'll miss any message that is sent before a lambda integration exists. If you're using Javascript, I recommend making your lambda handlers return promises rather than using callbacks and "rejecting" all failures that you want to retry, as they'll enter the retry policy for the lambda. Here's some info on retry: docs.aws.amazon.com/lambda/latest/...
We've found that they seem to retry at a steady pace with some amount of backoff.
You can also set a dead letter queue for messages that exceed the retry limit, which can be processed in a normal queue manner or by some manual process:
docs.aws.amazon.com/lambda/latest/...
We find a lot of reliability in our lambda + SNS + DLQ setups, to the point that SQS almost never gets used. We're not overly cautious about changing lambda code, but you want to ensure that you do reject on failures and you always keep the lambda SNS integration alive between code updates.
ah Lambda! that makes sense. thanks a lot for the detailed answer. Will keep that in mind for next time.