Error handling in AWS Lambda triggered by SQS events

Liquid syntax error: Unknown tag 'endraw'

Comments

[Victor Ferreira]

What if we do the other way around to handle errors in batches: remove from the queue the ones that succeeded and fail everything (this intend to make the ones that are in the batch, but were not removed from the queue to rerun).

Would that work?

[Marcin Piczkowski]

Hi, not sure I got you right. Are you refering to lambda processing a batch of messages from sqs queue? In this case when Lambda fails the whole batch of messages gets back to the queue. Sure, you can catch the exception and continue to process the remaining messages from the batch, then handle the failing messages at the end and removing the successful ones from the queue. This should work. It's a good question. I should have mentioned about such use case too.

[Eron Fontes]

Hello Marcin,

I solve this problem setting my DelaySeconds in 2 minutes for 300 regs. I belive the lambda lost yourself reading message SQS and repost. Maybe this problem comeback if the numbers regs up.

I hope help.

[anjali]

Hey, Thanks much for this article, really helpful, But when you used RedrivePolicy method, Don't you need to attach your source queue to the lambda through serverless.yml? or Are you doing it manually?, like giving the "Dead-letter queue service" Field in Lambda. (This comes in asyncronous invocation tab in lambda whene using consol), If this isn't given, how your lambda will send the failures to source queue's?

[Marcin Piczkowski]

Hi, My source queue (MyQueue) is attached to lambda in serverless.yaml via events.sqs section. What I do later is defining a new queue (ReceiverDeadLetterQueue) as a dlq for the source queue, instead defining dlq for lambda. In AWS you can define fallback behavior (dlq) either on Lambda or on SQS. I took the second approach, because for the first approach I would need to have SNS topic and then subscribe queue to it, which I think would be too much. So using dlq on SQS (source queue) was simpler, even if I had to write some cloudformation piece of code in resources section of my serverless.yaml. So now Lambda knows nothing about dlq. This is SQS MyQueue which knows that if the same message returns (which happens when Lambda fails) it should pass it to dlq. Hope you got the idea :)