DEV Community

MacAndersonUche
MacAndersonUche

Posted on

AWS SQS to Lambda Failed Processed Messages

Here’s a corrected version of your text:


Imagine a scenario where a Lambda function polls messages from an SQS queue but fails to process some messages.

Image description

Issue

During batch processing, SQS doesn't track what happens after triggering Lambda. If a message fails or is throttled, it remains invisible until deleted or made visible again after the VisibilityTimeout expires. Throttled messages are treated as failures, and their ReceiveCount increases each time they're retried. If a large number of messages arrives, some may be throttled and retried repeatedly until they reach the maxReceiveCount and are moved to the Dead Letter Queue (DLQ).

Fix

We implemented two solutions:

  1. maxReceiveCount: We increased the maxReceiveCount from 1 to 6 in the SQS queue. This was configured using CloudFormation:
   Queue:
     Type: AWS::SQS::Queue
     Properties:
       RedrivePolicy:
         maxReceiveCount: 6
Enter fullscreen mode Exit fullscreen mode
  1. VisibilityTimeout: We increased the VisibilityTimeout from 40 to 120 seconds. This was done using CloudFormation:
   Queue:
     Type: AWS::SQS::Queue
     Properties:
       VisibilityTimeout: 120
       DelaySeconds: 30
Enter fullscreen mode Exit fullscreen mode

Heroku

Build apps, not infrastructure.

Dealing with servers, hardware, and infrastructure can take up your valuable time. Discover the benefits of Heroku, the PaaS of choice for developers since 2007.

Visit Site

Top comments (0)

Image of Bright Data

Ensure Data Quality Across Sources – Manage and normalize data effortlessly.

Maintain high-quality, consistent data across multiple sources with our efficient data management tools.

Manage Data

👋 Kindness is contagious

Dive into an ocean of knowledge with this thought-provoking post, revered deeply within the supportive DEV Community. Developers of all levels are welcome to join and enhance our collective intelligence.

Saying a simple "thank you" can brighten someone's day. Share your gratitude in the comments below!

On DEV, sharing ideas eases our path and fortifies our community connections. Found this helpful? Sending a quick thanks to the author can be profoundly valued.

Okay