AWS SQS to Lambda Failed Processed Messages

MacAndersonUche — Wed, 18 Sep 2024 22:30:40 +0000

Here’s a corrected version of your text:

Imagine a scenario where a Lambda function polls messages from an SQS queue but fails to process some messages.

Issue

During batch processing, SQS doesn't track what happens after triggering Lambda. If a message fails or is throttled, it remains invisible until deleted or made visible again after the VisibilityTimeout expires. Throttled messages are treated as failures, and their ReceiveCount increases each time they're retried. If a large number of messages arrives, some may be throttled and retried repeatedly until they reach the maxReceiveCount and are moved to the Dead Letter Queue (DLQ).

Fix

We implemented two solutions:

maxReceiveCount: We increased the maxReceiveCount from 1 to 6 in the SQS queue. This was configured using CloudFormation:

   Queue:
     Type: AWS::SQS::Queue
     Properties:
       RedrivePolicy:
         maxReceiveCount: 6

VisibilityTimeout: We increased the VisibilityTimeout from 40 to 120 seconds. This was done using CloudFormation:

   Queue:
     Type: AWS::SQS::Queue
     Properties:
       VisibilityTimeout: 120
       DelaySeconds: 30

AWS SQS to Lambda 429 Errors

MacAndersonUche — Wed, 18 Sep 2024 22:04:56 +0000

Imagine a scenario where a Lambda function polls messages from an SQS queue and uses the payload from those messages to make a POST request to an AWS OpenSearch domain via the REST API endpoint.

Issue

During batch processing, multiple Lambda instances are triggered simultaneously as soon as messages arrive in the queue. This led to some requests being processed too quickly, causing a 429 (Too Many Requests) error due to rate limiting.

Fix

We implemented two solutions:

Reserved Concurrency: We limited the Lambda function's concurrency to 1 to ensure that only one instance processes messages at a time. This was done using CloudFormation:

   Lambda:
     Type: AWS::Serverless::Function
     Properties:
       ReservedConcurrentExecutions: 1

Queue Delay: We set a value for DelaySeconds in the SQS queue to introduce a delay before the next message is processed, allowing OpenSearch more time to handle requests. This was also configured using CloudFormation.

  Queue:
    Type: AWS::SQS::Queue
    Properties:
      VisibilityTimeout: 40
      DelaySeconds: 30

DEV Community: MacAndersonUche

AWS SQS to Lambda Failed Processed Messages

Issue

Fix

AWS SQS to Lambda 429 Errors

Issue

Fix