miruky

Posted on Aug 24

Fan Out and Filter Amazon SNS Messages to Two Amazon SQS Queues

#aws #sns #sqs #tutorial

Introduction

Hi, I'm miruky.

An Amazon SNS topic normally delivers each published message to every subscription. A subscription filter policy changes that behavior: SNS evaluates the selected message attributes or JSON body, then sends the message only to subscriptions whose policies match.

This Console run connects one standard topic to two standard SQS queues. Queue A accepts event_type=created, and Queue B accepts event_type=cancelled. A control publication without that attribute reaches neither queue.

The walkthrough uses only the AWS Management Console in N. Virginia. It does not provision always-on compute, but SNS and SQS still meter requests and deliveries; check the current pricing pages for your account and usage. The identity running the steps needs permission to create the topic and queues, subscribe the queues, edit subscription attributes, publish test messages, and receive messages from both queues.

1. Fix the Region and create a standard topic

This evidence session uses the Console in English in United States (N. Virginia). SNS topics and SQS queues are Regional resources, so keeping all three resources in us-east-1 also avoids a cross-Region subscription path.

The Region selector remains on United States (N. Virginia) as the SNS workflow begins. I opened Amazon SNS, chose Topics, and then chose Create topic with Standard and the generated name miruky-atfqphosahzwoonv.

The form keeps Standard selected and the generated name in place. I left the access policy at its same-account default and created the topic.

The green notification confirms that the generated standard topic now exists. No subscription endpoint is attached yet, so I moved to SQS to create Queue A.

2. Create two standard queues

Open Amazon SQS, choose Create queue, and keep Standard selected. Enter miruky-urjmozcztafyjhiq for Queue A and review the visibility timeout, retention period, encryption, and access-policy sections before creating it.

The form keeps Standard selected for Queue A. I created it with the reviewed defaults, then repeated the same process with miruky-tlmpyztrqraxqkse.

These are standard queues, so the routing test does not assert exactly-once delivery or strict ordering. A production consumer for this queue type must tolerate duplicate and out-of-order messages.

3. Subscribe both queues through the SQS Console

Open miruky-urjmozcztafyjhiq, choose Actions, and then choose Subscribe to Amazon SNS topic. The topic picker can use the same-Region list or accept the topic ARN directly. In this run, I chose Enter Amazon SNS topic ARN, entered the ARN for miruky-atfqphosahzwoonv, and saved the subscription. The account-scoped portion of that ARN is masked in the screenshot.

The retained form shows Queue A's SNS subscription ready to save with the account-scoped ARN masked. The supported SQS workflow adds the queue access-policy statement needed for the topic to call sqs:SendMessage, so I kept that statement scoped to the exact topic.

Open miruky-tlmpyztrqraxqkse and repeat the same ARN-based subscription flow with the same topic. Queue B needs its own subscription and its own topic-specific queue-policy statement.

The retained form shows Queue B using the same masked topic-ARN path. With both subscriptions saved, neither has a filter policy yet, so the next step changes them independently.

4. Apply one attribute filter to each subscription

Return to Amazon SNS and edit the subscription whose endpoint is miruky-urjmozcztafyjhiq. Use Subscription filter policy with Message attributes and this JSON:

{
  "event_type": ["created"]
}

The Queue A editor keeps Message attributes and accepts created for event_type. I saved that policy, then edited the subscription for miruky-tlmpyztrqraxqkse with the different value shown below.

{
  "event_type": ["cancelled"]
}

The Queue B editor keeps Message attributes and accepts cancelled for event_type. SNS applies filter-policy changes with eventual consistency, and AWS documents that a new or changed policy can require up to 15 minutes to take full effect, so I waited the full interval before publishing the control message.

5. Publish a control message without the required attribute

Open the topic and choose Publish message. Leave the message-attributes section empty. Enter control-without-event-type in the body and publish once.

The retained form shows the control body and an empty message-attributes row. The default filter scope is MessageAttributes, so a publication without event_type does not satisfy either policy.

I opened each queue and ran the Console receive-message poll repeatedly after the publication completed. Both polls returned zero messages for this control run.

Queue A remains at Messages (0) during the control poll. I repeated the same poll on Queue B to check the other subscription.

Queue B also remains at Messages (0). Together, the two zero results distinguish subscription filtering from ordinary topic fan-out.

6. Route the created event to Queue A

Publish the next message with body created-event. Use a String attribute named event_type with the value created.

The form shows created-event with a String attribute named event_type and value created. I published it once, then polled miruky-urjmozcztafyjhiq.

The Queue A panel now reports Messages (1), up from zero after the control. I then polled miruky-tlmpyztrqraxqkse separately to test the non-matching subscription.

Queue B remains at Messages (0) after the created publication. The difference between one in Queue A and zero in Queue B is the expected created-event routing result.

7. Route the cancelled event to Queue B

Publish the final message with body cancelled-event. Use the same event_type String attribute but change its value to cancelled.

The form now shows cancelled-event with event_type set to cancelled. I published it once, then polled Queue B again.

Queue B moves from zero to Messages (1) after the cancelled publication. A final Queue A poll remained at one, so the cancelled event did not add another Queue A delivery.

The result is one topic with two independent delivery rules. Producers publish to the topic once, while each subscription decides whether its queue should receive that event.

Wrap-up

This run separated three behaviors that are commonly blurred together. An unfiltered SNS subscription receives every message, an attribute filter suppresses messages that lack the required key, and two subscriptions can accept different values from the same topic.

The small Console test does not replace production design. Keep each queue policy scoped to the intended topic, wait for filter changes to propagate before testing, and make standard-queue consumers idempotent.

Thanks for reading this far.

See you in the next one.

Disclosure: This article was written with AI assistance and independently verified against the linked primary sources and observed results.

DEV Community