Bry

Posted on Sep 8 Originally published at Medium

Amazon SQS vs SNS: Queues, Fan-Out, and Picking the Right One from the CLI

Key Points

SQS is a queue — one message, delivered to one consumer, held until processed. SNS is pub/sub — one message, fanned out to every subscriber. The fan-out pattern combining both is the standard, not an either/or choice.
SNS-to-SQS delivery requires the queue's access policy to explicitly allow the topic to call SendMessage — skip this and messages vanish with no error on the publish side.
FIFO queues cap at 3,000 messages/second batched (300/s unbatched) unless you enable high-throughput mode, which raises that to 70,000/s. Both SQS and SNS also bill in 64 KB payload chunks, same mechanic as EventBridge.

Prerequisites

CLI/SDK version tested against: aws-cli/2.35.x
An IAM role with sqs:* and sns:* permissions
jq installed for parsing CLI JSON output in the examples below

Introduction

I still get asked "should I use SQS or SNS?" as though it's a fork in the road. Most of the time the honest answer is both, wired together — SNS decides who gets notified, SQS makes sure each of them actually gets to process the message at their own pace without losing it if they're briefly unavailable. Treating them as competing choices is how teams end up building a fan-out mechanism inside application code that SNS already does for free.

The part that trips people up isn't the concept. It's the access policy. Subscribe an SQS queue to an SNS topic without granting the topic permission to send to that queue, and the subscription succeeds, the topic publish succeeds, and the message simply never arrives — no error surfaces anywhere in that chain. I've debugged this exact silent failure in a client's fan-out pipeline, and it's the single most common gap in CLI tutorials for this pattern.

This article builds a standalone queue, a standalone topic, and the full fan-out pattern with the access policy step included, then covers where FIFO changes the throughput math.

Queue vs Topic, Conceptually

Diagram: SQS delivers each message to exactly one consumer; SNS delivers a copy to every subscriber.

A message sitting in an SQS queue waits until something polls for it. A message published to an SNS topic is pushed immediately to every current subscriber — there's no "waiting" concept on the topic itself, which is exactly why SNS alone is a poor fit for a subscriber that might be temporarily down. That's what the fan-out pattern fixes.

Standalone SQS

# Standard queue — no ordering guarantee, near-unlimited throughput.
aws sqs create-queue --queue-name orders-standard-queue

QUEUE_URL=$(aws sqs get-queue-url --queue-name orders-standard-queue --query QueueUrl --output text)

aws sqs send-message \
  --queue-url "$QUEUE_URL" \
  --message-body '{"orderId":"order-abc123","status":"pending"}'

aws sqs receive-message --queue-url "$QUEUE_URL" --max-number-of-messages 1

# FIFO queue — strict ordering within a message group, capped throughput.
aws sqs create-queue \
  --queue-name orders-fifo-queue.fifo \
  --attributes '{"FifoQueue":"true","ContentBasedDeduplication":"true"}'

Queue type is permanent. There's no update-queue-type command — you create a new queue and migrate, you don't convert an existing one. Decide standard vs FIFO before you have production traffic depending on the answer.

Standalone SNS

aws sns create-topic --name orders-topic

TOPIC_ARN=$(aws sns list-topics --query "Topics[?contains(TopicArn,'orders-topic')].TopicArn" --output text)

aws sns publish \
  --topic-arn "$TOPIC_ARN" \
  --message '{"orderId":"order-abc123","status":"pending"}' \
  --subject "Order Created"

That publish immediately pushes to every current subscriber — email, SMS, HTTPS endpoint, Lambda, or SQS. Nothing is retained after delivery attempts complete. If you need durability — a subscriber that's slow or briefly offline shouldn't lose the message — that's the case for combining SNS with SQS, not using SNS alone.

The Fan-Out Pattern — Including the Step Everyone Skips

Diagram: fan-out delivery depends entirely on each queue's access policy explicitly trusting the topic — there's no other permission gate in this chain.

aws sqs create-queue --queue-name orders-notifications-queue
QUEUE_URL=$(aws sqs get-queue-url --queue-name orders-notifications-queue --query QueueUrl --output text)
QUEUE_ARN=$(aws sqs get-queue-attributes --queue-url "$QUEUE_URL" --attribute-names QueueArn --query 'Attributes.QueueArn' --output text)

# This is the step that gets skipped. Without it, the subscription
# below will succeed and messages will vanish with zero errors.
aws sqs set-queue-attributes \
  --queue-url "$QUEUE_URL" \
  --attributes "{\"Policy\":\"{\\\"Version\\\":\\\"2012-10-17\\\",\\\"Statement\\\":[{\\\"Effect\\\":\\\"Allow\\\",\\\"Principal\\\":{\\\"Service\\\":\\\"sns.amazonaws.com\\\"},\\\"Action\\\":\\\"sqs:SendMessage\\\",\\\"Resource\\\":\\\"${QUEUE_ARN}\\\",\\\"Condition\\\":{\\\"ArnEquals\\\":{\\\"aws:SourceArn\\\":\\\"${TOPIC_ARN}\\\"}}}]}\"}"

aws sns subscribe \
  --topic-arn "$TOPIC_ARN" \
  --protocol sqs \
  --notification-endpoint "$QUEUE_ARN" \
  --attributes RawMessageDelivery=true

RawMessageDelivery=true matters as much as the policy step. Without it, the queue receives SNS's full JSON envelope — Type, MessageId, TopicArn, and the actual message nested inside a Message string field — instead of your original payload directly. Consumers that expect the raw body will parse the wrong structure and either error out or silently read undefined fields.

FIFO Throughput Limits

Configuration	Throughput ceiling
FIFO queue, batched	3,000 messages/second
FIFO queue, unbatched	300 messages/second
FIFO queue, high-throughput mode	70,000 messages/second
SNS FIFO topic	3,000 messages/second or 20 MB/second, whichever is hit first
Standard queue/topic	No published ceiling

High-throughput mode isn't automatic — it's an explicit setting (DeduplicationScope and FifoThroughputLimit attributes set to messageGroup and perMessageGroupId respectively) that trades a small amount of strict cross-group ordering guarantee for the throughput increase. If you need strict global ordering across all message groups, don't enable it. If your ordering requirement is per-customer or per-order (a common case), high-throughput mode is almost always the right call.

Common Mistakes

Mistake 1: Subscribing SQS to SNS without a queue access policy
The subscription API call succeeds regardless. The failure is invisible until someone notices messages aren't arriving — which could be minutes or weeks later depending on traffic patterns.

Mistake 2: Forgetting RawMessageDelivery
Consumers built against the raw payload shape break silently or throw confusing parsing errors when they receive SNS's wrapped envelope instead.

Mistake 3: Choosing FIFO by default "to be safe"
FIFO's throughput ceiling and stricter deduplication requirements are a real cost. Most systems don't actually need strict ordering — verify the requirement before paying for it in complexity and throughput headroom.

Mistake 4: Trying to convert a standard queue to FIFO
There's no such command. You create a new FIFO queue and migrate producers and consumers to it — plan for that as a deploy, not a config change.

Production Considerations

Performance: Long polling (--wait-time-seconds 20 on receive-message) eliminates the empty-receive charges that pile up from aggressive short polling — this alone is one of the two biggest SQS cost levers, the other being batching sends and receives.

Security: Scope queue access policies to the specific topic ARN, not a wildcard principal. The AWS Tip source in this article's research log describes a real incident where a queue policy pointed at a stale topic ARN after a topic recreation — silent failure, same root cause as skipping the policy entirely.

Cost: Both services bill in 64 KB payload chunks. A consistently large message body (nested JSON, embedded metadata) multiplies your bill the same way it does on EventBridge — trim payloads, pass references where you can.

Monitoring: Alarm on ApproximateAgeOfOldestMessage for SQS queues — a rising value means consumers aren't keeping up, well before the queue depth itself looks alarming.

Full Example: Fan-Out Setup Script

#!/usr/bin/env bash
set -euo pipefail

TOPIC_NAME="${TOPIC_NAME:-orders-topic}"
QUEUE_NAME="${QUEUE_NAME:-orders-notifications-queue}"

aws sns create-topic --name "$TOPIC_NAME" >/dev/null
TOPIC_ARN=$(aws sns list-topics --query "Topics[?contains(TopicArn,'${TOPIC_NAME}')].TopicArn" --output text)

aws sqs create-queue --queue-name "$QUEUE_NAME" >/dev/null
QUEUE_URL=$(aws sqs get-queue-url --queue-name "$QUEUE_NAME" --query QueueUrl --output text)
QUEUE_ARN=$(aws sqs get-queue-attributes --queue-url "$QUEUE_URL" --attribute-names QueueArn --query 'Attributes.QueueArn' --output text)

POLICY=$(cat << EOF
{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"sns.amazonaws.com"},"Action":"sqs:SendMessage","Resource":"${QUEUE_ARN}","Condition":{"ArnEquals":{"aws:SourceArn":"${TOPIC_ARN}"}}}]}
EOF
)

aws sqs set-queue-attributes --queue-url "$QUEUE_URL" --attributes "{\"Policy\":$(echo "$POLICY" | jq -Rs .)}"
aws sns subscribe --topic-arn "$TOPIC_ARN" --protocol sqs --notification-endpoint "$QUEUE_ARN" --attributes RawMessageDelivery=true

echo "Fan-out ready: $TOPIC_NAME -> $QUEUE_NAME"

Full source including a batched consumer with long polling: GitHub → cloud-apis/amazon-sqs-vs-sns-cli/

Conclusion

Stop framing SQS and SNS as competing choices — SNS decides who hears about something, SQS makes sure each listener actually gets to act on it without losing the message if they're briefly unavailable. The fan-out pattern combining both is the default for a reason. The one step that will actually cost you debugging time if skipped is the queue access policy: get it wrong and everything upstream reports success while the message goes nowhere.

DEV Community