How to route IoT messages across multiple regions with AWS IoT Core and Amazon SNS

Dmitry Frizner — Wed, 05 Jul 2023 22:45:53 +0000

Introduction

In this blog, I explain how to route AWS IoT Core messages across multiple AWS regions using Amazon Simple Notification Service (Amazon SNS). That is a common pattern when IoT telemetry should be ingested in different geographical regions and shipped to another region for further centralized processing of IoT messages. For instance, when an organization hosts a fleet of devices and collects telemetry in different regions providing lower latency however the collected data across regions should be proceed in one place due to operational or compliance requirements.

You will learn how to configure Rules for AWS IoT and Amazon SNS for delivering of MQTT messages from one AWS region into Amazon Simple Queue Service (Amazon SQS) in another region. A suggested approach can be used for cross-region routing of MQTT messages to other AWS services supported as a destination by Amazon SNS as well. A combination of Amazon SNS and Amazon SQS allows you to transfer messages from one AWS account to another also.

Solution Overview

In this solution, you will first create an Amazon SNS topic in the data ingestion region and an Amazon SQS queue in the data processing region. Then you will grant permissions for the SNS topic to send messages to the SQS queue and subscribe the SQS queue to the SNS topic. Next, you will create an AWS IoT rule to route MQTT messages to the SNS topic. Lastly, you will test the solution by sending IoT messages to an AWS IoT Core topic and polling them from the SQS Queue in another region.

Create an Amazon SNS topic called iot-transfer in the data ingestion region.
In the data processing region:
1. Create an Amazon SQS queue called iot-data.
2. Gant permissions for sending messages from the Amazon SNS topic iot-transfer to the Amazon SQS queue iot-data.
3. Subscribe the Amazon SQS queue iot-data to the Amazon SNS topic iot-transfer.
In the data ingestion region:
1. Create an Identity and Access Management (IAM) Role called iot-sns-allow with an inline policy that allows publishing to the Amazon SNS topic iot-transfer.
2. Create an IoT rule with the IAM role iot-sns-allow in the ingestion account to route messages from a MQTT topic called dt/transferdata to the Amazon SNS topic iot-transfer.
Publish messages to the MQTT topic dt/transferdata.
Verify a delivery of the messages by polling the messages from the Amazon SQS queue iot-data in the data processing region.

Solution Diagram

Solution Instructions

Prerequisites

One ore more AWS accounts.
Administrator privileges in the accounts.
AWS Command Line Interface (AWS CLI).
Command-line JSON processor jq.

To deploy the solution, I recommend to use AWS CloudShell. AWS CloudShell has preinstalled AWS CLI and jq. Some AWS CLI commands have --query parameter to filter the output of running command but I found out an usage of jq utility is more convenient to parse the output.

Define variables

Define environment variables that will be used for the solution deployment. Change values of the variables if needed.

Set up the data ingestion and data processing regions. I use us-east-2 as the data ingestion region and us-west-2 as the data processing region.

AWS_INGESTION_REGION=us-east-2
AWS_PROCESSING_REGION=us-west-2

Define the AWS_ACCOUNT_ID value with an account number of your AWS account. In case if you use different AWS accounts for the ingestion and processing data, set up the variable with the account number of the data ingestion account.

AWS_ACCOUNT_ID=111111111111

Lastly, define names for a SNS topic, SQS queue, IAM roles for Rules for AWS IoT.

AWS_SNS_NAME=iot-transfer
AWS_SQS_NAME=iot-data
AWS_SNS_ROLE_NAME=iot-sns-allow
AWS_REPUBLISH_ROLE_NAME=iot-republish-allow

Create a SNS topic

With the configured AWS CLI create an SNS topic in the data ingestion region and retrieve the TopicArn from the output as a value of a variable. If you use AWS CloudShell, AWS CLI will be configured automatically.

SNS_TOPIC_ARN=$(aws sns create-topic \
   --region $AWS_INGESTION_REGION \
   --name $AWS_SNS_NAME | jq -r .TopicArn)

Create a SQS queue

In the data processing region create a SQS queue with a name defined in the AWS_SQS_NAME variable and retrieve a QueueUrl and QueueArn.

SQS_URL=$(aws sqs create-queue \
   --region $AWS_PROCESSING_REGION \
   --queue-name $AWS_SQS_NAME \
   | jq -r .QueueUrl)

SQS_ARN=$(aws sqs get-queue-attributes \
    --region $AWS_PROCESSING_REGION \
    --queue-url $SQS_URL \
    --attribute-names QueueArn \
    | jq -r .Attributes.QueueArn)

Subscribe the SQS queue to the SNS topic and retrieve a SubscriptionArn.

SUBSCRIPTION_ARN=$(aws sns subscribe \
   --region $AWS_INGESTION_REGION \
   --topic-arn $SNS_TOPIC_ARN \
   --protocol sqs \
   --notification-endpoint $SQS_ARN | jq -r .SubscriptionArn)

Grant permissions to the SNS topic sending messages to SQS

In order the SNS topic can send messages into the SQS Queue created in the previous steps, I need to change a resource policy for the SQS queue and allow the sqs:SendMessage action for the SNS TopicArn.

Create a file with the resource policy running the following command.

cat > policy.json << EOF
{
  "Id": "SNStoSQSCrossRegion",
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "sns.amazonaws.com"
      },
      "Action": "sqs:SendMessage",
      "Resource": "${SQS_ARN}",
      "Condition": {
        "ArnEquals": {
          "aws:SourceArn": "${SNS_TOPIC_ARN}"
        }
      }
    }
  ]
}
EOF

Create a file for SQS attributes and include the resource policy created on the previous step as a string.

cat > sqs-attributes.json << EOF
{
    "Policy": $(jq -Rs . policy.json)
}
EOF

Change the resource policy of the SQS queue running the following command.

aws sqs set-queue-attributes \
    --region $AWS_PROCESSING_REGION \
    --queue-url $SQS_URL \
    --attributes file://sqs-attributes.json

Create the IAM role and policy for publishing to SNS

To publish to the SNS topic from AWS IoT Core, create an IAM role with a policy allowing the publishing action.

Create a file named iot-role-trust-policy.json with a policy allowing to assume the role by iot.amazonaws.com service.

cat > iot-role-trust-policy.json <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
          "Effect": "Allow",
          "Principal":
               {
                    "Service": "iot.amazonaws.com"
               },
          "Action": "sts:AssumeRole"
        }
     ]
}
EOF

Create an IAM role and retrieve a RoleArn

IOT_ROLE_ARN=$(aws iam create-role \
    --role-name $AWS_SNS_ROLE_NAME \
    --assume-role-policy-document file://iot-role-trust-policy.json \
     | jq -r .Role.Arn)

Create a file with IAM policy allowing the sns:Publish action to the SNS topic.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "sns:Publish",
            "Resource": "${SNS_TOPIC_ARN}"
        }
    ]
}
EOF

Attach the policy to the IAM role.

aws iam put-role-policy \
     --role-name $AWS_SNS_ROLE_NAME \
     --policy-name iot-sns-policy \
     --policy-document file://allow_send_sns.json

Create the IAM role and policy for publishing errors to IoT topic

To republish error messages to another MQTT topic with an AWS IoT rule, you will create a role and attach a needed policy to the role.

Create an AWS IAM role and retrieve a RoleArn.

IOT_REPUBLISH_ROLE_ARN=$(aws iam create-role \
    --role-name $AWS_REPUBLISH_ROLE_NAME \
    --assume-role-policy-document file://iot-role-trust-policy.json \
     | jq -r .Role.Arn)

Create a policy allowing publishing to the topic/errors topic and attach the policy to the role created in the previous step.

cat > allow_republish.json <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "iot:Publish",
            "Resource": "arn:aws:iot:${AWS_INGESTION_REGION}:${AWS_ACCOUNT_ID}:topic/errors"
        }
    ]
}
EOF
aws iam put-role-policy \
     --role-name $AWS_REPUBLISH_ROLE_NAME \
     --policy-name iot-republish \
     --policy-document file://allow_republish.json

Create an IoT rule in the data ingestion region to evaluate messages and republish errors

Next, create the IoT Rule that will route messages to the SNS topic and republish any messages that encounter an error to a topic named topic/errors.

Create a file with the rule definition.

cat > ing-rule.json <<EOF
{
    "sql": "SELECT * FROM 'dt/transferdata'" ,
    "description": "Publishing of IoT messages to SNS.",
    "awsIotSqlVersion": "2016-03-23",
    "ruleDisabled": false,
    "actions": [{
        "sns": {
            "roleArn": "${IOT_ROLE_ARN}",
            "targetArn": "${SNS_TOPIC_ARN}",
            "messageFormat": "JSON"
        }
    }], 
    "errorAction": {
            "republish": {
                "roleArn": "${IOT_REPUBLISH_ROLE_ARN}",
                "topic": "error/rules",
                "qos": 0
            }
    }
}
EOF

Create the IoT rule

aws iot create-topic-rule \
    --region $AWS_INGESTION_REGION \
    --rule-name "Publishing2SNS" \
    --topic-rule-payload file://ing-rule.json

Publish messages in the data ingestion region and verify their delivery to the data processing region

In the data ingestion region run the following command to publish a message into the SNS topic.

aws iot-data publish \
    --region $AWS_INGESTION_REGION \
    --topic dt/transferdata \
    --cli-binary-format raw-in-base64-out \
    --payload '{"Default": "data"}'

Use Default as a field name in a json document in order to follow the message format for Amazon SNS.

Verify receiving messages in the data processing region.

aws sqs receive-message \
    --region $AWS_PROCESSING_REGION \
    --queue-url $SQS_URL

If don’t receive the messages, in the data ingestion region, check the topic/errors topic in AWS Console with the AWS IoT MQTT client for error messages related to delivery.

Cleaning Up

It is good practice to clean up any resources you no longer want to use. Cleaning up AWS resources prevents your account from incurring any further charges.

Delete IoT Rule.

aws iot delete-topic-rule \
    --region $AWS_INGESTION_REGION \
    --rule-name "Publishing2SNS"

Unsubscribe the SQS queue from the SNS topic.

aws sns unsubscribe \
    --region $AWS_INGESTION_REGION \
    --subscription-arn $SUBSCRIPTION_ARN

Delete the SNS topic and the SQS queue.

aws sns delete-topic \
    --region $AWS_INGESTION_REGION \
    --topic-arn $SNS_TOPIC_ARN
aws sqs delete-queue \
    --region $AWS_PROCESSING_REGION \
    --queue-url $SQS_URL

Delete the IAM roles and policies.

aws iam delete-role-policy \
    --role-name $AWS_SNS_ROLE_NAME \
    --policy-name iot-sns-policy
aws iam delete-role \
    --role-name $AWS_SNS_ROLE_NAME
aws iam delete-role-policy \
    --role-name $AWS_REPUBLISH_ROLE_NAME \
    --policy-name iot-republish
aws iam delete-role \
    --role-name $AWS_REPUBLISH_ROLE_NAME

Conclusion

In this blog I showed you how to route AWS IoT messages from one AWS Region to Amazon SQS in another region. The pattern allows use different AWS regions for ingestion and processing data.

DEV Community: Dmitry Frizner