AWS IoT Core: Connect Millions of Devices Securely

The managed hub that lets huge fleets of devices talk to the cloud safely, and routes their data wherever you need it. The final stop in the AWS Hidden Gems series.

About this series

Most AWS learning stops after EC2, S3, IAM, and Lambda. But AWS has over two hundred services, and many of the most useful ones rarely appear in tutorials.

AWS Hidden Gems covers those underrated services you shouldn't ignore. Each article picks one, then explains why it exists, what it does, where it fits, and how to set it up from the console. Know the four basics above and you can follow along. Everything else gets explained as it comes up.

Today's service: AWS IoT Core

A smart thermostat, a delivery truck tracker, a factory sensor: physical devices that send data to the cloud and take commands back. Connecting one is straightforward. Connecting millions, securely, and doing something useful with their data, is a real engineering problem. AWS IoT Core is the managed service that solves it.

Why does this service exist?

Getting a fleet of devices talking to the cloud is harder than it sounds. Devices are often small and low powered, so they need a lightweight messaging method rather than heavy web requests. Every device needs its own secure identity, so a stolen one can be shut off without affecting the rest. Connections number in the thousands or millions and come and go as devices sleep and wake. And once the data arrives, you need to route it to storage and processing.

Building all of that, a secure and scalable message broker with per device identity and routing, is a huge undertaking. IoT Core provides it as a managed service, so you focus on your devices and what their data does, not the connectivity plumbing.

What is AWS IoT Core?

IoT Core is a managed service that connects IoT devices to AWS and routes their messages. IoT, the Internet of Things, means everyday physical objects fitted with sensors and network connections.

Its main parts are:

• A device gateway that devices connect to, speaking MQTT, a lightweight publish and subscribe messaging protocol designed for devices with little power or bandwidth
• Secure authentication, usually with a unique certificate per device, so each one has its own identity
• A rules engine that watches incoming messages and routes them to other AWS services using simple SQL like statements
• A device shadow, which is a stored copy of each device's last reported and desired state, so your app can read or set a device's state even while it is offline With publish and subscribe, devices publish messages to named topics, and anything interested subscribes to those topics, rather than talking to each other directly. IoT Core can scale this to very large fleets.

A real world problem

A startup makes internet connected air quality monitors. Thousands are now in homes, each sending a reading every minute, and more ship every week.

They need every device connected securely, so a compromised unit can be revoked on its own. They need the readings stored for history and dashboards. And they cannot manage servers for a message broker as the fleet grows.

IoT Core covers it. Each monitor connects with its own certificate over MQTT, a rule routes every reading into a database automatically, and the whole thing scales with the fleet without the team running any messaging infrastructure.

Real world use cases

• Smart home products connecting devices like thermostats, lights, and sensors
• Industrial monitoring collecting data from factory machines and equipment
• Fleet and asset tracking reporting location and status from vehicles and containers
• Agriculture gathering readings from field sensors for soil and weather
• Energy managing smart meters and grid devices at scale
• Consumer electronics giving products a secure cloud connection and remote control The pattern is many devices sending data to the cloud securely and having it routed somewhere useful.

Where it fits in AWS

Devices connect to IoT Core over MQTT and publish their data. The rules engine inspects each message and sends it to the right place, such as Timestream for time series storage, Lambda for custom logic, DynamoDB, or S3. The device shadow keeps each device's state so apps can interact with it any time. This pairs naturally with Timestream from earlier in this series, feeding sensor readings straight into a time series database for dashboards.

flowchart LR
    A[Devices] -->|MQTT| B[IoT Core device gateway]
    B --> C[Rules engine]
    C --> D[Timestream]
    C --> E[Lambda]
    C --> F[DynamoDB or S3]
    B --> G[Device shadow: device state]

IoT Core is the secure front door and traffic router for your devices. It gets their data in and sends it onward to the services that store and use it.

How the workflow runs

You register a device, often called a thing, in IoT Core and give it a certificate and a policy that says what it is allowed to do. The device uses that certificate to connect securely and publish messages to a topic. The rules engine matches messages on that topic and routes them to other AWS services. Optionally, the device and your app read and update the device shadow to sync state. As you grow, you can register and provision devices in bulk.

flowchart TD
    A[Register a thing and create its certificate] --> B[Attach a policy of allowed actions]
    B --> C[Device connects with the certificate over MQTT]
    C --> D[Device publishes messages to a topic]
    D --> E[Rules engine routes messages to AWS services]

Setting it up in the AWS Console

You will register a device, get its certificate, and send a test message using the built in MQTT client.

1. Sign in to the AWS Console, search for IoT Core, and open it. Check the region in the top right corner.
2. In the left menu, expand Manage, then Things, and click Create things, choosing to create a single thing. Give it a name and continue.
3. When asked about a certificate, choose to auto generate a new certificate. IoT Core creates the files that give this device its identity.
4. Download the device certificate, the private key, and the root CA file, and store them safely. The device uses these to connect, and the private key cannot be downloaded again later.
5. Attach a policy to the certificate that allows the IoT actions the device needs, such as connecting, publishing, and subscribing. For a first test you can create a simple policy that allows these actions, then finish creating the thing.
6. In the left menu open the MQTT test client. Subscribe to a topic such as test/device, then use the publish tab to send a small JSON message to the same topic. You should see the message appear in the subscription, which confirms messages are flowing through IoT Core.
7. To route real device data, go to Message routing and create a rule with a statement like selecting all fields from your topic, and choose an action such as sending the data to another service. This is how readings reach storage automatically. Common mistakes: if a device cannot connect, the certificate is usually not active or its policy does not allow the connect and publish actions, so check both. If messages publish but a rule does not fire, confirm the rule's topic filter matches the topic the device is publishing to.

Going live from code

Devices connect using the AWS IoT Device SDK. This minimal example connects with a certificate and publishes one reading over MQTT.

from awscrt import mqtt
from awsiot import mqtt_connection_builder
import json

connection = mqtt_connection_builder.mtls_from_path(
    endpoint="your-endpoint.iot.us-east-1.amazonaws.com",
    cert_filepath="device-certificate.pem.crt",
    pri_key_filepath="private-key.pem.key",
    ca_filepath="root-CA.pem",
    client_id="air-monitor-1",
)

connection.connect().result()

connection.publish(
    topic="test/device",
    payload=json.dumps({"pm25": 12.4}),
    qos=mqtt.QoS.AT_LEAST_ONCE,
)

print("Published one reading")

The endpoint comes from the Settings page in the IoT Core console, and the three files are the certificate and keys you downloaded when you created the thing.

Pricing

Item	Detail
Connectivity	Per million minutes devices are connected
Messaging	Per million messages sent and received
Rules engine	Per million rules triggered and actions run
Device shadow and registry	Per million operations on device state and records
Free tier	A monthly allowance of connectivity, messages, and operations for 12 months

The AWS IoT services family

AWS IoT Services
├── IoT Core            connect and route device messages securely
├── IoT Greengrass      run code and machine learning on the devices
├── IoT SiteWise        collect and model industrial equipment data
├── IoT Device Defender audit and monitor device security
└── FreeRTOS            an operating system for small microcontrollers

IoT Core is the connectivity hub at the center. Greengrass extends AWS onto the devices themselves so they can act without a round trip to the cloud. SiteWise focuses on industrial equipment data. Most IoT projects on AWS start with IoT Core and add the others as needs grow.

Wrapping up

IoT Core is the secure, scalable hub between your devices and the cloud. It handles connection, per device identity, and routing, so a fleet of thousands or millions can send data and receive commands without you running the messaging infrastructure. That wraps our tour of AWS Hidden Gems.

Series progress

This was the final stop in AWS Hidden Gems. The full series:

1. AWS Elemental MediaConvert
2. Amazon IVS
3. Amazon Rekognition
4. Amazon Personalize
5. AWS AppSync
6. Amazon Timestream
7. Amazon Textract
8. Amazon Kendra
9. AWS DataSync
10. AWS IoT Core (you are here) Thanks for following the whole series. Each of these services solves a real problem well, and now you know when to reach for them.

Let's connect

Questions, corrections, or want to talk through where any of these fit in your own project? Reach me at [e-mail].