This post was originally written on DevOpStar. Check it out here
The IoT Community in 2020 is weird.
There's a fair amount of hand waving and people talking about doing IoT but it's pretty hard to have a conversation with anyone about the specific services that under-pin the 3-letter acronym.
Because of this I would like to share my A-Z of IoT from the perspective of developing solutions on AWS. This guide doesn't include sections using the buzzwords you're probably a little tired of hearing about such as Digital Twin or Edge. Instead focuses on the Services & Patterns that take those buzzwords and turns them into reality.
I'll be upfront, It's pretty hard to find an IoT service, let alone AWS ones to meet every single letter in the alphabet. This list serves more as a way to highlight some of the pieces that make up the IoT service offering right now, and also things to keep an eye on moving into the future.
🇦 - Alexa Gadgets
When Amazon Alexa launched in 2014 as the first major home assistant platform I was captivated by the opportunities it offered someone like me. Clearly I wasn't alone either as over the coming years we saw a boom of new generation electronics incorporating virtual assistants in their sales pitch.
Now having voice assistant support is almost a must when launching any kind of home appliance, however this consumer demand is beginning to force hardware manufacturers to operate in a space that is very new and full of bad actors. Not all companies are technology first, so when their teams are asked to equip their white goods with internet capabilities there is a lot of room for error. This is why I believe Amazon Alexa Gadgets are a really big deal.
Gadgets convert an overly complex architecture to a simple bluetooth device removing a lot of technical junk that isn't required for the core functionality to deliver value for the end user.
Examples
- Alexa Gadgets Introduction - Voice Controlled Cat Feeder - Making a Voice controlled cat feeder using Alexa Gadgets.
🇧 - Billing Groups
Billing Groups within the context of IoT are very unique in AWS land. Typically you rely on billing under the account level, or optionally you can use tags and and cost allocation tags to better understand where cost is coming from.
IoT service
The idea is that groups of devices can be tagged and then audited independently. This means that teams can be held accountable for their own budgets.
Note: Billing groups within the context of a particular service is quite unique within AWS land. I've pondered why AWS IoT is special enough to get it's own and figure it might be something to do with organizations preferring to group all their IoT devices in one account.
🇨 - Certificate vending
If you've ever been through the process of setting up a new device with AWS IoT Core then you'll probably recall a sense of unease when creating new certificates for your devices.
I'm an advocate of allowing people to create and manage their own devices, however a set of guardrails are needed to ensure that a level of consistency is adhered to. Because of this I highly recommend looking into methods of performing certificate vending
Certificate vending is an automated process for generating X.509 certificates with correct roles and policies attached and handing these over to users safely.
Examples
- AWS IoT - Certificate Vending - Generating X.509 certificates automatically and attaching appropriate roles via an API gateway endpoint backed with DynamoDB.
🇩 - Device Defender Detect
Device Defender Detect helps you identify abnormal behaviour by interpreting cloud-side and device-side metrics from your IoT Devices. Detect relies on an agent running on the device to publish metrics on network usage statistics. These statistics when paired with the cloud-side stats; can begin to paint a picture of the security of a given device.
Some typical security incidents that Device Defender Detect might be able to report on are:
- Denial of Service attacks
- Participation in a Botnet
- Data exfiltration
- Incorrect device configuration
- Remote management ports open
Examples
- Greengrass - Device Defender - Detect - AWS IoT Device Defender provides a suite of options for Audit, Detection and Mitigation of IoT related security threats.
🇪 - IoT Enhanced Custom Authorizer | ⭐ FAVOURITE ⭐
The IoT Enhanced Custom Authorizer is a diamond in the rough. It's one of those services that solves a very difficult problem that's hard to articulate until you've been in a scenario where custom authorization for MQTT would be handy.
Currently this service is only in Beta in us-east-1 however I suspect as IoT adoption begins to increase, the need for this service to go GA will become more apparent.
I have a blog post planned on this topic coming soon, definitely follow me on Twitter if you're interested.
🇫 - Amazon FreeRTOS
One fantastic thing with the IoT space is there's just so much variation in device chips that can be used. This does make deploying software to each device a bit of a pain when every chip does things slightly differently or has more / less functionality.
A solution to this problem was FreeRTOS that provides common kernel for devices to build on. Amazon later began supporting its own version called Amazon FreeRTOS that helps you deploy embedded firmware that's Cloud Ready.
Amazon bundle and support common libraries that run on FreeRTOS such as WiFi, Bluetooth LE, MQTT and extend it to better integrate with Greengrass and other pieces of AWS Cloud.
🇬 - Greengrass | ⭐ FAVOURITE ⭐
AWS IoT Greengrass is a managed service that allows you to easily manage and communicate with edge devices. When you hear people talk about Edge devices, they probably mean something that will be running Greengrass eventually.
An Edge device is capable of handling incoming data from its surrounding sensors and then acuating, alerting, logging & responding to these metrics. Greengrass is the AWS service for managing this for you.
Greengrass will also manage versions of Lambda & Secrets Manager at the edge for you, so if your teams already make use of serverless architectures it is very easy to move that logic to the edge.
Greengrass is my favourite AWS service for too many reasons, and I've blogged about it a lot in the past so I recommend checking those out!
Examples
- Greengrass - Device Setup - Greengrass base setup using CDK.
- Greengrass - Device Setup - Raspberry Pi - Greengrass specific setup for Raspberry Pi.
- Greengrass - Docker Deployments - Using Greengrass Connectors for deploying docker configurations to the edge
- Greengrass - Device Defender - Detect - AWS IoT Device Defender provides a suite of options for Audit, Detection and Mitigation of IoT related security threats.
- Greengrass - Secure Tunneling - AWS IoT Secure Tunneling is a managed proxy meant for devices positioned behind secure firewalls on remote sites.
🇮 - Industrial IoT
I know I said I wouldn't use buzzwords, but Industrial IoT is a legitimate thing and there are solutions available in the AWS ecosystem to solve problems. I highly recommend this series of posts on AWS IoT SiteWise that uses an example of on OPC-UA server to ingest data streams from industrial systems.
- Collecting, organizing, monitoring, and analyzing industrial data at scale using AWS IoT SiteWise (Part 1)
- Collecting, organizing, monitoring, and analyzing industrial data at scale using AWS IoT SiteWise (Part 2)
- Collecting, organizing, monitoring, and analyzing industrial data at scale using AWS IoT SiteWise (Part 3)
🇯 - AWS IoT Jobs
AWS IoT Jobs help you deploy and manage tasks to be actioned by your fleet of IoT devices. Jobs underpin the Over the Air update service also mentioned later on in this list, however it is capable of executing any task you want on the IoT define.
Note: Your device code needs to be setup to handle incoming job requests from the AWS defined topics below. For more information check the developer guide
$aws/things/MyThing/jobs/notify
$aws/things/MyThing/jobs/get/accepted
$aws/things/MyThing/jobs/get/rejected
$aws/things/MyThing/jobs/jobId/get/accepted
$aws/things/MyThing/jobs/jobId/get/rejected
🇰 - Kinesis Streams
Although Amazon Kinesis isn't technically part of the AWS IoT service, it is heavily used by teams using AWS IoT due to its ability to ingest enormous streams of data and make sense of it.
A very common architecture for IoT projects is to leverage with Kinesis Data Analytics and do real-time eventing based on the trends. Below is an example architecture from AWS Solutions.
🇱 - AWS IoT Logging
By default AWS IoT doesn't log anything! This is done intentionally, however if you're a regular user like me you'll want to enable it to give better visibility.
You can enable AWS IoT Logging to CloudWatch by heading to the Settings in the appropriate AWS IoT region.
Edit the Logging settings and create / specify a role with the following policy. Note that the UI will step you through setting up this role automatically.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents",
"logs:PutMetricFilter",
"logs:PutRetentionPolicy"
],
"Resource": [
"arn:aws:logs:*:123456789012:log-group:*:log-stream:*"
]
}
]
}
... trust policy ...
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "iot.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
Now if you check CloudWatch under the log group called AWSIotLogsV2 you should see log streams.
🇲 - MQTT
MQTT is what underpins all communication by devices that use AWS IoT (or all IoT platforms for that matter). The IoT service that AWS provides is just an abstraction on top of the general architecture shown below.
The AWS version of MQTT enables many of the traditional things you would expect from AWS services to be integrated nicely such as eventing based on rules, IAM authorization and logging. Plus its a fully managed broker for you at basically no cost.
🇴 - Over-the-Air (OTA) Updates
Over-the-air (OTA) updates are crucial to get right, especially these days when patching a vulnerability or security flaw has to be done seamlessly.
AWS IoT Over-the-air updates leverage AWS IoT Jobs to deliver firmware or code updates to a device deployed in the field.
If you make use of AWS IoT Greengrass you would also be familiar that patches to Greengrass on devices are also done via an Over-the-air update (on top of IoT Jobs). Below is an example of how you could kick off a Greengrass core OTA update; the pattern is very similar for FreeRTOS OTA as well.
AWS_REGION="us-east-1"
ACCOUNT_ID=$(aws sts get-caller-identity | jq -r '.Account')
THING_NAME="thing_name"
THING_OS="raspbian" # ubuntu, amazon_linux, raspbian, openwrt
THING_ARCH="armv7l" # armv7l, armv6l, x86_64, aarch64
THING_SOFTWARE="core" # core, ota_agent
otaUpdateRole=$(aws cloudformation describe-stacks --stack-name greengrass-cdk \
--query 'Stacks[0].Outputs[?OutputKey==`otaUpdateRole`].OutputValue' \
--region ${AWS_REGION} \
--output text)
aws greengrass create-software-update-job \
--region ${AWS_REGION} \
--update-targets-architecture ${THING_ARCH} \
--update-targets arn:aws:iot:${AWS_REGION}:${ACCOUNT_ID}:thing/${THING_NAME} \
--update-targets-operating-system ${THING_OS} \
--software-to-update ${THING_SOFTWARE} \
--update-agent-log-level DEBUG \
--s3-url-signer-role ${otaUpdateRole}
🇵 - Policies
Policies are the way permissions are assigned to devices using AWS IoT certificates. A policy document is much like IAM policies in that it provides a set of permissions to do things with other AWS services
An example of a policy statement can be seen below and will apply permissions allowing an IoT device to subscribe to topics with the name format some/+/topic where the + is a wildcard
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iot:Connect"
],
"Resource": [
"arn:aws:iot:us-east-1:123456789012:client/${iot:Connection.Thing.ThingName}"
]
},
{
"Effect": "Allow",
"Action": [
"iot:Subscribe"
],
"Resource": [
"arn:aws:iot:us-east-1:123456789012:topicfilter/some/+/topic"
]
}
]
}
🇷 - AWS RoboMaker
AWS RoboMaker is a mostly managed service for deploying & running ROS packages to Robots. RoboMaker might appear niche at first glance however as the robotics industry expands, mechanisms to manage pushing software to devices in the field will be very important.
Not only does RoboMaker do ROS package deployments, but it will also do simulation for you by running managed Gazebo, rqt & rviz for you.
If you're a roboticist all of this should get you very excited!
Examples
- AWS RoboMaker - Beginner's Guide to Robot Simulation - Moderately good beginners guide to getting started with the entire RoboMaker suite
- AWS RoboMaker - Raspberry Pi Bootstrap - More beginner targeted with a focus on setting up ROS & RoboMaker on a Raspberry Pi.
🇸 - Device Shadow
Device Shadows are often exaggerated to be called a Digital Twin, however in reality its simply a mechanism to hold the state of a device.
The state is JSON formatted and can be read and wrote too by either the device or AWS IoT.
Examples
- AWS Sumerian Magic Cube - Device Shadows are used in order to store the orientation state of a device, then display this data visually using Amazon Sumerian
🇹 - Secure Tunneling | ⭐ FAVOURITE ⭐
If you've ever deployed an IoT device to the field and had to perform an OTA (Over-the-air update) you will know the fear of breaking core parts of our hardware device through a bad firmware deploy.
Imagine if there was a service that could be always running and capable of getting you access back into the device! AWS Secure Tunneling is a service that runs a localproxy and securely connects you back to your device.
Examples
- Greengrass - Secure Tunneling - A definitive guide to using the AWS IoT Secure Tunneling service.
🇻 - Computer Vision
Computer Vision is a very common pattern used on the Edge due to the bandwidth requirements and latency of streaming live video off to a remote location. You might be surprised to find that the AWS DeepLens runs as a Greengrass core device in order to manage the models deployed to it at the edge.
If you are planning on doing an IoT project using video at the edge, I would highly recommend checking out the ML Object Detection Connector for AWS IoT Greengrass.
🇼 - WebSockets
AWS IoT connections via MQTT can be provided over WebSockets as well as HTTPS. allows for a lot more flexibility of use in places like web browsers.
The Amplify framework has fantastic support for these websocket style connections via the MqttOverWSProvider. This means your web applications can connect directly to AWS IoT topics and publish / subscribe data.
Summary
Are you using AWS IoT, or doing something interesting with IoT in general? Well I'm very interested in hearing about it! Success or failure, please reach out to me on Twitter @nathangloverAUS and start a discussion.
Top comments (2)
Great Read!! Also I have something for the DevOps beginners : Comprehensive guide for DevOps
This is a great overview of the IoT Community in 2020. I particularly appreciate the section on Amazon FreeRTOS. I'm also interested in learning more about AWS IoT Jobs and Kinesis Streams. Overall, this is a valuable resource for anyone who is interested in learning more about the IoT space.
findtheneedle.co.uk/companies/inte...