Managing Per-Device Configuration with an AWS IoT Greengrass Fleet

Introduction

One of the best parts of working on AWS IoT Greengrass is the opportunity to talk to our customers and see all the varied use cases they have. Through these discussions we’ve identified some common patterns that I will share here with the wider IoT builder community.

In this first article, I’ll cover a couple approaches to manage a fleet of many devices using AWS IoT Thing Groups, but with per-device configuration.

Imagine that you are building a solution for ACME Corporation’s factories to collect sensor readings and upload them into AWS IoT Core using MQTT. There are 10 factories around the world and each factory has 10 lines and 10 cells in a line. In this example, there is 1 Greengrass core device per cell, so this means that you need to manage 10x10x10 = 1,000 Greengrass devices. Greengrass allows you to deploy to a single device or to a group of devices in an AWS IoT Thing Group. This means that you could manage all 1,000 devices uniquely, but that may be an unreasonable operational burden and you instead want to manage the 1,000 devices as a group.

Now that you decided to manage the devices as a single Thing Group, there is a problem because your solution requires that each device has some amount of unique configuration such as 1. what factory is it in, 2. which line, and 3. which workcell. These configurations are unlikely to ever change, and if they do change then it will be at a very low frequency (less than once per day for example). Other solutions would be more appropriate for high frequency configuration changes.

Solution

To solve the problem with treating all the devices as a group while needing unique configuration I suggest to create a “configuration holder” Greengrass component such as:

---
RecipeFormatVersion: "2020-01-25"
ComponentName: "ConfigHolder"
ComponentVersion: "1.0.0"
ComponentType: "aws.greengrass.generic"
ComponentDescription: "Holds configuration, does nothing."
ComponentPublisher: "ACME Corp"
ComponentConfiguration:
  DefaultConfiguration: {}
Manifests:
- Lifecycle: {}

This component will do absolutely nothing on its own, it only exists to hold onto the per-device configuration that you need in your solution. I will show two different ways to use this component, first is using it with a one-time per-device deployment and the second is to setup the component during Greengrass installation.

One-time Per-device Deployment

As mentioned before, you can deploy to Greengrass devices either individually or to a group of devices. With this solution, you will deploy the ConfigHolder component with the unique device configuration to the specific individual device that needs that configuration. At the same time, you will have a different Greengrass deployment to a group of devices which deploys your business logic components. Your business logic components will depend on the configuration holder deployment to provide the necessary unique configuration. Any shared configuration may go into the business logic components.

1. Create a deployment targeting the individual device

2. Add the ConfigHolder component - Important: Add only the ConfigHolder here and nothing else

   

3. Configure the ConfigHolder component with the unique configuration

   

4. Deploy this deployment and wait for it to complete successfully

Your device now has the ConfigHolder component properly configured with unique configurations, now we need to talk about how to actually use this configuration on the device.

As an example, I have created a business logic component helpfully named BusinessLogic as shown below:

---
RecipeFormatVersion: "2020-01-25"
ComponentName: "BusinessLogic"
ComponentVersion: "1.0.0"
ComponentType: "aws.greengrass.generic"
ComponentDescription: "Does the work"
ComponentPublisher: "ACME Corp"
ComponentConfiguration:
  DefaultConfiguration: {}
ComponentDependencies:
  ConfigHolder:
    VersionRequirement: ^1.0.0
Manifests:
- Lifecycle:
    Run: >-
      echo "Running with config from holder: {ConfigHolder:configuration:/workcell}"

This component has a dependency on the ConfigHolder component and uses interpolation to extract the unique configuration and use it.

Important: In this example, BusinessLogic is deployed to the Greengrass core device via a group deployment and ConfigHolder is deployed via an individual deployment. If the Greengrass core device is already in the thing group, the thing group deployment may execute before the individual deployment which means that BusinessLogic will execute when ConfigHolder hasn’t been configured.

There are 2 ways to address this issue, the most conceptually simple would be to ensure that the individual deployment completes before adding the device into the thing group which means that the thing group deployment will not execute until ConfigHolder is already configured. The more robust way to handle this is to have validation logic in your business logic which checks to see if the configuration is present and makes sense. If it does not make sense, then your business logic should not execute because it doesn’t have all the information it needs to execute correctly. Make sure that your business logic component does not exit with an error because this will make the thing group deployment fail and rollback (if configured to rollback), and you would then need to manually retry the deployment. When ConfigHolder is later deployed with the configuration, the BusinessLogic component will be restarted because the configuration that is interpolated into the run script has changed.

Individual device deployments utilize the IoT Thing’s Shadow in order to send the deployment information, you will be charged for the Shadow usage required to execute the deployment on each of your devices. You can avoid this cost and the deployment ordering issue mentioned above by trying the second way to use ConfigHolder, which is to configure it when installing Greengrass.

Configure ConfigHolder on Installation

With this approach, you will provide an initial configuration file to Greengrass during the installation which contains the ConfigHolder component and the desired unique configuration. This approach is not mutually exclusive to the individual device deployment described above, you may want to use this approach at first and then update the unique configuration over time using individual device deployments.

You may already use the initial configuration file during installation for port, proxy, or provisioning settings, but if not don’t worry, it is quite simple. When installing Greengrass, add the command line option --init-config initial-config.yaml this option can be combined with other options that you’re using such as --provision.

I will create a file: initial-config.yaml with the following contents:

services:
  ConfigHolder:
    componentType: "GENERIC"
    configuration:
      factory: "Madison"
      line: "1"
      workcell: "weld"
    dependencies: []
    lifecycle: {}
    version: "1.0.0"

Here I describe the ConfigHolder component that you’ve seen before along with the same configuration as before. When Greengrass finishes the installation, it will have this component as part of its configuration. It can then receive the thing group deployment and BusinessLogic will be able to pick up the configuration as before.

Configuration over IPC

In BusinessLogic, I’ve shown how configuration is interpolated into the recipe and that will work for any component. Now though, I will show how to use the configuration in a more “native” way by using Greengrass IPC to read configuration and subscribe to configuration changes in order to react without restarting the business logic component and without needing to interpolate the configuration into the recipe. I will also take this opportunity to show off a component using NodeJS as we now have a developer preview version of Greengrass IPC for Node.

In the code below I use the developer preview version of Greengrass IPC for NodeJS in order to

1. Get the configuration from ConfigHolder
2. Subscribe to changes in configuration in ConfigHolder
3. Get the configuration from ConfigHolder again if any of it ever changes

import { greengrasscoreipc } from 'aws-iot-device-sdk-v2';

const CONFIG_COMPONENT = "ConfigHolder";

async function main() {
    try {
        let client = greengrasscoreipc.createClient();

        await client.connect();

        const config = await client.getConfiguration({ componentName: CONFIG_COMPONENT, keyPath: [] });
        console.log("Got initial config", JSON.stringify(config.value));
        console.log("Subscribing to config changes");

        // Setup subscription handle
        const subscription_handle = client.subscribeToConfigurationUpdate({ componentName: CONFIG_COMPONENT, keyPath: [] });
        // Setup listener for config change events
        subscription_handle.on("message", async (event) => {
            console.log("Config changed, will pull full new config immediately", JSON.stringify(event.configurationUpdateEvent?.keyPath));

            const config = await client.getConfiguration({ componentName: CONFIG_COMPONENT, keyPath: [] });
            console.log("Got new full config", JSON.stringify(config.value));
        });

        // Perform the subscription
        await subscription_handle.activate();
        console.log("Subscribed to config changes");
    } catch (err) {
        console.log("Aw shucks: ", err);
    }
}

main();

The full component code is available here, see the readme in the repository for instructions to build and publish the component into your account.

Conclusion

I’ve covered multiple ways that you can use two components to have a mix of per-device and fleet-wide configurations. I hope that this blog is helpful to builders who want to simplify Greengrass fleet management while still allowing for per-device configurations as needed.

Follow me on GitHub and I look forward to your comments and suggestions for future topics in the comments!