DEV Community: Nenad Ilic

Developing Your First Greengrass Publish Component on Raspberry Pi

Nenad Ilic — Thu, 20 Jul 2023 14:13:19 +0000

In the previous blog post, we delved into the world of AWS IoT Greengrass V2, focusing on the usage of the Greengrass Development Kit (GDK) and how it can be automated with GitHub Actions. Today, we're going to take it a step further. We'll walk through the process of developing your first AWS IoT Greengrass Publish component on a Raspberry Pi.

Step 1: Setting Up Your Environment

Before we start, ensure that you have your Raspberry Pi is set up by using the Fleet Provisioning mehtod as we will be using now those GitHub Actions and the project we created in our previous blog post to deploy our component. In scenarios that you are working with a different OS, you would need to follow the instruction on installing AWS IoT Greengrass as well.

Step 2: Designing the Message Publisher Component

The first step in creating our messaging system is to create the message publisher component. This component will be responsible for sending data, such as sensor readings or application logs, to a specified topic.

In the AWS IoT Greengrass, components are defined by a recipe, which is a JSON or YAML file that specifies the component's metadata, lifecycle, configuration, and dependencies. For our message publisher component, we'll need to define a recipe that includes the necessary AWS IoT Greengrass component to publish messages to a topic.

Here's an example of what the recipe might look like:

---
RecipeFormatVersion: "2020-01-25"
ComponentName: "{COMPONENT_NAME}"
ComponentVersion: "{COMPONENT_VERSION}"
ComponentDescription: "A component that publishes temperature data to AWS IoT Core"
ComponentPublisher: "{COMPONENT_AUTHOR}"
ComponentConfiguration:
  DefaultConfiguration:
    accessControl:
      aws.greengrass.ipc.mqttproxy:
        'com.example.pub:mqttproxy:1':
          policyDescription: Allows access to publish the temperature to topic.
          operations:
            - aws.greengrass#PublishToTopic
          resources:
            - 'CPU/info'
Manifests:
  - Platform:
      os: all
    Artifacts:
      - URI: "s3://BUCKET_NAME/COMPONENT_NAME/COMPONENT_VERSION/com.example.pub.zip"
        Unarchive: ZIP
    Lifecycle:
      Run: "python3 -u {artifacts:decompressedPath}/com.example.pub/main.py"

In this recipe, we're creating a component with the name com.example.pub. This component is different from our previous examples due to the inclusion of an accessControl: configuration. This configuration allows the component to publish messages to a specific MQTT topic.

In our case, the resource is set to CPU/info. This setting means that our component has permission to publish only to the CPU/info MQTT topic.

If you need the component to publish to multiple topics, you can extend the list of resources with additional topic names. Alternatively, if you want the component to have permission to publish to any topic, you can replace CPU/info with *. This wildcard character represents all possible topics, granting the component full publishing access.

Step 3: Implementing the Message Publisher Component

With the component designed, we can now move on to implementing the message publisher component. This involves writing the main.py script that we referenced in the component recipe.

The main.py script will use the AWS SDK for Python which uses awsiot.greengrasscoreipc.clientv2 to interact with Greengrass IPC in order to publish messages to a specified AWS IoT topic. Here's an example of what the script would look like:

import os
import time
import json
import awsiot.greengrasscoreipc.clientv2 as clientV2

TOPIC="CPU/info"

def get_cpu_temp():
    temp_file = open("/sys/class/thermal/thermal_zone0/temp")
    cpu_temp = temp_file.read()
    temp_file.close()
    return float(cpu_temp)/1000

def main():
    # Create an IPC client.
    ipc_client = clientV2.GreengrassCoreIPCClientV2()

    while True:
        cpu_temp = get_cpu_temp()
        print("CPU temperature: {:.2f} C".format(cpu_temp))

        # Create a payload.
        payload = json.dumps({"temperature": cpu_temp})

        # Publish the payload to AWS IoT Core.
        resp = ipc_client.publish_to_iot_core(topic_name=TOPIC, qos="1", payload=payload)

        time.sleep(1)  # sleep for 1 second

    ipc_client.close()

if __name__ == "__main__":
    main()

In this script, we're creating a loop that continuously publishes a message to the CPU/info topic. The message contains a json payload with temperature value, which could be then extended further in a real-world application.

Step 4: Deploying the Message Publisher Component

For deploying the new component to the device we should extend the deployment.json.template:

{
    "targetArn": "arn:aws:iot:$AWS_REGION:$AWS_ACCOUNT_ID:thinggroup/$THING_GROUP",
    "deploymentName": "Main deployment",
    "components": {
        "com.example.hello": {
            "componentVersion": "LATEST",
            "runWith": {}
        },
        "com.example.world": {
            "componentVersion": "LATEST",
            "runWith": {}
        },
        "com.example.pub": {
            "componentVersion": "LATEST",
            "runWith": {}
        }
    },
    "deploymentPolicies": {
        "failureHandlingPolicy": "ROLLBACK",
        "componentUpdatePolicy": {
            "timeoutInSeconds": 60,
            "action": "NOTIFY_COMPONENTS"
        },
        "configurationValidationPolicy": {
            "timeoutInSeconds": 60
        }
    },
    "iotJobConfiguration": {}
}

If you've followed our previous blog post, you should have your GitHub Actions set up for your repository. This setup allows for automatic deployment of your components.

When you create a new component and commit it to your repository, make sure to add it to the deployment.json.template. This step is crucial as it ensures your component is included in the deployment process.

After committing the new component, the GitHub Actions workflow will trigger, resulting in the automatic deployment of your component to the targeted device, in this case, a Raspberry Pi.

Once deployed, the component will start running on the Raspberry Pi. It will begin publishing messages to the specified AWS IoT topic.

To verify that your component is functioning correctly, you can subscribe to the topic in the AWS IoT Core console. Here, you'll be able to observe the incoming messages, confirming that your component is publishing as expected.

Conclusion

In this blog post, we've walked through the process of developing a Greengrass Publish component on a Raspberry Pi. This is a great way to use the messaging system for your IoT applications, and I hope it helps you on your journey with AWS IoT Greengrass.

For reference, please refer to this GitHub repo.
Stay tuned for more posts on advanced Greengrass component development and other IoT topics. Happy coding!

If you have any feedback about this post, or you would like to see more related content, please reach out to me here, or on Twitter or LinkedIn.

AWS IoT Greengrass Components Deployment with GDK using GitHub Actions

Nenad Ilic — Tue, 06 Jun 2023 11:01:34 +0000

As IoT deployments grow, a set of distinct challenges arise when it comes application consistency across a broad spectrum of devices: efficiently managing update rollbacks, and maintaining comprehensive, auditable change logs can all present significant obstacles. However, these challenges can be effectively managed using AWS IoT Greengrass and the Greengrass Development Kit (GDK).

AWS IoT Greengrass enables the deployment of applications directly to your IoT devices, simplifying the management and control of device-side software. The GDK further enhances this capability by streamlining the configuration and packaging of these applications for deployment, making it easier to get your applications onto your devices.

To create a robust and efficient system, we can also introduce GitOps methodologies. GitOps leverages version control, automated deployments, and continuous monitoring to improve the reliability and efficiency of your deployment processes. By using these methodologies with GitHub Actions, we can automate the deployment process, triggering it with every commit or merge to a specified branch.

In this blog post, we'll explore how these technologies and methodologies can be combined to create a powerful, scalable, and automated IoT deployment system. We'll walk through the setup of the GDK, the definition and deployment of Greengrass components, and the setup of a GitHub Actions workflow to automate the entire process.

Let’s dive into and get started with this setup.

Overview

The AWS IoT Greengrass Development Kit Command-Line Interface (GDK CLI) is an open-source tool designed to streamline the creation, building, and publishing of custom Greengrass components. It simplifies the version management process, allows starting projects from templates or community components, and can be customized to meet specific development needs.

It can also be effectively utilized with GitHub Actions to automate the process of building and publishing Greengrass components. This could be a crucial part of a Continuous Integration/Continuous Deployment (CI/CD) pipeline. Here's a general outline of how this could be done:

Install Dependencies: Create a GitHub Actions workflow file (e.g., .github/workflows/main.yml) and start with a job that sets up the necessary environment. This includes installing Python and pip (since GDK CLI is a Python tool), AWS CLI, and the GDK CLI itself.
Configure AWS Credentials: Use GitHub Secrets to securely store your AWS credentials (Access Key ID and Secret Access Key). In your workflow, configure AWS CLI with these credentials so that the GDK CLI can interact with your AWS account.
Build and Publish Components: Use GDK CLI commands in your workflow to build and publish your components. For example, you might have steps that run commands like gdk component build and gdk component publish.
Integrate with Other Workflows: If you have other workflows in your CI/CD pipeline (such as running tests or deploying to other environments), you can use the output of the GDK CLI commands as inputs to these workflows.

In this way, every time you push a change to your Greengrass component source code on GitHub, the GDK CLI can automatically build and publish the updated component, ensuring that your Greengrass deployments are always using the latest version of your components.

Furthermore, we can initiate a Greengrass deployment based on the deployment template in the next stage of the pipeline, after a successful build and publish. This can target a specific Thing Group, enabling us to reflect changes across a fleet of devices. With this approach, if we have dev and test branches, each of these can be mapped to selected Thing Groups. This allows us to perform field validation on selected devices, providing an efficient way to test changes in a controlled environment before wider deployment.

For further details on creating GitHub Actions workflows, refer to the GitHub Actions documentation. For more information about using the GDK CLI, please refer to the GDK CLI documentation.

Development Project Setup

Based on the high level overview we can structure our project as such:

project
├── .github
│   └── workflows
│       └── main.yml
├── cfn
│   └── github-oidc
│       ├── oidc-provider.yaml
│       └── oidc-role.yaml
└── components
    ├── com.example.hello
    │   ├── gdk-config.json
    │   ├── main.py
    │   └── recipe.yaml
    ├── com.example.world
    │   ├── gdk-config.json
    │   ├── main.py
    │   └── recipe.yaml
    └── deployment.json.template

Where we have the following:

.github/workflows/main.yml: Which is the GitHub Actions workflow file where the CI/CD pipeline is defined. The GDK CLI and AWS CLI setup, component building, publishing, and deployment tasks are defined here.
cfn/github-oidc: This directory contains AWS CloudFormation templates (oidc-provider.yaml and oidc-role.yaml) that are used to set up an OIDC provider and role on AWS for authenticating GitHub Actions with AWS.
components: This directory contains the Greengrass components (com.example.hello and com.example.world) that you are developing. Each component has its own directory with:
- gdk-config.json: This is the configuration file for the GDK CLI for the specific component.
- main.py: This is the main Python script file for the component's functionality.
- recipe.yaml: This is the component recipe that describes the component and its dependencies, lifecycle scripts, etc.
deployment.json.template: This is a deployment template file for Greengrass deployments. It is used to generate the actual deployment file (deployment.json) that is used when initiating a Greengrass deployment

GitHub Actions and GDK Deployment Role Setup

The CloudFormation template will be used to create an IAM role (oidc-gdk-deployment) that provides the necessary permissions for building and deploying Greengrass components using GDK CLI and GitHub Actions. The role has specific policies attached that allow actions such as describing and creating IoT jobs, interacting with an S3 bucket for Greengrass component artifacts, and creating Greengrass components and deployments.

AWSTemplateFormatVersion: 2010-09-09
Description: 'GitHub OIDC:| Stack: oidc'

Parameters:
  FullRepoName:
    Type: String
    Default: example/gdk-example

Resources:
  Role:
    Type: AWS::IAM::Role
    Properties:
      RoleName: oidc-gdk-deployment
      Policies:
        - PolicyName: iot-thing-group
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - iot:DescribeThingGroup
                  - iot:CreateJob
                Resource:
                  - !Sub arn:aws:iot:${AWS::Region}:${AWS::AccountId}:thinggroup/*
        - PolicyName: iot-jobs
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - iot:DescribeJob
                  - iot:CreateJob
                  - iot:CancelJob
                Resource:
                  - !Sub arn:aws:iot:${AWS::Region}:${AWS::AccountId}:job/*
        - PolicyName: s3-greengrass-bucket
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - s3:CreateBucket
                  - s3:GetBucketLocation
                  - s3:ListBucket
                Resource:
                  - !Sub arn:aws:s3:::greengrass-component-artifacts-${AWS::Region}-${AWS::AccountId}
        - PolicyName: s3-greengrass-components
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - s3:GetObject
                  - s3:PutObject
                Resource:
                  - !Sub arn:aws:s3:::greengrass-component-artifacts-${AWS::Region}-${AWS::AccountId}/*
        - PolicyName: greengrass-components
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - greengrass:CreateComponentVersion
                  - greengrass:ListComponentVersions
                Resource:
                  - !Sub arn:aws:greengrass:${AWS::Region}:${AWS::AccountId}:components:*
        - PolicyName: greengrass-deployment
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - greengrass:CreateDeployment
                Resource:
                  - !Sub arn:aws:greengrass:${AWS::Region}:${AWS::AccountId}:deployments
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Action: sts:AssumeRoleWithWebIdentity
            Principal:
              Federated: !Sub arn:aws:iam::${AWS::AccountId}:oidc-provider/token.actions.githubusercontent.com
            Condition:
              StringLike:
                token.actions.githubusercontent.com:sub: !Sub repo:${FullRepoName}:*

Outputs:
  OidcRoleAwsAccountId:
    Value: !Ref AWS::AccountId
  OidcRoleAwsRegion:
    Value: !Ref AWS::Region
  OidcRoleAwsRoleToAssume:
    Value: !GetAtt Role.Arn

The FullRepoName parameter is used to specify the repository that the GitHub Actions workflow will be running in. This is important for the sts:AssumeRoleWithWebIdentity action in the AssumeRolePolicyDocument, which allows GitHub Actions to assume this IAM role for the specified repository.

To deploy this CloudFormation stack, you would use the AWS Management Console, AWS CLI, or an AWS SDK. You would need to specify the FullRepoName parameter as an input when you create the stack. For example, with the AWS CLI, you would use the aws cloudformation deply command and provide the template file and the FullRepoName parameter as inputs:

aws cloudformation deploy \
    --template-file cfn/github-oidc/oidc-role.yaml \
    --stack-name ga-gdk-role \
    --capabilities CAPABILITY_NAMED_IAM \
    --parameter-overrides FullRepoName=<your org>/<your repo name>

This setup lays the foundation for the GitHub Actions and GDK CLI to work together to automate the building and deployment of Greengrass components.

In scenario that you would like to use OIDC Provider for GitHub actions (suggested) you would also need to set it up in you AWS account. Please not that this is needed only once per account region:

aws cloudformation deploy \
    --template-file cfn/github-oidc/oidc-provider.yaml \
    --stack-name oidc-provider

Now we can go and prepare our Greengrass components.

Greengrass Components GDK Setup

Here we focus here on the configuration and implementation of our Greengrass components. These components form the core of our IoT solution, providing the required functionality on our Greengrass core devices.

The gdk-config.json file is where we configure our Greengrass component.

{
    "component": {
      "com.example.hello": {
        "author": "Example",
        "version": "NEXT_PATCH",
        "build": {
          "build_system": "zip"
        },
        "publish": {
          "bucket": "greengrass-component-artifacts",
          "region": "eu-west-1"
        }
      }
    },
    "gdk_version": "1.2.0"
}

In the provided example, we have a single component named "com.example.hello". This file specifies the author, the build system (which is set to "zip" here), and the AWS S3 bucket details where the component will be published. The version field is set to "NEXT_PATCH", which means GDK will automatically increment the patch version of the component every time it's built.

The recipe.yaml file is the recipe for our Greengrass component.

---
RecipeFormatVersion: "2020-01-25"
ComponentName: "{COMPONENT_NAME}"
ComponentVersion: "{COMPONENT_VERSION}"
ComponentDescription: "This is a simple Hello World component written in Python."
ComponentPublisher: "{COMPONENT_AUTHOR}"
ComponentConfiguration:
  DefaultConfiguration:
    Message: "Hello"
Manifests:
  - Platform:
      os: all
    Artifacts:
      - URI: "s3://BUCKET_NAME/COMPONENT_NAME/COMPONENT_VERSION/com.example.hello.zip"
        Unarchive: ZIP
    Lifecycle:
      Run: "python3 -u {artifacts:decompressedPath}/com.example.hello/main.py {configuration:/Message}"

It contains essential metadata about the component, like its name, version, description, and publisher. It also specifies the default configuration, which, in this case, sets the default message to "Hello". The Manifests section describes the component's artifacts and the lifecycle of the component. In this instance, it specifies the location of the component's zipped artifacts and the command to run the component.

Finally, the main.py file is the Python script that our Greengrass component runs.

import sys

message = f"Hello, {sys.argv[1]}!"

# Print the message to stdout, which Greengrass saves in a log file.
print(message)

This script simply prints out a greeting message. The message is constructed from the argument passed to the script, which comes from the component configuration in the recipe.yaml file. This setup demonstrates how you can pass configuration to your Greengrass components, as we can see in the “com.example.world” example where we provide “World” as a configuration message.

In addition to the component configuration and scripts, we also define a deployment.json.template file.

{
    "targetArn": "arn:aws:iot:$AWS_REGION:$AWS_ACCOUNT_ID:thinggroup/$THING_GROUP",
    "deploymentName": "Main deployment",
    "components": {
        "com.example.hello": {
            "componentVersion": "LATEST",
            "runWith": {}
        },
        "com.example.world": {
            "componentVersion": "LATEST",
            "runWith": {}
        }
    },
    "deploymentPolicies": {
        "failureHandlingPolicy": "ROLLBACK",
        "componentUpdatePolicy": {
            "timeoutInSeconds": 60,
            "action": "NOTIFY_COMPONENTS"
        },
        "configurationValidationPolicy": {
            "timeoutInSeconds": 60
        }
    },
    "iotJobConfiguration": {}
}

This file specifies the deployment configuration for our Greengrass components. The targetArn is the Amazon Resource Name (ARN) of the Greengrass thing group where we aim to deploy our components. In this example, we're deploying two components, com.example.hello and com.example.world, both set to use their latest versions. The deploymentPolicies section sets the policies for failure handling, component update, and configuration validation. This file is vital as it governs how the deployment of our Greengrass components is handled in the target IoT devices.

Please note that this is a template and we will be using this in our pipeline to replace the ARN accordingly.

Taken together, these files form the basis of a Greengrass component. By modifying these templates and scripts, you can create your own custom Greengrass components with GDK. The next step is to set up GitHub Actions to automate the build and deployment of these components.

GitHub Actions Setup

This GitHub workflow file sets up two jobs, namely publish and deploy, which are run when either a push to the main branch occurs or the workflow is manually dispatched.

1. Publish Job

The publish job runs on the ubuntu-latest environment.

jobs:
  publish:
    name: Component publish
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read

    steps:
    - name: Checkout
      uses: actions/checkout@v3
      with:
        fetch-depth: 0
        ref: ${{ github.head_ref }}
    - uses: actions/setup-python@v3
      with:
        python-version: '3.9'

    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v1
      with:
        role-to-assume: ${{ secrets.OIDC_ROLE_AWS_ROLE_TO_ASSUME }}
        aws-region: ${{ secrets.OIDC_ROLE_AWS_REGION }}

    - name: Install Greengrass Development Kit
      run: pip install -U git+https://github.com/aws-greengrass/aws-greengrass-gdk-cli.git@v1.2.3

    - name: GDK Build and Publish
      id: build_publish
      run: |

        CHANGED_COMPONENTS=$(git diff --name-only HEAD~1 HEAD | grep "^components/" | cut -d '/' -f 2)

        echo "Components changed -> $CHANGED_COMPONENTS"        

        for component in $CHANGED_COMPONENTS
        do
          cd $component
          echo "Building $component ..."
          gdk component build
          echo "Publishing $component ..."
          gdk component publish
          cd ..
        done

      working-directory: ${{ env.working-directory }}

We start with checking out the code from the repository and setting up Python 3.9. The AWS credentials are then configured using the aws-actions/configure-aws-credentials@v1 action. The credentials used here are fetched from the GitHub secrets OIDC_ROLE_AWS_ROLE_TO_ASSUME and OIDC_ROLE_AWS_REGION. The OIDC_ROLE_AWS_ROLE_TO_ASSUME secret should contain the ARN of the AWS role that the GitHub Actions should assume when executing the workflow. This is the role we created in the firs step we can obtain it by executing the following:

aws cloudformation describe-stacks --stack-name ga-gdk-role --query 'Stacks[0].Outputs[?OutputKey==`OidcRoleAwsRoleToAssume`].OutputValue' --output text

The OIDC_ROLE_AWS_REGION secret should contain the AWS region where your resources are located. After that these variables needs to be added under github.com/<org>/<repo>/settings/secrets/actions .

Next, the Greengrass Development Kit (GDK) is installed using pip. The GDK CLI is used to build and publish any components that have changed between the current and previous commit.

The changed components are identified by looking at the differences between the current and previous commit and extracting the component names. Here it is important that the name of the folder mathces the name of the component.

2. Deploy Job

The deploy job runs after the publish job has completed successfully.

  deploy:
    name: Component deploy
    runs-on: ubuntu-latest
    needs: publish
    permissions:
      id-token: write
      contents: read

    steps:
    - name: Checkout
      uses: actions/checkout@v3
      with:
        fetch-depth: 0
        ref: ${{ github.head_ref }}

    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v1
      with:
        role-to-assume: ${{ secrets.OIDC_ROLE_AWS_ROLE_TO_ASSUME }}
        aws-region: ${{ secrets.OIDC_ROLE_AWS_REGION }}

    - name: Deploy Greengrass components
      run: |
        export AWS_ACCOUNT_ID=$(aws sts get-caller-identity |  jq -r '.Account')
        export AWS_REGION=${GREENGRASS_REGION}
        # Thing Group is the name of the branch
        export THING_GROUP=${GITHUB_REF#refs/heads/}

        CHANGED_COMPONENTS=$(git diff --name-only HEAD~1 HEAD | grep "^components/" | cut -d '/' -f 2)

        if [ -z "$CHANGED_COMPONENTS" ]; then
          echo "No need to update deployment"
        else
          envsubst < "deployment.json.template" > "deployment.json"

          for component in $CHANGED_COMPONENTS
          do
            version=$(aws greengrassv2 list-component-versions \
              --output text \
              --no-paginate \
              --arn arn:aws:greengrass:${AWS_REGION}:${AWS_ACCOUNT_ID}:components:${component} \
              --query 'componentVersions[0].componentVersion')

            jq '.components[$component].componentVersion = $version' --arg component $component --arg version $version deployment.json > "tmp" && mv "tmp" deployment.json

          done

          # deploy
          aws greengrassv2 create-deployment \
            --cli-input-json file://deployment.json \
            --region ${AWS_REGION}

          echo "Deployment finished!"
        fi

      working-directory: ${{ env.working-directory }}

It follows a similar structure to the publish job, starting with checking out the code from the repository and configuring AWS credentials using the same secrets.

In the deployment step, it first identifies any changed components in a similar way as in the publish job. If no components have changed, it does not proceed with deployment. If there are changed components, it prepares a deployment.json file from the template, replacing placeholders with the actual values. It then gets the version of the changed components from AWS Greengrass and updates the deployment.json file with these versions.

Finally, it creates a deployment using the aws greengrassv2 create-deployment command, providing the deployment.json file as input and setting the region to the one specified in the AWS_REGION environment variable.
Here it is important to note that Thing Group is taken from the name of the branch THING_GROUP=${GITHUB_REF#refs/heads/} as that way we can have different branches related to different Thing Groups as discussed above. In case the thing group is not create you can use the following command:

aws iot create-thing-group --thing-group-name main

At this point, when ever there is a new commit on the main branch a process will kick start and issue a deployment to the specified group of devices.

Conclusion

In this blog post, we've taken a deep dive into AWS IoT Greengrass V2, focusing on the usage of the Greengrass Development Kit (GDK) and how it can be automated with GitHub Actions. We started by setting up the GDK, explored the various components and how they interact, then we moved on to setting up a GitHub Actions workflow to automate the entire process.

By leveraging AWS IoT Greengrass, the GDK, and GitHub Actions, you can create a powerful, scalable, and automated IoT solution. Whether you're managing a small group of IoT devices or a large fleet, this approach offers a robust and efficient way to handle your IoT application deployments.

That's all for this blog post. We hope you found it informative and that it helps you on your journey to creating and managing IoT solutions with AWS IoT Greengrass. Happy coding!

All the above code and setup can be referenced here:
https://github.com/aws-iot-builder-tools/greengrass-continuous-deployments

If you have any feedback about this post, or you would like to see more related content, please reach out to me here, or on Twitter or LinkedIn.

Fleet Provisioning for Embedded Linux Devices with AWS IoT Greengrass

Nenad Ilic — Thu, 04 May 2023 16:48:17 +0000

Introduction

Managing a large fleet of embedded devices can be complex and challenging, particularly when it comes to creating a single image that can be flashed onto multiple devices. These devices must be able to self-provision, utilizing unique information such as their serial number, upon initial boot. In this blog post, we will discuss how AWS IoT Greengrass - Fleet Provisioning can streamline this process for embedded Linux devices, making it more efficient and reliable.

For embedded systems engineers experienced in Embedded Linux and Yocto, we will guide you through building a Raspberry Pi Yocto image with Greengrass with Fleet Provisioning Plugin. This ensures seamless device provisioning and management, as well as automatic registration and configuration.

Please note here that the pre-provisioning lambda is optional but encouraged in order to additional layer of security. We will not be covering it in this post. You can learn more about it here.

With the stage set, let's dive into the prerequisites for setting up AWS IoT Greengrass and Fleet Provisioning for your embedded Linux devices.

Prerequisites

Before diving into the process of preparing the host and configuring the Yocto image build, it's essential to set up AWS IoT Core. This involves creating policies, obtaining claim certificates, and ensuring that the AWS CLI is installed and configured. General information on how to accomplish this can be found in the AWS IoT Greengrass Developer Guide.
In summary, we will need to:

A token exchange IAM role, which core devices use to authorize calls to AWS services and An AWS IoT role alias that points to the token exchange role.
An AWS IoT fleet provisioning template. The template must specify information needed for creating thing and policy which will be attached to greengrass core device created. You can either use existing IoT policy name or define the policy on the template.
An AWS IoT provisioning claim certificate and private key for the fleet provisioning template.

Devices can be manufactured with a provisioning claim certificate and private key embedded in them. When the device connects first time to AWS IoT, it uses the claim certificate to register the new device and exchange it to unique device certificate. Provisioning claim certificate needs to have AWS IoT policy attached which allows devices to register and use the fleet provisioning template.

To make this process more efficient, we can utilize a CloudFormation template that automates most of these steps:

AWSTemplateFormatVersion: "2010-09-09"

Parameters:
  ProvisioningTemplateName:
    Type: String
    Default: 'GreengrassFleetProvisioningTemplate' 
  GGTokenExchangeRoleName:
    Type: String
    Default: 'GGTokenExchangeRole'
  GGFleetProvisioningRoleName:
    Type: String
    Default: 'GGFleetProvisioningRole'
  GGDeviceDefaultPolicyName:
    Type: String
    Default: 'GGDeviceDefaultIoTPolicy'
  GGProvisioningClaimPolicyName:
    Type: String
    Default: 'GGProvisioningClaimPolicy'

Resources:

  GGTokenExchangeRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Ref GGTokenExchangeRoleName
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - credentials.iot.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      Path: '/'
      Policies:
        - PolicyName: !Sub ${GGTokenExchangeRoleName}Access
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: Allow
                Action:
                  - 'iot:DescribeCertificate'
                  - 'logs:CreateLogGroup'
                  - 'logs:CreateLogStream'
                  - 'logs:PutLogEvents'
                  - 'logs:DescribeLogStreams'
                  - 's3:GetBucketLocation'
                Resource: '*'

  GGTokenExchangeRoleAlias:
    Type: AWS::IoT::RoleAlias
    Properties:
      RoleArn: !GetAtt GGTokenExchangeRole.Arn
      RoleAlias: !Sub ${GGTokenExchangeRoleName}Alias

  GGFleetProvisioningRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Ref GGFleetProvisioningRoleName
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - iot.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      Path: '/'
      ManagedPolicyArns:
        - 'arn:aws:iam::aws:policy/service-role/AWSIoTThingsRegistration'

  GGDeviceDefaultPolicy:
    Type: AWS::IoT::Policy
    Properties:
      PolicyName: !Ref GGDeviceDefaultPolicyName
      PolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Action:
            - 'iot:Connect'
            - 'iot:Publish'
            - 'iot:Subscribe'
            - 'iot:Receive'
            - 'iot:Connect'
            - 'greengrass:*'
          Resource: '*'
        - Effect: Allow
          Action:
            - 'iot:AssumeRoleWithCertificate'
          Resource: !GetAtt GGTokenExchangeRoleAlias.RoleAliasArn

  GGFleetProvisionTemplate:
    Type: AWS::IoT::ProvisioningTemplate
    Properties:
      TemplateName: !Ref ProvisioningTemplateName
      Description: 'Fleet Provisioning template for AWS IoT Greengrass.'
      Enabled: True
      ProvisioningRoleArn: !GetAtt GGFleetProvisioningRole.Arn
      TemplateBody: !Sub |+ 
        {
          "Parameters": {
            "ThingName": {
              "Type": "String"
            },
            "ThingGroupName": {
              "Type": "String"
            },
            "AWS::IoT::Certificate::Id": {
              "Type": "String"
            }
          },
          "Resources": {
            "GGThing": {
              "OverrideSettings": {
                "AttributePayload": "REPLACE",
                "ThingGroups": "REPLACE",
                "ThingTypeName": "REPLACE"
              },
              "Properties": {
                "AttributePayload": {},
                "ThingGroups": [
                  {
                    "Ref": "ThingGroupName"
                  }
                ],
                "ThingName": {
                  "Ref": "ThingName"
                }
              },
              "Type": "AWS::IoT::Thing"
            },
            "GGDefaultPolicy": {
              "Properties": {
                "PolicyName": "${GGDeviceDefaultPolicyName}"
              },
              "Type": "AWS::IoT::Policy"
            },
            "GGCertificate": {
              "Properties": {
                "CertificateId": {
                  "Ref": "AWS::IoT::Certificate::Id"
                },
                "Status": "Active"
              },
              "Type": "AWS::IoT::Certificate"
            }
          }
        }

  GGProvisioningClaimPolicy:
    Type: AWS::IoT::Policy
    Properties:
      PolicyName: !Ref GGProvisioningClaimPolicyName
      PolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Action:
            - 'iot:Connect'
          Resource: '*'
        - Effect: Allow
          Action:
            - 'iot:Publish'
            - 'iot:Receive'
          Resource: 
            - !Sub 'arn:aws:iot:${AWS::Region}:${AWS::AccountId}:topic/$aws/certificates/create/*'
            - !Sub 'arn:aws:iot:${AWS::Region}:${AWS::AccountId}:topic/$aws/provisioning-templates/${ProvisioningTemplateName}/provision/*'
        - Effect: Allow
          Action:
            - 'iot:Subscribe'
          Resource:
            - !Sub 'arn:aws:iot:${AWS::Region}:${AWS::AccountId}:topicfilter/$aws/certificates/create/*'
            - !Sub 'arn:aws:iot:${AWS::Region}:${AWS::AccountId}:topicfilter/$aws/provisioning-templates/${ProvisioningTemplateName}/provision/*'

Outputs:

  GGTokenExchangeRole:
    Description: Name of token exchange role.
    Value: !Ref GGTokenExchangeRole
  GGTokenExchangeRoleAlias:
    Description: Name of token exchange role alias.
    Value: !Ref GGTokenExchangeRoleAlias
  GGFleetProvisionTemplate:
    Description: Name of Fleet provisioning template.
    Value: !Ref GGFleetProvisionTemplate
  GGProvisioningClaimPolicy:
     Description: Name of claim certificate IoT policy.
     Value: !Ref GGProvisioningClaimPolicy

Save the file and create CloudFormation stack from template.yaml:

aws cloudformation create-stack --stack-name GGFleetProvisoning --template-body file://gg-fp.yaml --capabilities CAPABILITY_NAMED_IAM

Wait few minutes for resources being created. You can check status from CloudFormation console or with command:

aws cloudformation describe-stacks --stack-name GGFleetProvisoning

Create claim certificate

These we will be embedded in our RPi SD Card Image and used to provision our devices.

mkdir claim-certs

export CERTIFICATE_ARN=$(aws iot create-keys-and-certificate \
    --certificate-pem-outfile "claim-certs/claim.cert.pem" \
    --public-key-outfile "claim-certs/claim.pubkey.pem" \
    --private-key-outfile "claim-certs/claim.pkey.pem" \
    --set-as-active \
    --query certificateArn)

curl -o "claim-certs/claim.root.pem" https://www.amazontrust.com/repository/AmazonRootCA1.pem

Attach the AWS IoT policy to the provisioning claim certificate

As we created IoT policy named GGProvisioningClaimPolicy with CloudFormation we can just use the name to attach the policy:

aws iot attach-policy --policy-name GGProvisioningClaimPolicy --target ${CERTIFICATE_ARN//\"}

Create a Thing Group

Once our devices get provisioned they will become part of this Thing Group allowing us later to target Thing Group Fleet Deployments.

aws iot create-thing-group --thing-group-name EmbeddedLinuxFleet

As of now we should be good to go and build our RPI image.

Building RPi Image

Building a Yocto image for Raspberry Pi requires several steps, including setting up the build environment, cloning the necessary repositories, configuring the build, and finally, building the image itself. Here's a step-by-step guide to help you through the process:

Open a terminal window on your workstation which has all the prerequisits based on the Yocto Project Build Doc.

NOTE For the sake of this tutorial, the variable BASE refers to the build environment parent directory. Here, this will be set to $HOME. If you are using another partition as the base directory, please set it accordingly.

export BASEDIR=$(pwd)
export DIST=poky-rpi4
export B=kirkstone

Clone the Poky base layer to include OpenEmbedded Core, Bitbake, and so forth to seed the Yocto build environment.

git clone -b $B git://git.yoctoproject.org/poky.git $BASEDIR/$DIST

Clone additional dependent repositories. Note that we are cloning only what is required for AWS IoT Greengrass.

git clone -b $B git://git.openembedded.org/meta-openembedded \
    $BASEDIR/$DIST/meta-openembedded
git clone -b $B git://git.yoctoproject.org/meta-raspberrypi \
    $BASEDIR/$DIST/meta-raspberrypi
git clone -b $B git://git.yoctoproject.org/meta-virtualization \
    $BASEDIR/$DIST/meta-virtualization
git clone -b $B https://github.com/aws4embeddedlinux/meta-aws \
    $BASEDIR/$DIST/meta-aws

Source the Yocto environment script. This seeds the build/conf directory.

cd $BASEDIR/$DIST
. ./oe-init-build-env

Add necessary layers to bblayers.conf using bitbake-layer add-layer:

bitbake-layers add-layer ../meta-openembedded/meta-oe
bitbake-layers add-layer ../meta-openembedded/meta-python
bitbake-layers add-layer ../meta-openembedded/meta-filesystems
bitbake-layers add-layer ../meta-openembedded/meta-networking
bitbake-layers add-layer ../meta-virtualization
bitbake-layers add-layer ../meta-raspberrypi
bitbake-layers add-layer ../meta-aws

Configure the local.conf:

Here it is important to note that apart from standard raspberry pi configuration:

MACHINE ?= "raspberrypi4-64"

DISABLE_VC4GRAPHICS = "1"

# Parallelism Options
BB_NUMBER_THREADS ?= "${@oe.utils.cpu_count()}"
PARALLEL_MAKE ?= "-j ${@oe.utils.cpu_count()}"

# Additional image features
USER_CLASSES ?= "buildstats"

# By default disable interactive patch resolution (tasks will just fail instead):
PATCHRESOLVE = "noop"

# Disk Space Monitoring during the build
BB_DISKMON_DIRS = "\
    STOPTASKS,${TMPDIR},1G,100K \
    STOPTASKS,${DL_DIR},1G,100K \
    STOPTASKS,${SSTATE_DIR},1G,100K \
    HALT,${TMPDIR},100M,1K \
    HALT,${DL_DIR},100M,1K \
    HALT,${SSTATE_DIR},100M,1K"

CONF_VERSION = "2"

DISTRO_FEATURES += "systemd"
DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
VIRTUAL-RUNTIME_init_manager = "systemd"
VIRTUAL-RUNTIME_initscripts = ""

IMAGE_FSTYPES = "rpi-sdimg"

We should focus on our Greengrass FleetProvisioning configuration part, which should look like this:


IMAGE_INSTALL:append = " greengrass-bin "
GGV2_DATA_EP     = "xxx-ats.iot.<your aws region>.amazonaws.com"
GGV2_CRED_EP     = "xxx.iot.<your aws region>.amazonaws.com"
GGV2_REGION      = "<your aws region>"
GGV2_THING_NAME  = "ELThing"
GGV2_TES_RALIAS  = "GGTokenExchangeRoleAlias" # we got this from the cloudformation
GGV2_THING_GROUP = "EmbeddedLinuxFleet"

PACKAGECONFIG:pn-greengrass-bin = "fleetprovisioning"

Here it is important to note that we are adding greengrass-bin to our image and then providing additional configuration required by the config.yaml as well as adding PACKAGECONFIG:pn-greengrass-bin = "fleetprovisioning" in order to enable the functionality.

In order to get the AWS region and the IoT endpoints we can do the following:

echo "GGV2_REGION="$(aws configure get region)
echo "GGV2_DATA_EP="$(aws --output text iot describe-endpoint \
    --endpoint-type iot:Data-ATS \
    --query 'endpointAddress')
echo "GGV2_CRED_EP="$(aws --output text iot describe-endpoint \
    --endpoint-type iot:CredentialProvider \
    --query 'endpointAddress')

Please note that we will need a unique Thing Name to be generated for each device, so here the Thing Name is taken as a prefix and there is script inside of a greengreass-bin recipe that appends the unique device id to the Thing Name using the MAC address.

#!/bin/sh
file_path="$1"
default_iface=$(busybox route | grep default | awk '{print $8}')
mac_address=$(busybox ifconfig "$default_iface" | grep -o -E '([[:xdigit:]]{1,2}:){5}[[:xdigit:]]{1,2}' | tr ':' '_')
sed -i "s/<unique>/$mac_address/g" "$file_path"

meta-aws
└── recipes-iot
    └── aws-iot-greengrass
        └──files
            └── replace_board_id.sh

Feel free to replace this file with any other way of obtaining the uniqueness such as serial number or similar

Finally we should copy our claim credentials we generated at the beginning to be included in our build:

cp  "claim-certs/claim.cert.pem" \
    "claim-certs/claim.pkey.pem" \
    "claim-certs/claim.root.pem" \
    $BASEDIR/$DIST/meta-aws/recipes-iot/aws-iot-greengrass/files/

Please adjust the paths based on the location of generated certs and the recipe.

After all of this we should proceed with building our image.

bitbake core-image-minimal

⌛ Couple of hours later ⌛ the build should be complete, and we can find the resulting image in the following directory:

ls tmp/deploy/images/raspberrypi4-64/*sdimg

To flash the image onto an SD card, use a tool like 'dd'

sudo dd if=tmp/deploy/images/raspberrypi4-64/core-image-minimal-raspberrypi4-64.sdimg of=/dev/sdX bs=4M

Where we need to make sure to replace "/dev/sdX" with the appropriate device identifier for the SD card.

⚠️ Please double check the SD card identifier as a mistake here can wipe your workstation system

Powering the Device for the First Time

Once the SD card is reinserted into the Raspberry Pi with power and internet connected, the device should perform provisioning and appear in the list of Greengrass core devices.:

aws greengrassv2 list-core-devices

{
    "coreDevices": [
        {
            "coreDeviceThingName": "ELThing_11_22_33_44_55_60",
            "status": "HEALTHY",
            "lastStatusUpdateTimestamp": "2023-04-25T15:39:00.703000+00:00"
        },
       {
            "coreDeviceThingName": "ELThing_11_22_33_44_55_61",
            "status": "HEALTHY",
            "lastStatusUpdateTimestamp": "2023-03-31T03:11:17.911000+00:00"
        },
       {
            "coreDeviceThingName": "ELThing_11_22_33_44_55_62",
            "status": "HEALTHY",
            "lastStatusUpdateTimestamp": "2023-02-25T15:17:29.505000+00:00"
        },        
    ]
}

Success!

Conclusion

To sum it up, managing a large fleet of embedded Linux devices can be a complex and challenging task, especially when it comes to creating a single image that can be flashed onto multiple devices. However, AWS IoT Greengrass with Fleet Provisioning can streamline this process and make it more efficient and reliable. In this blog post, we have discussed the prerequisites for setting up AWS IoT Greengrass and Fleet Provisioning, including creating policies, obtaining claim certificates, and configuring the Yocto image build. We have also provided a step-by-step guide to building a Yocto image for Raspberry Pi with Greengrass Fleet Provisioning configuration. Using AWS IoT Greengrass - Fleet Provisioning, managing a large fleet of embedded devices can be made easier, more efficient, and secure.

If you have any feedback about this post, or you would like to see more related content, please reach out to me here, or on Twitter or LinkedIn.

Feel free to checkout this video that goes over the mentioned setup: https://youtu.be/Eeo7GLVr0jw

Orchestrating Application Workloads in Distributed Embedded Systems: Writing and Scaling a Pub Application - Part 2

Nenad Ilic — Thu, 30 Mar 2023 19:23:24 +0000

Introduction

This blog covers the second part of Orchestrating Application Workloads in Distributed Embedded Systems. We will go over how to expose Greengrass IPC in a Nomad cluster and have containerized applications publishing metrics to AWS IoT Core using the Greengrass IPC.

Prerequisites

It is essential to follow the first part of the blog, which covers bootstrapping devices with AWS IoT Greengrass and HashiCorp Nomad. Once that is done, we can jump into the application part and the required configuration. As a reminder, the source code is located here: https://github.com/aws-iot-builder-tools/greengrass-nomad-demo

Greengrass IPC Proxy

In order for applications to access Greengrass IPC, we need to create a proxy. We will use socat to forward the ipc.socket via the network (TCP) and then use socat on the application side to create an ipc.socket file. The example can be found under ggv2-nomad-setup/ggv2-proxy/ipc/recipe.yaml. Here we deploy the Nomad job:

job "ggv2-server-ipc" {
    datacenters = ["dc1"]
    type = "system"
    group "server-ipc-group" {

        constraint {
            attribute = "\${meta.greengrass_ipc}"
            operator  = "="
            value     = "server"
        }

        network {
            port "ipc_socat" {
                static = 3307
            }
        }
        service {
            name = "ggv2-server-ipc"
            port = "ipc_socat"
            provider = "nomad"
        }

        task "server-ipc-task" {
            driver = "raw_exec"
            config {
                command = "socat"
                args = [
                    "TCP-LISTEN:3307,fork,nonblock",
                    "UNIX-CONNECT:$AWS_GG_NUCLEUS_DOMAIN_SOCKET_FILEPATH_FOR_COMPONENT,nonblock"
                ]
            }
        }
    }
}

This job will use the AWS_GG_NUCLEUS_DOMAIN_SOCKET_FILEPATH_FOR_COMPONENT provided by the Greengrass Component deployment and run a socat command by connecting to the defined socket and exposing it over TCP on a reserved port 3307. Note that the deployment of this job will have constraints and will only target the devices tagged as greengrass_ipc=server, as this is intended to be deployed only on a client where Greengrass is running.

To deploy this to our Greengrass device, we will use the same methods from the previous blog post. Which should look something like this:

Start with building and publishing the component by doing gdk build and gdk publish, making sure you are in the ggv2-nomad-setup/ggv2-proxy/ipc/ directory.

Additionally, to deploy this to the targets, we will need to add this to a deployment.json:

        "ggv2.nomad.proxy.ipc": {
            "componentVersion": "1.0.0",
            "runWith": {}
        }

respecting the name of the component and the version provided by the GDK.

After that executing the command below will deploy it to our target:

aws greengrassv2 create-deployment \
    --cli-input-json file://deployment.json\
    --region ${AWS_REGION}

Once the command executes successfully, we will be ready to move forward with our application.

Application Overview

We will have a simple application written in python that publishes information about used memory and CPU and publishes this information using the Greengrass IPC to AWS IoT Core. The topic here is constructed by having NOMAD_SHORT_ALLOC_ID as a prefix followed by /iot/telemetry. We will use this info later once we scale the application across the cluster and start receiving messages on multiple MQTT topics.

Here is the Python code for the application:

import json
import time
import os

import awsiot.greengrasscoreipc
import awsiot.greengrasscoreipc.model as model


NOMAD_SHORT_ALLOC_ID = os.getenv('NOMAD_SHORT_ALLOC_ID')

def get_used_mem():
    with open('/proc/meminfo', 'r') as f:
        for line in f:
            if line.startswith('MemTotal:'):
                total_mem = int(line.split()[1]) * 1024  # convert to bytes
            elif line.startswith('MemAvailable:'):
                available_mem = int(line.split()[1]) * 1024  # convert to bytes
                break

    return total_mem - available_mem

def get_cpu_usage():
    with open('/proc/stat', 'r') as f:
        line = f.readline()
        cpu_time = sum(map(int, line.split()[1:]))
        idle_time = int(line.split()[4])

    return (cpu_time - idle_time) / cpu_time

if __name__ == '__main__':
    ipc_client = awsiot.greengrasscoreipc.connect()

    while True:
        telemetry_data = {
            "timestamp": int(round(time.time() * 1000)),
            "used_memory": get_used_mem(),
            "cpu_usage": get_cpu_usage()
        }

        op = ipc_client.new_publish_to_iot_core()
        op.activate(model.PublishToIoTCoreRequest(
            topic_name=f"{NOMAD_SHORT_ALLOC_ID}/iot/telemetry",
            qos=model.QOS.AT_LEAST_ONCE,
            payload=json.dumps(telemetry_data).encode(),
        ))
        try:
            result = op.get_response().result(timeout=5.0)
            print("successfully published message:", result)
        except Exception as e:
            print("failed to publish message:", e)

        time.sleep(5)

The application can be found under examples/nomad/nomad-docker-pub/app.py. On top of this we will be using a Dokerfile to containerized. In order for this to work with GDK, we will be using build_system: "custom" and specify the script for building and publishing the image to ECR:

{
  "component": {
    "nomad.docker.pub": {
      "author": "Nenad Ilic",
      "version": "NEXT_PATCH",
      "build": {
        "build_system": "custom",
        "custom_build_command": [
          "./build.sh"
         ]
      },
      "publish": {
        "bucket": "greengrass-component-artifacts",
        "region": "eu-west-1"
      }
    }
  },
  "gdk_version": "1.1.0"
}

Where build.sh will look like this:


set -e
AWS_ACCOUNT_ID=$(aws sts get-caller-identity |  jq -r '.Account')
AWS_REGION=$(jq -r '.component | to_entries[0] | .value.publish.region' gdk-config.json)
COMPONENT_NAME=$(jq -r '.component | keys | .[0]' gdk-config.json)
COMPONENT_AUTHOR=$(jq -r '.component | to_entries[0] | .value.author' gdk-config.json)
COMPONENT_NAME_DIR=$(echo $COMPONENT_NAME | tr '.' '-')

rm -rf greengrass-build
mkdir -p greengrass-build/artifacts/$COMPONENT_NAME/NEXT_PATCH
mkdir -p greengrass-build/recipes
cp recipe.yaml greengrass-build/recipes/recipe.yaml
sed -i "s/{COMPONENT_NAME}/$COMPONENT_NAME/" greengrass-build/recipes/recipe.yaml
sed -i "s/{COMPONENT_AUTHOR}/$COMPONENT_AUTHOR/" greengrass-build/recipes/recipe.yaml
sed -i "s/{AWS_ACCOUNT_ID}/$AWS_ACCOUNT_ID/" greengrass-build/recipes/recipe.yaml
sed -i "s/{AWS_REGION}/$AWS_REGION/" greengrass-build/recipes/recipe.yaml
sed -i "s/{COMPONENT_NAME_DIR}/$COMPONENT_NAME_DIR/" greengrass-build/recipes/recipe.yaml

docker build -t $COMPONENT_NAME_DIR .
docker tag $COMPONENT_NAME_DIR:latest $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/$COMPONENT_NAME_DIR:latest

if aws ecr describe-repositories --region $AWS_REGION --repository-names $COMPONENT_NAME_DIR > /dev/null 2>&1
then
    echo "Repository $COMPONENT_NAME_DIR already exists."
else
    # Create the repository if it does not exist
    aws ecr create-repository --region $AWS_REGION --repository-name $COMPONENT_NAME_DIR
    echo "Repository $COMPONENT_NAME_DIR created."
fi
aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com
docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/$COMPONENT_NAME_DIR:latest

The script assumes the AWS CLI is installed and gets all the necessary configuration from the gdk-config.json . The build script will create the appropriate recipe, build the docker image, login to ECR and push it referencing the component name set in the gdk-config.json.

Finally the Nomad Job for deploying the application will look like this:

job "nomad-docker-pub-example" {
    datacenters = ["dc1"]
    type = "service"
    group "pub-example-group" {
        count = 1
        constraint {
            attribute = "\${meta.greengrass_ipc}"
            operator  = "="
            value     = "client"
        }
        task "pub-example-task" {
            driver = "docker"
            config {
                image = "{AWS_ACCOUNT_ID}.dkr.ecr.{AWS_REGION}.amazonaws.com/{COMPONENT_NAME_DIR}:latest"
                command = "/bin/bash"
                args = ["-c", "socat UNIX-LISTEN:\$AWS_GG_NUCLEUS_DOMAIN_SOCKET_FILEPATH_FOR_COMPONENT,fork,nonblock TCP-CONNECT:\$GGV2_SERVER_IPC_ADDRESS,nonblock & python3 -u /pyfiles/app.py "]
            }
            env {
                AWS_GG_NUCLEUS_DOMAIN_SOCKET_FILEPATH_FOR_COMPONENT = "/tmp/ipc.socket"
                SVCUID="$SVCUID"
            }
            template {
                data = <<EOF
# Get all services and add them to env variables with their names
{{ range nomadServices }}
    {{- range nomadService .Name }}
    {{ .Name | toUpper | replaceAll "-" "_" }}_ADDRESS={{ .Address}}:{{ .Port }}{{- end }}
{{ end -}}
EOF
                destination = "local/env"
                env = true
            }
        }
    }
 }

Constraint - We are starting with our constraint where this application should be deployed. In this scenario, it would be targeting only Nomad clients where greengrass_ipc=client.
Task - Next, we have our task with the Docker driver. Here, we get the image from the ECR, where the variables AWS_ACCOUNT_ID, AWS_ACCOUNT_ID, and COMPONENT_NAME_DIR will be replaced by the build script with the appropriate values. Finally, we come to our command and args. These values would override what is already defined by the Dockerfile. In this scenario, we first create the ipc.socket required by the application using socat. The SVCUID will be then provided by the Greengrass component at the time of running the job, thus provided as an environment variable inside the Docker container.
Template - After that, we have a template section that we require to obtain the IP address of our ggv2-server-ipc service that we created earlier. We do this by listing all the services and getting the IP addresses and exporting them as environment variables by also converting their names to uppercase letters and appending _ADDRESS at the end. This provides our env variable GGV2_SERVER_IPC_ADDRESS for our socat command that then looks like this:

socat UNIX-LISTEN:$AWS_GG_NUCLEUS_DOMAIN_SOCKET_FILEPATH_FOR_COMPONENT,fork,nonblock TCP-CONNECT:$GGV2_SERVER_IPC_ADDRESS,nonblock

Which provides the ipc.socket before running the app:

python3 -u /pyfiles/app.py

Once we have this set, we can then go build and publish the component by doing gdk build and gdk publish.

Additionally in order to deploy this to the targets we will need to add this to a deployment.json:

        "nomad.docker.pub": {
            "componentVersion": "1.0.0",
            "runWith": {}
        }

respecting the name of the component and the version provided by the GDK.
After that executing the command below will deploy it to our target:

aws greengrassv2 create-deployment \
    --cli-input-json file://deployment.json\
    --region ${AWS_REGION}

Now we will be ready to scale our application.

Scaling the Application

As of now we should have our application running and publishing the data from a single client, however if we require to spread this application across the cluster, in this scenario second client, and have it report the memory and CPU usage, we can do this by simply changing the count=1 to count=2 in our job file:

--- a/examples/nomad/nomad-docker-pub/recipe.yaml
+++ b/examples/nomad/nomad-docker-pub/recipe.yaml
@@ -31,7 +31,7 @@ Manifests:
             type = "service"

             group "pub-example-group" {
-              count = 1
+              count = 2
               constraint {
                 attribute = "\${meta.greengrass_ipc}"
                 operator  = "="

And use the same method to redeploy.

Now if we go to AWS console and under AWS IoT → MQTT test client we can subscribe to topics <NOMAD_SHORT_ALLOC_ID>/iot/telemetry and should be able to see messages coming. In order to get this ID we can simply run the following command on our device where the nomad server is running:

nomad status nomad-docker-pub-example

And we will find the ID int the Allocations section that looks something like this:

Allocations
ID        Node ID   Task Group         Version  Desired  Status  Created     Modified
5dce9e1a  6ad1a15c  pub-example-group  10       run      running 1m28s ago   32s ago
8c517389  53c96910  pub-example-group  10       run      running 1m28s ago   49s ago

This will then allow us to construct those MQTT topics and start seeing the messages coming from those two instances of our application:

In the next part we will take a look on how to access AWS services from the device using Token Exchange Service (TES).

Conclusion

In this blog post, we covered how to expose the Greengrass IPC in a Nomad cluster, allowing a containerized application to publish metrics to AWS IoT Core using Greengrass IPC. We demonstrated how to create a proxy using socat to forward the ipc.socket via network (TCP) and how to set up an application that reports memory and CPU usage. We also showed how to scale the application across multiple clients in the cluster.

By using AWS IoT Greengrass and HashiCorp Nomad, you can effectively manage, scale and monitor distributed embedded systems, making it easier to deploy and maintain complex IoT applications.

If you have any feedback about this post, or you would like to see more related content, please reach out to me here, or on Twitter or LinkedIn.

Orchestrating Application Workloads in Distributed Embedded Systems: Setting up a Nomad Cluster with AWS IoT Greengrass - Part 1

Nenad Ilic — Thu, 02 Mar 2023 12:51:18 +0000

Introduction

As interconnected, purpose-built Systems on Chips (SoCs) have become more popular in industrial and automotive settings, managing their software components, including application deployments, data processing, and network communication, has become more challenging. This is particularly important in factories and cars, where downtime and maintenance costs can have a significant impact on productivity. To address these issues, developers can use available orchestration tools to ensure that the system operates efficiently and effectively, and interface those distributed embedded systems from the cloud in order to enable changes and updates throughout their lifecycle.

In this blog post series, we will demonstrate how to use AWS IoT Greengrass and Hashicorp Nomad to seamlessly interface with multiple interconnected devices and orchestrate service deployments on them. Greengrass will allow us to view the cluster as a single "Thing" from the cloud perspective, while Nomad will serve as the primary cluster orchestration tool.

In Part 1 of this series, we will cover how to set up a cluster of 3 devices with AWS IoT Greengrass already installed on one device, and then set up a cluster of 1 Nomad server/client and 2 Nomad clients.

It's important to note that Nomad is a lightweight orchestration tool for distributed environments that comes as a single binary installation. This makes it easy to operate while still providing features that allow applications to run in high availability. In addition to running Docker containers, Nomad enables the execution of tasks on the cluster in other forms such as Java, exec, QEMU, and more.

That being said, this tutorial is targeted for IoT engineers, embedded systems developers, and IoT DevOps engineers with a technical background in building and running applications on embedded systems, as well as basic knowledge about Greengrass.

Prerequisites

For this specific example, we will be using 3 Raspberry Pis, but this setup should generally work on devices running Linux, thus having a 3 EC2’s running Ubuntu would be a good replacement. To get started, we will pick one of those 3 devices to be our server which will need to have AWS IoT Greengrass installed, by following these steps and by taking note of you Things Name and/or Thing Group. In this example we have used nomad-cluster as a Thing Group. Additionally in order to follow along, please have a GDK installed on development host.

Next, on two other devices, in order to be able to ssh into them from the server and set up the Nomad clients, we will need to add the server’s public key to them. This can be done by using ssh-keygen to generate the pair and then using ssh-copy-id user@remote_host to copy the public key to other two devices. This will allow easier client setup using Greengrass components provided in the example repository.

Please also note that we will be using root in this setup in order to allow certain application access, but this is not advised in production (we will discuss this in one of the following blogs). Also, instead of using IP addresses, it is advised for each device to have a unique hostname and responding DNS entry for easier use.

Bootstrapping the Nomad Cluster

Now that we have all that in place we can start with bootstrapping the Nomad cluster. For this we will be using the following GitHub repository as a starting point by cloning it:

git clone https://github.com/aws-iot-builder-tools/greengrass-nomad-demo.git

Nomad Server

Next navigate to ggv2-nomad-setup/cluster-bootstrap/nomad-server directory in the cloned repository. Here we can find a recipe.yaml for our nomad.bootstrap.server component as well as our gdk-config.json which we can modify to reflect the current setup (region, author, bucket and so on...).

Now we would be ready to build and publish our component by executing gdk build and gdk publish which will create the component, publish the artifact to an S3 bucket and version it.

The component will include a server.hcl file for setting up the nomad server/client that looks like this:

# Increase log verbosity
log_level = "DEBUG"

# Setup data dir
data_dir = "/opt/nomad/data"

# Enable the server
server {
  enabled = true
}
# Enable the client as well
client {
  enabled = true
  meta {
    greengrass_ipc = "server"
  }
  options = {
    "driver.raw_exec.enable" = "1"
  }
}

# port for the ui
ports {
  http = 8080
}

For the Nomad client, we will add additional metadata greengrass_ipc = "server" which will allow us to deploy services on a host where Greengrass is running and also enable raw_exec driver by setting "driver.raw_exec.enable" = "1" , and finally we will expose the Nomads UI on the 8080 port allowing us to view it in our local network for debugging purposes, of course this should be disabled in production environment.

In the root of the repository we have deployment.json.template which we can use to create a Greengrass deployment targeting our Thing Group, which in this scenario we called nomad-cluster.

To reflect your set-up, you can adapt the template file to your needs and then run the commands below to populate the required variables:

AWS_ACCOUNT_ID=$(aws sts get-caller-identity |  jq -r '.Account')
AWS_REGION=eu-west-1
envsubst < "deployment.json.template" > "deployment.json"

Now, we should be able to modify the deployment.json file to only include the nomad.bootstrap.server and the appropriate component version, which should look like this:

{
    "targetArn": "arn:aws:iot:<AWS Region>:<AWS Account>:thinggroup/nomad-cluster",
    "deploymentName": "Deployment for nomad-cluster group",
    "components": {
        "nomad.bootstrap.server": {
            "componentVersion": "1.0.0",
            "runWith": {}
        }
    },
    "deploymentPolicies": {
        "failureHandlingPolicy": "ROLLBACK",
        "componentUpdatePolicy": {
            "timeoutInSeconds": 60,
            "action": "NOTIFY_COMPONENTS"
        },
        "configurationValidationPolicy": {
            "timeoutInSeconds": 60
        }
    },
    "iotJobConfiguration": {}
}

And finally deploy it to our target by executing:

aws greengrassv2 create-deployment \
    --cli-input-json file://deployment.json\
    --region ${AWS_REGION}

This bootstraps the Nomad server on a device with Greengrass installed, and configures the Nomad client to enable the deployment of proxies for Greengrass IPC and Token Exchange Service. We will provide a detailed guide for setting up the client in the next blog post.

Nomad Clients

For Nomad clients the process would be similar, with a difference of configuring and adding the DNS Names (or IPs) with usernames for each client, as well as server’s DNS (or IP) in the recipe.yaml under ggv2-nomad-setup/cluster-bootstrap/nomad-clients/recipe.yaml:

ComponentConfiguration:
  DefaultConfiguration:
    Server:
      DnsName: "rpi-server.local"
    Client:
      "1":
        UserName: root
        DnsName: "rpi-client1.local"
      "2":
        UserName: root
        DnsName: "rpi-client2.local"

This is to allow the Greengrass component to install the Nomad clients using ssh and adding configuring them so they can connect with the server.

The client.hcl which will be added to each Nomad client will look like this where the <SERVER_DNS_NAME>will be replaced from the configuration provided in the recipe.

# Increase log verbosity
log_level = "DEBUG"

# Setup data dir
data_dir = "/opt/nomad/data"

client {
  enabled = true
  servers = ["<SERVER_DNS_NAME>"]
  meta {
    greengrass_ipc = "client"
  }
  options = {
    "driver.raw_exec.enable" = "1"
  }
}

# different port than server
ports {
  http = 5656
}

Once that is done, we can proceed with building and publishing the component by doing gdk build and gdk publish .

Additionally in order to deploy this to the targets we will need to add:

        "nomad.bootstrap.clients": {
            "componentVersion": "1.0.0",
            "runWith": {}
        }

And execute the same command to deploy it to our target:

aws greengrassv2 create-deployment \
    --cli-input-json file://deployment.json\
    --region ${AWS_REGION}

With the clients bootstrapped, we are ready to move on to the next steps in deploying a demo component to Greengrass and Nomad, and how to publish data to AWS IoT Core using Greengrass IPC, which we will do in the second part of this blog.

It's worth noting that this is a demo deployment and not suitable for production environments. In a future blog post in this series, we will discuss how to set up Nomad for production environments, including security considerations and best practices.

Conclusion

In Part 1 of this series, we have covered the importance of orchestrating application workloads in distributed embedded systems and how to set up a Nomad cluster with AWS IoT Greengrass. By following these steps, developers can ensure that their systems operate efficiently, reliably, and with optimal performance, while also allowing them to develop and deploy new services even after the systems are operational.

In the next blog post, we will cover the next steps in the process of setting up a demo component to Greengrass and Nomad, and how to publish data to AWS IoT Core using Greengrass IPC. Stay tuned for Part 2 of this series!

If you have any feedback about this post, or you would like to see more related content, please reach out to me here, or on Twitter or LinkedIn.

Getting started with AWS IoT Greengrass and C# .NET development

Nenad Ilic — Thu, 19 Jan 2023 15:32:39 +0000

Introduction

AWS IoT Greengrass is a powerful tool for creating and deploying IoT software components to devices, allowing you to perform tasks such as data processing, analytics, and device control, orchestrating all of this from the cloud. This can be especially useful for IoT systems that need to operate in low-bandwidth or offline environments but also want to re-use existing software stacks and make sure they are up to date while allowing new functionality to be added as well.

While there are examples for developers using C/C++, Java and Python to develop components that utilize the Greengrass Inter-Process Communication (IPC) features to interface with the cloud, C# .NET is not officially supported in the device SDK to interface with the Greengrass IPC. This means that developers using C# .NET cannot take advantage of the benefits of developing components using C#, thus limit the access to a large number of pre-existing libraries and a large developer community.

However, it's still possible to develop Greengrass components using C# .NET by using EventStream for Greengrass IPC. In this post, we'll walk through an example of developing an AWS IoT Greengrass component using C# .NET on Linux development machine. We will be showing you how to set up the development environment, how to develop the component, and how to deploy it to a Greengrass device, where this component will be publishing to AWS IoT Core current process information.

Prerequisites

For this example, we'll be using a development machine with the following software installed:

.NET Core SDK 6.0+
Visual Studio Code with the C# extension installed
An AWS account and AWS IoT Greengrass provisioned on a compatible device. For more info take a look at the Getting Started.
Greengrass Development Kit CLI

Setting up the Development Environment

The example is a located under greeneyes repository, which will utilise some of the scripts introduced in order to make easier development. For more details take a look at the initial blog.

That being said, let’s checkout the repository with the command:

git clone git@github.com:aws-iot-builder-tools/greeneyes.git

Next, we'll proceed to our C# example located under greeneyes/blog-posts/002/shared/CSharpExample , and explore the code. The part which adds the IPC communication between the Greengrass component and the Greengrass Nucleus can be found under Events directory, while the main program is in the Program.cs .

Developing the Greengrass Component

Now we're ready to start looking at our Greengrass component. Let's open the project in Visual Studio Code using the following command :

code .

In the Program.cs file, we can see our Main function and change it to suit our needs. In this example, we have a main loop that publishes current process information representing our Greengrass device to a topic greengrass/[AWS_IOT_THING_NAME] where AWS_IOT_THING_NAME will be retrieved from a an environment set by the Greeengrass Nucleus.

using System;
using System.Diagnostics;
using System.Threading;

namespace GGCSharp
{
    public static class Program
    {
        public static bool Running { get; set; } = true;
        private static string AwsIotThingName => Environment.GetEnvironmentVariable("AWS_IOT_THING_NAME");
        public static string ThingTopic => string.Join("/", "greengrass", AwsIotThingName);

        private static void Main(string[] args)
        {
            InstallShutdownHandler();

            while (Running)
            {
                var currentProcess = Process.GetCurrentProcess();
                AwsIotGreengrassIpc.Publish(ThingTopic, 0, $"Hello from C# process: {currentProcess.WorkingSet64}");
                Thread.Sleep(10000);
            }

            AwsIotGreengrassIpc.Publish(ThingTopic, 1, "Shutting down");
        }

        private static void InstallShutdownHandler()
        {
            // If the process is exiting set the global running flag to false
            AppDomain.CurrentDomain.ProcessExit += (_, _) => Running = false;
        }
    }
}

The Main method also calls a method named "InstallShutdownHandler" which is attaching an event handler to the "ProcessExit" event of the current app domain. The event handler is just a lambda that sets the Running variable to false, which stops the infinite loop and sends a last message "Shutting down" on the topic with a qos=1 before the program exits.

Deploying the Greengrass Component

Once our code is complete, we can deploy it to our Greengrass device. We need to create a deployment package containing our code, by running the following commands in the terminal:

dotnet publish -c Release

This will fetch the necessary dependencies, create a release build of the project and create a deployment package in the bin/Release/net6.0/publish directory. Now we can build the Greengrass artifact:

mkdir -p greengrass-build/artifacts/greeneyes.GGCSharp/NEXT_PATCH
pushd .
cd bin/Release/net6.0/publish
zip -r -X GGCSharp.zip .
popd
mv bin/Release/net6.0/publish/GGCSharp.zip greengrass-build/artifacts/greeneyes.GGCSharp/NEXT_PATCH

The above commands will create a directory structure for the Greengrass component artifact, zip the build and move it.

Before we generate an appropriate component recipe we first need to make sure we have a correct gdk-config.json as we will be using Greengrass Development Kit CLI for the deployment.

Here we have a gdk-config.json.template file where we need to modify: author, bucket and region and save it as a gdk-config.json

{
  "component": {
    "greeneyes.GGCSharp": {
      "author": "<PLACEHOLDER_AUTHOR>",
      "version": "NEXT_PATCH",
      "build": {
        "build_system": "custom",
        "custom_build_command": [ "./build.sh" ]
      },
      "publish": {
        "bucket": "<PLACEHOLDER_BUCKET>",
        "region": "<PLACEHOLDER_REGION>"
      }
    }
  },
  "gdk_version": "1.0.0"
}

For author we can simply put BlogReader, but for s3 bucket and region we can use AWS CLI to populate those.
For example for region we can use:

aws configure get region

and replace the output, and for the bucket we suggest creating a unique bucket that is related to the project. For example taking the hostname and machine-id:

echo $(sed 's/-//g' <<< $(hostname))$(head -c 8 /etc/machine-id)

Next we can move to the component recipe:

---
RecipeFormatVersion: "2020-01-25"
ComponentName: "{COMPONENT_NAME}"
ComponentVersion: "{COMPONENT_VERSION}"
ComponentDescription: "This is simple Hello World component written in C#."
ComponentPublisher: "{COMPONENT_AUTHOR}"
ComponentConfiguration:
  DefaultConfiguration:
    accessControl:
      aws.greengrass.ipc.mqttproxy:
        "mqtt:1":
          operations:
            - "aws.greengrass#PublishToIoTCore"
          resources:
            - "*"
ComponentDependencies:
  aws.greengrass.TokenExchangeService:
    VersionRequirement: '^2.0.0'
    DependencyType: HARD
Manifests:
  - Platform:
      os: all
    Artifacts:
      - URI: "s3://BUCKET_NAME/COMPONENT_NAME/COMPONENT_VERSION/GGCSharp.zip"
        Unarchive: ZIP
    Lifecycle:
      Run: "chmod +x {artifacts:decompressedPath}/GGCSharp/GGCSharp && {artifacts:decompressedPath}/GGCSharp/GGCSharp"

Where we need to make sure we update the {COMPONENT_NAME} and {COMPONENT_AUTHOR} respectively, after moving the recipe to the greengrass-build directory.

mkdir -p greengrass-build/recipes
cp recipe.yaml greengrass-build/recipes/recipe.yaml

Once finished we can then simply use the gdk tool to build and publish the our Greengrass component:

gdk component build
gdk component publish

In order to make this easer a build.sh script with the same instructions is located in the root folder of the example.

Once the component is created we can simply go to the AWS Console under AWS IoT → Greengrass → Deployments and create a deployment to target our desired device and go through a process of deploying our component, or just using the command:

aws greengrassv2 create-deployment \
          --cli-input-json file://deployment.json \
          --region ${AWS_REGION}

where the deployment.json could look something like this:

{
    "targetArn": "arn:aws:iot:$AWS_REGION:$AWS_ACCOUNT_ID:thinggroup/<Thing Group>",
    "deploymentName": "Deployment for our group",
    "components": {
        "<COMPONENT_NAME>": {
            "componentVersion": "$LATEST_COMPONENT_VERSION",
            "runWith": {}
        },
        "aws.greengrass.Nucleus": {
            "componentVersion": "2.8.1"
        }
    },
    "deploymentPolicies": {
        "failureHandlingPolicy": "ROLLBACK",
        "componentUpdatePolicy": {
            "timeoutInSeconds": 60,
            "action": "NOTIFY_COMPONENTS"
        },
        "configurationValidationPolicy": {
            "timeoutInSeconds": 60
        }
    },
    "iotJobConfiguration": {}
}

Alternatively we can use our project helper tools to deploy the component by simply executing:

gg-cloud-deploy

which should do all the steps above.

Viewing Publishing Data

Once the component is deployed and running we can should be able to see the incoming data by going to AWs IoT → MQTT test client and subscribing to a topic greengrass/[AWS_IOT_THING_NAME]

Limitations

Currently, the only available method for IPC is PublishToIotCore. However, other methods can be implemented by referencing the APIs provided in the python SDK, located a here. In the future, we aim to integrate these methods directly into our RpcMessages.cs for greater convenience.

Conclusion

In this post, we've walked through an example of developing an AWS IoT Greengrass component using C# .NET. We've seen how to set up the development environment, write the code for the component, and deploy it to a Greengrass device. With the power of C# and the AWS IoT Greengrass, you can easily create powerful and robust Greengrass components for IoT devices. We acknowledge that there are limitations in interfacing the IPC, but per need bases this can be extended. We did not cover the methods for interacting with other AWS cloud services in this discussion, however it is possible to do so by utilizing the C# SDK as outlined in the documentation provided at here.

Please reach out if you have any questions or need help with your development, we're always happy to help.

Home Treadmill with AWS IoT Greengrass and RPi 0 W

Nenad Ilic — Thu, 24 Nov 2022 19:16:11 +0000

This setup was inspired by a blog post Using GitHub Actions for Greengrass v2 Continuous Deployment after a talk at IoT Builders Session. After the discussion, I wanted to try out the Greengrass Continues Deployment setup using the Github Actions and realized I had the treadmill, which I might be able to retrofit for this purpose.

In the setup, I'll use the same principles for deploying a Greengrass component with GitHub Actions and OIDC, which will allow me to develop and test a Greengrass component that publishes treadmill speed measurements to AWS IoT Core and saves them in Amazon Timestream using IoT Rules. After that, I'll configure an Amazon Managed Grafana to visualize and analyze how far I've run as well as my current, maximum, and average speed.

As of now I’m at 20.5 km/h max speed 🐌

The Hardware Setup

Personally, I’ve been trying to increase my sprinting speed and have been training on a curved treadmill, which doesn’t have any motors or require a power supply. One of the reasons I like running on this type of self-powered treadmill is that it allows me to run naturally on the balls of my feet. This motion creates momentum to turn the treadmill belt by pushing your body forward. The point of contact is significantly ahead of the center of mass; thus, the experience of support is different than with other non-motorized treadmill options and more similar to that of running on the ground.
So after the previously mentioned talk with Nathan, I thought it would be nice to have a dashboard that tells me how much I’ve run and provides me with my current and maximal speed.

I’ve noticed that there is a magnetic reed-switch that is triggered on each revolution of the conveyor drum, thus making the speed measurement setup easy to wire, so I decided to give it a try and setup my Greengrass CI/CD based on the mentioned blog and Github. Additionally, I had a spare Raspberry Pi Zero W with a battery module laying around, and I wanted to try using it with Greengrass.

However, I needed to figure out how to determine the running speed based on the number of revolutions per second. In order to do this, I had to use the original speed meter display that came with my treadmill and simulate reed switch impulses by providing a 1Hz signal.

This gave me a running (walking) speed of 2.2 km/h. I then went on to test other frequencies ranging from 2 Hz to 17 Hz.

The conclusion was that, as expected, there was a linear distribution between number of revolutions and treadmill speed, and the factor of conversion is 2.2, which made the code in RPi quite easy to write. Before jumping into the code, I had to wire the reed switch and connect it properly to the Raspberry using the wiring diagram below.

In this setup, every time a magnet crosses near the reed switch, I get a falling edge signal from 3.3V to 0V; thus, the time between falling edges gives one revolution of the conveyor drum.

Once the magnet rotates further, the voltage rises back to 3.3V, and we have a rising edge that will stay there for the next cycle until the magnet's magnetic field interacts with the reed switch, as shown in the image above.

The Speed Measurement Code

The code below can be used to get the average speed over a period of time and print it out. This is the initial version I’ve used to compare the results measured by the RPi and the original speed meter display, but as mentioned above, it measures the time between two falling edges and divides this by the determined conversion factor in order to get the current speed.

import time
import RPi.GPIO as GPIO

BUTTON_GPIO = 18
# 1Hz is 2.2km/h
CONVERSION_FACTOR = 2.2

start = time.time()
# Used to calculate average speed
speeds = deque([0, 0, 0, 0, 0])

# Callback on the signal falling edge
def reed_switch_callback(channel):
    global start
    # Elapsed time between previous falling edge
    elapsed = time.time() - start
    start = time.time()
    
    speed = CONVERSION_FACTOR / elapsed

GPIO.setmode(GPIO.BCM)
GPIO.setup(BUTTON_GPIO, GPIO.IN, pull_up_down=GPIO.PUD_UP)
GPIO.add_event_detect(BUTTON_GPIO, GPIO.FALLING, callback=reed_switch_callback, bouncetime=10)

while True:

    no_activity = time.time() - start
    if no_activity > 1:
        # if there is no activity for more then 1s
        # this means we are bellow 2.2km/h so we add 0km/h measurement
        speeds.append(0)
        speeds.popleft()
    av_speed = sum(speeds) / len(speeds)

    print(f'average speed: {av_speed}')
    time.sleep(0.5)

The Greengrass Setup

Since I've been running this on a Raspberry Pi Zero W, before installing Greengrass I had to install openjdk-8-jre as newer versions of java will not run on the ARMv6 instruction set:

sudo apt update
sudo apt install openjdk-8-jre

After this, I had to run the following on the device, while making sure I had AWS Credentials setup on the device before running it, as instructed here.

curl -s https://d2s8p88vqu9w66.cloudfront.net/releases/greengrass-nucleus-latest.zip > greengrass-nucleus-latest.zip && unzip greengrass-nucleus-latest.zip -d GreengrassCore

export AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
export AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
export AWS_REGION="eu-west-1"
export GREENGRASS_THING_GROUP="home"
export GREENGRASS_THING_NAME="home-treadmill"

# For more: https://docs.aws.amazon.com/greengrass/v2/developerguide/getting-started.html#install-greengrass-v2
sudo -E java \
  -Droot="/greengrass/v2" \
  -Dlog.store=FILE -jar ./GreengrassCore/lib/Greengrass.jar \
  --aws-region ${AWS_REGION} \
  --thing-name ${GREENGRASS_THING_NAME} \
  --thing-group-name ${GREENGRASS_THING_GROUP} \
  --thing-policy-name GreengrassV2IoTThingPolicy \
  --tes-role-name GreengrassV2TokenExchangeRole \
  --tes-role-alias-name GreengrassCoreTokenExchangeRoleAlias \
  --component-default-user ggc_user:ggc_group \
  --provision true \
  --setup-system-service true \
  --deploy-dev-tools true

This sets up the Greengrass on the device with the appropriate Thing Group and Thing Name, in this scenario, home and home-treadmill respectively.
As for the Greengrass components continuous deployment (CD) part where I use Github Actions, I’ve followed the instructions provided in the blog post, and extended the application to publish the speed measurements to the AWS IoT topic home/treadmill/speed using the awsiot.greengrasscoreipc.client.

The extended code looks like this:

# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0
import time
import traceback
import json

import signal
import sys
import RPi.GPIO as GPIO
import time
from collections import deque

import awsiot.greengrasscoreipc
import awsiot.greengrasscoreipc.client as client
from awsiot.greengrasscoreipc.model import (
    IoTCoreMessage,
    QOS,
    PublishToIoTCoreRequest
)

BUTTON_GPIO = 18
# 1Hz is 2.2km/h
CONVERSION_FACTOR = 2.2

start = time.time()
# Used to calculate average speed
speeds = deque([0, 0, 0, 0, 0])

TIMEOUT = 10

ipc_client = awsiot.greengrasscoreipc.connect()

topic = "home/treadmill/speed"
qos = QOS.AT_MOST_ONCE

def signal_handler(sig, frame):
    GPIO.cleanup()
    sys.exit(0)

def reed_switch_callback(channel):
    global start
    # Elapsed time between previous falling edge
    elapsed = time.time() - start
    start = time.time()
    speed = CONVERSION_FACTOR / elapsed
    # Discard the noise as we can't measure speed above 36
    if speed < 36:
        speeds.append(speed)
        speeds.popleft()

GPIO.setmode(GPIO.BCM)
GPIO.setup(BUTTON_GPIO, GPIO.IN, pull_up_down=GPIO.PUD_UP)
GPIO.add_event_detect(BUTTON_GPIO, GPIO.FALLING, callback=reed_switch_callback, bouncetime=10)

signal.signal(signal.SIGINT, signal_handler)
while True:
    no_activity = time.time() - start
    
    if no_activity > 1:
        # if there is no activity for more then 1s
        # this means we are bellow 2.2km/h so we add 0km/h measurement
        speeds.append(0)
        speeds.popleft()
    av_speed = sum(speeds) / len(speeds)
    if av_speed > 0:

        print(f'average speed: {av_speed}')

        request = PublishToIoTCoreRequest()
        request.topic_name = topic
        message = json.dumps({
            'time': time.time(),
            'speed': av_speed
        })
        request.payload = bytes(message, "utf-8")
        request.qos = qos
        operation = ipc_client.new_publish_to_iot_core()
        operation.activate(request)
        future_response = operation.get_response()
        future_response.result(TIMEOUT)        

    time.sleep(0.5)

Additionally, I’ve tried to compare my measurements back with the readings from the original speed meter display and got the same precession, which meant that even with the overhead of Greengrass, the system was running just fine.

Using AWS IoT Rules

In order to route the speed measurements coming from the device directly to the Timestream Database Table, I’ve used the AWS IoT Rules with the following select rule:

SELECT speed FROM "home/treadmill/speed" WHERE speed>0

and had the Amazon Timestream Database Table as an Action.
This will interpret the values from a JSON payload message having speed as a key and put the value in the selected Timestream table.

Here is the Cloudformation stack that sets this up:

AWSTemplateFormatVersion: '2010-09-09'
Resources:
  HomeTreadmillRule:
    Type: AWS::IoT::TopicRule
    Properties:
      RuleName: "HomeTreadmill"
      TopicRulePayload:
        RuleDisabled: false
        Sql: SELECT speed FROM "home/treadmill/speed" WHERE speed>0
        Actions:
        - Timestream:
            DatabaseName: "home-treadmill"
            TableName: "measurments"
            Dimensions: 
            - Name: "Speed"
              Value: 0
            RoleArn: !GetAtt HomeTreadmillTopicRuleRole.Arn
  HomeTreadmillDB:
    Type: AWS::Timestream::Database
    Properties: 
      DatabaseName: "home-treadmill"
  HomeTreadmillTable:
    Type: AWS::Timestream::Table
    Properties: 
      DatabaseName: !Ref HomeTreadmillDB
      RetentionProperties:
        MemoryStoreRetentionPeriodInHours: "24"
        MagneticStoreRetentionPeriodInDays: "7"
      TableName: "measurments"
  HomeTreadmillTopicRuleRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - iot.amazonaws.com
            Action:
              - sts:AssumeRole
      Policies:
        - PolicyName: AllowTimestream
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: Allow
                Action:
                  - timestream:WriteRecords
                Resource:
                  - !GetAtt HomeTreadmillTable.Arn
              - Effect: Allow
                Action:
                  - timestream:DescribeEndpoints
                Resource: "*"

Deploy command:

aws cloudformation deploy --template-file cfn/amazon-timestream/timestream.yaml --stack-name home-treadmill-timestream --capabilities CAPABILITY_IAM

Grafana Setup

There are several ways to set up Grafana and use it to gain insights from Timestream table measurements; however, the simplest for me was to create a workspace using Amazon Managed Grafana.

I intend to use it for other projects and give multiple users access to Grafana, which is part of my AWS IAM Identity Center.
To do this, I had to create a role that allows reading from the Timestream:

AWSTemplateFormatVersion: '2010-09-09'
Resources:
  AmazonGrafanaServiceRoleHomeTreadmill:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - grafana.amazonaws.com
            Action:
              - sts:AssumeRole
      ManagedPolicyArns: 
      - "arn:aws:iam::aws:policy/AmazonTimestreamReadOnlyAccess"
Outputs:
  RoleAwsAccountId:
    Value: !Ref AWS::AccountId
  RoleAwsRegion:
    Value: !Ref AWS::Region
  WorkspaceRoleToAssume:
    Value: !GetAtt AmazonGrafanaServiceRoleHomeTreadmill.Arn

with the deploy command:

aws cloudformation deploy --template-file cfn/grafana/grafana-role.yaml --stack-name grafana-role --capabilities CAPABILITY_IAM

and the get the Role ARN in order to use it to create a workspace:

aws cloudformation describe-stacks --stack-name grafana-role --query "Stacks[0].Outputs[0].OutputValue"

The output looked something like this:

"arn:aws:iam::<account id>:role/grafana-role-AmazonGrafanaServiceRoleHomeTreadmill-<id>"

which I’ve used to create a Grafana workspace that is authenticated by AWS_SSO in the current account:

aws grafana create-workspace --account-access-type CURRENT_ACCOUNT --authentication-providers AWS_SSO --permission-type SERVICE_MANAGED --workspace-data-sources TIMESTREAM --workspace-role-arn <ARN from grafana-role stack>

Once that was done, I was able to see the new workspace and get the Grafana endpoint:

aws grafana list-workspaces

{
    "workspaces": [
        {
            "authentication": {
                "providers": [
                    "AWS_SSO"
                ]
            },
            "created": "...",
            "description": "",
            "endpoint": "<id>.grafana-workspace.<region>.amazonaws.com",
            "grafanaVersion": "8.4",
            "id": "...",
            "modified": "...",
            "name": "home",
            "notificationDestinations": [],
            "status": "ACTIVE",
            "tags": {}
        }
    ]
}

At this point, everything is preconfigured to create my dashboard, with Timestream as my source. The first panel is used to graph the speed changes:

With the query:

SELECT time, measure_value::double as speed FROM $__database.$__table WHERE  measure_name='speed'

and other three to show Latest Speed:

SELECT measure_value::double as speed FROM $__database.$__table ORDER BY time DESC LIMIT 1

Average Speed

SELECT AVG(measure_value::double) FROM $__database.$__table WHERE measure_name='speed'

Max Speed

SELECT MAX(measure_value::double) FROM $__database.$__table WHERE measure_name='speed'

With the end goal looking like this:

After running on my treadmill I can see the data coming in in real-time which will allow me to measure my progress and hopefully increase my maxim running speed.

Conclusion

As this started just as an inspiration from an IoT Builder Session I’m super happy how the whole process in setting this up lasted just couple of hours and at the end I got something that I’ll be using almost everyday. Additionally, having managed services to help me out on this journey meant I could have just focused on the RPi code and not worry about the complexity of the infrastructure.

If you are interested in more info about the project, or want to try out something similar, take a look at the repo in our IoT Builders Github organisation: https://github.com/aws-iot-builder-tools/ggv2-home-treadmill-example. As well as find a video that follows this blog here: https://youtu.be/o6GhNFYXZKY

Additionally if you have questions, fell free to reach out on LinkedIn or Twitter.

Debugging C/C++ AWS IoT Greengrass Components using VSCode

Nenad Ilic — Wed, 02 Nov 2022 08:49:53 +0000

In most scenarios, simply logging the output of the application is enough to understand what is happening and perform debugging in case there is an application misbehaviour. However sometimes in order to catch some stubborn bugs stepping through the code while application is executing might be the most efficient way to do it. AWS IoT Greengrass provides a CLI tool that allows local components to be deployed and tested by also accessing the application logs, but in the scenario where we want to step through the code and understand the stack and other parts of the memory of a C/C++ component than we would need some additional configuration, which we will cover in this post.

Prerequisites

For this setup I will be using an EC2 instance running Ubuntu 22.04 and having tools like gdb, gdbserver, cmake and build-essential installed as well as having my VSCode configured for Remote Development using SSH. For installing AWS IoT Greengrass, I’ll be using an installation instructions provided by a Getting Started guide, but any variation of this that provides the tools mentioned above, as well as AWS IoT Greengrass and Greengrass CLI installed and running, should work.

Note that for installing Greengrass CLI together with Greengrass, supply the installer with the parameter --deploy-dev-tools true which will add the Greengrass CLI component.

Building and Running the C++ Application

Before we can build our C++ we need to make sure the we have the AWS IoT Device SDK for C++ v2 build and available to link against as it provides the Greengrass IPC library.

# Create a workspace directory to hold all the SDK files
mkdir sdk-workspace
cd sdk-workspace
# Clone the repository
git clone --recursive https://github.com/aws/aws-iot-device-sdk-cpp-v2.git
# Ensure all submodules are properly updated
cd aws-iot-device-sdk-cpp-v2
git submodule update --init --recursive
cd ..
# Make a build directory for the SDK. Can use any name.
# If working with multiple SDKs, using a SDK-specific name is helpful.
mkdir aws-iot-device-sdk-cpp-v2-build
cd aws-iot-device-sdk-cpp-v2-build
# Generate the SDK build files.
# -DCMAKE_INSTALL_PREFIX needs to be the absolute/full path to the directory.
#     (Example: "/home/ubuntu/sdk-workspace/aws-iot-device-sdk-cpp-v2-build).
cmake -DCMAKE_INSTALL_PREFIX="/home/ubuntu/sdk-workspace/aws-iot-device-sdk-cpp-v2-build" ../aws-iot-device-sdk-cpp-v2
# Build and install the library. Once installed, you can develop with the SDK and run the samples
# -config can be "Release", "RelWithDebInfo", or "Debug"
cmake --build . --target install --config "Debug"

If everything builds successfully we can now go back and create our application directory and a C++ example:

cd ../../
mkdir hello-world
cd hello-world

In here let’s create a simple CMakeLists.txt file:

cmake_minimum_required(VERSION 3.1)
project (hello-world)

file(GLOB MAIN_SRC
        "*.h"
        "*.cpp"
        )
add_executable(${PROJECT_NAME} ${MAIN_SRC})

set_target_properties(${PROJECT_NAME} PROPERTIES
        LINKER_LANGUAGE CXX
        CXX_STANDARD 11)

find_package(aws-crt-cpp PATHS ~/sdk-workspace/aws-iot-device-sdk-cpp-v2-build)
find_package(EventstreamRpc-cpp PATHS ~/sdk-workspace/aws-iot-device-sdk-cpp-v2-build)
find_package(GreengrassIpc-cpp PATHS ~/sdk-workspace/aws-iot-device-sdk-cpp-v2-build)
target_link_libraries(${PROJECT_NAME} AWS::GreengrassIpc-cpp)

Finally let’s create our c++ example file which will subscribe to a specific MQTT topic (test/topic/cpp) and print out received messages main.cpp:

#include <iostream>
#include <thread>

#include <aws/crt/Api.h>
#include <aws/greengrass/GreengrassCoreIpcClient.h>

using namespace Aws::Crt;
using namespace Aws::Greengrass;

class IoTCoreResponseHandler : public SubscribeToIoTCoreStreamHandler {

    public:
        virtual ~IoTCoreResponseHandler() {}

    private:

        void OnStreamEvent(IoTCoreMessage *response) override {
            auto message = response->GetMessage();
            if (message.has_value() && message.value().GetPayload().has_value()) {
                auto messageBytes = message.value().GetPayload().value();
                std::string messageString(messageBytes.begin(), messageBytes.end());
                std::string messageTopic = message.value().GetTopicName().value().c_str();
                std::cout << "Received new message on topic: " << messageTopic << std::endl;
                std::cout << "Message: " << messageString << std::endl;
            }
        }

        bool OnStreamError(OperationError *error) override {
            std::cout << "Received an operation error: ";
            if (error->GetMessage().has_value()) {
                std::cout << error->GetMessage().value();
            }
            std::cout << std::endl;
            return false; // Return true to close stream, false to keep stream open.
        }

        void OnStreamClosed() override {
            std::cout << "Subscribe to IoT Core stream closed." << std::endl;
        }
};

class IpcClientLifecycleHandler : public ConnectionLifecycleHandler {
    void OnConnectCallback() override {
        std::cout << "OnConnectCallback" << std::endl;
    }

    void OnDisconnectCallback(RpcError error) override {
        std::cout << "OnDisconnectCallback: " << error.StatusToString() << std::endl;
        exit(-1);
    }

    bool OnErrorCallback(RpcError error) override {
        std::cout << "OnErrorCallback: " << error.StatusToString() << std::endl;
        return true;
    }
};

int main() {
    String topic("test/topic/cpp");
    QOS qos = QOS_AT_LEAST_ONCE;
    int timeout = 10;

    ApiHandle apiHandle(g_allocator);
    Io::EventLoopGroup eventLoopGroup(1);
    Io::DefaultHostResolver socketResolver(eventLoopGroup, 64, 30);
    Io::ClientBootstrap bootstrap(eventLoopGroup, socketResolver);
    IpcClientLifecycleHandler ipcLifecycleHandler;
    GreengrassCoreIpcClient ipcClient(bootstrap);
    auto connectionStatus = ipcClient.Connect(ipcLifecycleHandler).get();
    if (!connectionStatus) {
        std::cerr << "Failed to establish IPC connection: " << connectionStatus.StatusToString() << std::endl;
        exit(-1);
    }

    SubscribeToIoTCoreRequest request;
    request.SetTopicName(topic);
    request.SetQos(qos);
    auto streamHandler = MakeShared<IoTCoreResponseHandler>(DefaultAllocator());
    auto operation = ipcClient.NewSubscribeToIoTCore(streamHandler);
    auto activate = operation->Activate(request, nullptr);
    activate.wait();

    auto responseFuture = operation->GetResult();
    if (responseFuture.wait_for(std::chrono::seconds(timeout)) == std::future_status::timeout) {
        std::cerr << "Operation timed out while waiting for response from Greengrass Core." << std::endl;
        exit(-1);
    }

    auto response = responseFuture.get();
    if (response) {
        std::cout << "Successfully subscribed to topic: " << topic << std::endl;
    } else {
        // An error occurred.
        std::cout << "Failed to subscribe to topic: " << topic << std::endl;
        auto errorType = response.GetResultType();
        if (errorType == OPERATION_ERROR) {
            auto *error = response.GetOperationError();
            std::cout << "Operation error: " << error->GetMessage().value() << std::endl;
        } else {
            std::cout << "RPC error: " << response.GetRpcError() << std::endl;
        }
        exit(-1);
    }

    // Keep the main thread alive, or the process will exit.
    while (true) {
        std::this_thread::sleep_for(std::chrono::seconds(10));
    }

    operation->Close();
    return 0;
}

At this point we should be able to build the application:

mkdir build
cd build
cmake -DCMAKE_PREFIX_PATH="/home/ubuntu/sdk-workspace/aws-iot-device-sdk-cpp-v2-build" -DCMAKE_BUILD_TYPE="Debug" ..
cmake --build . --config "Debug"

Since this can work only in the context of Greengrass we need to prepare the component yaml file as well as the artifact:

cd ..
mkdir -p gg/artifacts/com.example.HelloWorld/1.0.0
mkdir -p gg/recipes
touch gg/recipes/com.example.HelloWorld-1.0.0.yaml

Where the content of the com.example.HelloWorld-1.0.0.yaml would be:

RecipeFormatVersion: '2020-01-25'
ComponentName: com.example.HelloWorld
ComponentVersion: 1.0.0
ComponentDescription: My C++ component.
ComponentConfiguration:
  DefaultConfiguration:
    accessControl:
      aws.greengrass.ipc.mqttproxy:
        com.example.HelloWorld:mqttproxy:1:
          policyDescription: Allows access to subscribe to a topics.
          operations:
            - aws.greengrass#SubscribeToIoTCore
          resources:
            - "test/topic/cpp"
Manifests:
- Platform:
    os: linux
  Lifecycle:
    Run: "{artifacts:path}/hello-world"

Now we can copy the hello-world binary to artifacts directory and deploy the component to Greengrass:

cp build/hello-world gg/artifacts/com.example.HelloWorld/1.0.0/

sudo /greengrass/v2/bin/greengrass-cli deployment create \
  --recipeDir gg/recipes \
  --artifactDir gg/artifacts \
  --merge "com.example.HelloWorld=1.0.0"

After this if look at the logs, we should see:

sudo bash -c "cat /greengrass/v2/logs/com.example.HelloWorld.log"
...
com.example.HelloWorld: stdout. Successfully subscribed to topic: test/topic/cpp. {scriptName=services.com.example.HelloWorld.lifecycle.Run, serviceName=com.example.HelloWorld, currentState=RUNNING}

Once we verify that this is working properly, we can jump to the part where we actually do a step through debugging.

Debugging using GDB and VSCode

First let’s set up our .vscode/launch.json to use the gdbserver

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "HelloWorld GG Component",
            "type": "cppdbg",
            "request": "launch",
            "program": "${workspaceFolder}/hello-world/gg/artifacts/com.example.HelloWorld/1.0.0/hello-world",
            "miDebuggerServerAddress": "localhost:9091",
            "args": [],
            "stopAtEntry": true,
            "cwd": "${workspaceRoot}",
            "environment": [],
            "externalConsole": false,
            "serverStarted": "Listening on port",
            "filterStderr": true,
            "setupCommands": [
                {
                    "description": "Enable pretty-printing for gdb",
                    "text": "enable-pretty-printing",
                    "ignoreFailures": true,
                }
            ],
            "MIMode": "gdb",
        }
    ]
}

While this is a typical configuration two things we need to make sure are set right.

program is pointing to where the actual artifact is being installed by the Greengrass CLI,
miDebuggerServerAddress is set to right host and port. In this scenario since we are doing this locally we will be using the localhost and port of choice is 9091 which we need to match when modifying the Greengrass component recipe.

Next we will instruct Greengrass to start the gdbserver when starting the component, which will allow us to use gdb for remote debugging.

RecipeFormatVersion: '2020-01-25'
ComponentName: com.example.HelloWorld
ComponentVersion: 1.0.0
ComponentDescription: My C++ component.
ComponentConfiguration:
  DefaultConfiguration:
    accessControl:
      aws.greengrass.ipc.mqttproxy:
        com.example.HelloWorld:mqttproxy:1:
          policyDescription: Allows access to subscribe to a topics.
          operations:
            - aws.greengrass#SubscribeToIoTCore
          resources:
            - "test/topic/cpp"
Manifests:
- Platform:
    os: linux
  Lifecycle:
    Run: "gdbserver :9091 {artifacts:path}/hello-world"

Here the only difference is the Run command which we prefixed with gdbserver :9091. After this is done we can redeploy the component:

sudo /greengrass/v2/bin/greengrass-cli deployment create \
  --recipeDir gg/recipes \
  --artifactDir gg/artifacts \
  --merge "com.example.HelloWorld=1.0.0"

After which we should see the following in the component logs:

sudo bash -c "cat /greengrass/v2/logs/com.example.HelloWorld.log"
com.example.HelloWorld: stderr. Listening on port 9091. {scriptName=services.com.example.HelloWorld.lifecycle.Run, serviceName=com.example.HelloWorld, currentState=RUNNING}

At this point, we can just create a breakpoint and start the debugging session:

We can then put a break point at line 19, which we will let us stop the application upon a retrieval of a message from the AWS IoT Core.
In order to test this we can go to AWS console → AWS IoT Core → MQTT test client → Publish to a topic and publish a message to the test/topic/cpp like the example bellow.

We will be able to catch this with the breakpoint and step through if necessary.

Additionally after the message got received and processed we will see it in the component log as well.

Conclusion

This setup allows us to use gdbserver / gdb and VSCode to visualise and inspect what is happening in our GGv2 components. We can go further and even debug multiple components at the same time, where the only thing we need to change would be the gdbserver ports on which those applications are running and modify it in the recipe respectively.

If you find this interesting or have suggestions for future topics feel free to reach out here or @nenadilc84 on Twitter or LinkedIn.

There is also a video version of this blog