Azure IoT Edge Integration Test template - Part.3

Summary

This article is a part of series of article about azure-iot-edge-integration-test-template. Following Part.2, I am going to focus on ManifestGenerator in this article.

Deployment manifest

In this sample template, the .NET app ManifestGenerator running on Azure Pipelines agent generates Azure IoT Edge Deployment Manifest and deploys it to Azure IoT Hub. IoT Edge modules connected to the IoT Hub are going to pull container images pointed on the manifest.

What ManifestGenerator does

The main purpose of ManifestGenerator app is to generate manifest.json. It includes the information of all IoT Edge modules including system default and custom.

Default module: edgeAgent and edgeHub are default modules that manage modules and their communications.
Custom module: You can specify what modules you want to deploy to the IoT Edge environment.
IoT Edge communication route: You have to specify route name here. This is very important that those names are synchronized with routes defined in each module apps. For example, the route specified in the manifest is like FROM /messages/modules/IothubConnector/outputs/reportRequest INTO BrokeredEndpoint("/modules/WeatherObserver/inputs/reportRequest"). IothubConnector module needs to specify reportRequest route name on its send message method, and WeatherObserver to specify reportRequest route name on its callback method.
Image URL, Environment variables, Bind mount directory: You have to specify image URL, environment variables, bind mount directory, and other configurations like below.

You can see a sample here - manifest.example.json

"FileGenerator": {
  "version": "1.0",
  "type": "docker",
  "status": "running",
  "restartPolicy": "always",
  "settings": {
    "image": "crsample1.azurecr.io/file-generator:20220818.2",
    "createOptions": "{\"HostConfig\":{\"Binds\":[\"/edge/upload/reports:/genroot\"]}}"
  },
  "env": {
    "OUTPUT_DIRECTORY_PATH": {
      "value": "/genroot"
    },
    "ROS_TOPIC_NAME": {
      "value": "ros2_topic_download"
    }
  }
},

Configurations

In this section, I am going to describe how to set configurations for manifest.json.

Extract credentials

manifest.json needs some credentials including IoT Hub connection string, Container Registry key, Storage Account key, local blob storage key. The first three can be extracted through Azure CLI command as Azure Pipelines agent has the access right with Azure Service Connection. For the local blob storage key, it could be anything of base64 string with length in 16 bytes. The Azure Pipeline agent generates the one randomly and sets it as environment variables.

az extension add --name azure-iot
iothubcs=$(az iot hub connection-string show --hub-name $(IOTHUB_NAME) -o tsv)
echo "##vso[task.setvariable variable=iotHubCs]$iothubcs"
acrkey=$(az acr credential show --name $(ACR_NAME) --query passwords[0].value -o tsv)
echo "##vso[task.setvariable variable=AcrKey]$acrkey"
storagekey=$(az storage account keys list --resource-group $(RESOURCE_GROUP_NAME) --account-name $(STORAGE_ACCOUNT_NAME) --query [0].value -o tsv)
echo "##vso[task.setvariable variable=storageAccountKey]$storagekey"
localstoragekey=$(openssl rand -base64 16)
echo "##vso[task.setvariable variable=LocalStorageKey]$localstoragekey"

Environment variables

With Azure Pipelines .NET Core CLI task, you can pass neccesary variables as .NET environment variables to ManifestGenerator app when running it. Those variables include credentials Azure Pipelines agent generated in the previous step.

- task: DotNetCoreCLI@2
  displayName: Generate/deploy IoT Edge manifest
  inputs:
    command: run
    projects: $(Build.SourcesDirectory)/src/apps/ManifestGenerator/ManifestGenerator/ManifestGenerator.csproj
    arguments: --configuration Release
  env:
    STORAGE_ACCOUNT_NAME: $(STORAGE_ACCOUNT_NAME)
    STORAGE_ACCOUNT_KEY: $(storageAccountKey)
    ACR_NAME: $(ACR_NAME)
    ACR_PASS: $(AcrKey)
    IOTHUB_CONNECTOR_IMAGE: $(ACR_NAME).azurecr.io/${{ parameters.EdgeImages.module1.repository }}:${{ parameters.EdgeImages.module1.tag }}
    WEATHER_OBSERVER_IMAGE: $(ACR_NAME).azurecr.io/${{ parameters.EdgeImages.module2.repository }}:${{ parameters.EdgeImages.module2.tag }}
    FILE_GENERATOR_IMAGE: $(ACR_NAME).azurecr.io/${{ parameters.EdgeImages.module3.repository }}:${{ parameters.EdgeImages.module3.tag }}
    FILE_UPLOADER_IMAGE: $(ACR_NAME).azurecr.io/${{ parameters.EdgeImages.module4.repository }}:${{ parameters.EdgeImages.module4.tag }}
    FILE_UPDATER_IMAGE: $(ACR_NAME).azurecr.io/${{ parameters.EdgeImages.module5.repository }}:${{ parameters.EdgeImages.module5.tag }}
    IOTHUB_DEVICE_ID: $(IOTHUB_DEVICE_ID)
    IOTHUB_CONNECTION_STRING: $(iotHubCs)
    LOCAL_STORAGE_KEY: $(LocalStorageKey)
    ORGANIZATION_NAME: $(TEST_ORGANIZATION_NAME)

appsettings.json

ManifestGenerator app retains appsettings.json that specifies design values. You do not need to change it unless you change the design such as bind mount paths or SAS token expiration period.

{
  "RouteTelemetry": "telemetry",
  "RouteReportRequest": "reportRequest",
  "RouteReportResponse": "reportResponse",
  "RouteUpdateRequest": "updateRequest",
  "RouteUpdateResponse": "updateResponse",
  "Ros2Topic": "ros2_topic_download",
  "FileGeneratorContainerBind": "/edge/upload/reports:/genroot",
  "FileUploaderContainerBind": "/edge/upload:/uploadroot",
  "FileUpdaterContainerBind": "/edge/download:/downloadroot",
  "LocalBlobStorageBind": "/edge/localblob:/blobroot",
  "CloudBlobContainerName": "weather",
  "LocalBlobContainerName": "weather",
  "LocalBlobAccountName": "stlocal",
  "LocalBlobEndpoint": "http://LocalBlobStorage:11002",
  "FileGeneratorWorkdir": "/genroot",
  "FileUploaderWorkdir": "/uploadroot",
  "FileUpdaterWorkdir": "/downloadroot",
  "SasExpirationMonths": 6
}

IoTEdgeObjectModel NuGet package

This sample template uses IoTEdgeObjectModel NuGet package. This package helps you reduce lines of code you write for manifest.json. You do not need to specify system module configuration or other default configurations. In my experience in the last project, I wrote a manifest.json all by myself from scratch with C# dictionary instances. By using IoTEdgeObjectModel package, you can reduce roughly 50% of your codes.

The three main classes of this package are EdgeAgentDesiredProperties, EdgeHubDesiredProperties, ModuleSpecificationDesiredProperties.

You specifies edgeAgent properties like below with EdgeModuleSpecification class.

EdgeAgentDesiredProperties edgeAgentDesiredProperties = new ()
{
    SystemModuleVersion = "1.3",
    RegistryCredentials = new List<RegistryCredential>()
    {
        new RegistryCredential(acrName, $"{acrName}.azurecr.io", acrName, acrPass),
    },
    EdgeModuleSpecifications = new List<EdgeModuleSpecification>()
    {
        new EdgeModuleSpecification(name:"IothubConnector", image:iothubConnectorImage, environmentVariables:iothubConnectorEnv),
        new EdgeModuleSpecification(name:"WeatherObserver", image:weatherObserverImage),
        new EdgeModuleSpecification(name:"FileGenerator", image:fileGeneratorImage, createOptions:fileGeneratorCreateOptions, environmentVariables:fileGeneratorEnv),
        new EdgeModuleSpecification(name:"FileUploader", image:fileUploaderImage, createOptions:fileUploaderCreateOptions, environmentVariables:fileUploaderEnv),
        new EdgeModuleSpecification(name:"FileUpdater", image:fileUpdaterImage, createOptions:fileUpdaterCreateOptions, environmentVariables:fileUpdaterEnv),
        new EdgeModuleSpecification(name:"LocalBlobStorage", image:localBlobStorageImage, createOptions:localBlobStorageCreateOptions, environmentVariables:localBlobStorageEnv),
    },
};

EdgeHubDesiredProperties mainly specifies Azure IoT Edge route communication.

EdgeHubDesiredProperties edgeHubConfig = new ()
{
    Routes = new List<Route>()
    {
        new Route("route_telemetry", route_telemetry),
        new Route("route_c2w", route_c2w),
        new Route("route_w2c", route_w2c),
        new Route("route_w2u", route_w2u),
        new Route("route_u2w", route_u2w),
    },
};

ModuleSpecificationDesiredProperties specifies custom modules and their module twin desired properties.

ModuleSpecificationDesiredProperties localBlobStorage = new ()
{
    Name = "LocalBlobStorage",
    DesiredProperties = new Dictionary<string, object>
    {
        ["deviceAutoDeleteProperties"] = new Dictionary<string, object>
        {
            ["deleteOn"] = true,
            ["deleteAfterMinutes"] = 5,
            ["retainWhileUploading"] = true,
        },
        ["deviceToCloudUploadProperties"] = new Dictionary<string, object>
        {
            ["uploadOn"] = true,
            ["uploadOrder"] = "NewestFirst",
            ["deleteAfterUpload"] = true,
            ["cloudStorageConnectionString"] = cloudStorageSasConnectionString,
            ["storageContainersForUpload"] = new Dictionary<string, object>
            {
                [localBlobContainerName] = new Dictionary<string, object>
                {
                    ["target"] = iotHubDeviceId,
                }
            },
        },
    },
};

SAS token

ManifestGenerator has a service class SasService.cs that generates a SAS(Shared Access Signature) token. In this way, you can have one Azure Blob Storage for multiple edge devices from different entities. This SAS token makes edge devices follow the security boundary so edge devices cannot access data of different entities.

It is important to convert a SAS token generated to a connection string. This connection string is set as an environment variable of LocalBlobStorage. For FileUpdater, you can use AzureSasCredential to convert the SAS token into the one readable for BlobClient.

string[] sasContents = weatherFileInfo.BlobSasUrl.Split('?');
AzureSasCredential azureSasCredential = new (sasContents[1]);
Uri blobUri = new (sasContents[0]);
BlobClient blobClient = new (blobUri, azureSasCredential, null);
await blobClient.DownloadToAsync(zipFilePath).ConfigureAwait(false);

However, LocalBlobClient by default needs a blob connection string, not SAS token. So you need to convert the SAS token into a blob connection string in ManfiestGenerator Program.cs

string sasUri = directoryClient.GenerateSasUri(sasBuilder).ToString();
string[] sasContents = sasUri.Split('?');
string sasConnectionString = $"BlobEndpoint=https://{this.dataLakeServiceClient.AccountName}.blob.core.windows.net/{blobContainerName}/{sasDirectory};SharedAccessSignature={sasContents[1]}";