DEV Community: Yokogawa Technologies Solutions India

Data Visualization on the e-RT3 using Node-RED, InfluxDB Cloud, and Grafana

Yokogawa-NetraGandhi — Mon, 23 Sep 2024 15:35:09 +0000

Setup
- Introduction
- Overview
  - Materials
  - Software
  - Manuals
  - Wiring
  - Network Addresses
Procedure
- SD Card Setup
- Setting up the Cloud
- Configuring the Network
- Wiring Connections
- e-RT3 Dependencies
- Boot
- Grafana Dashboard Setup
- Gateway Node-RED Setup
- e-RT3 Node-RED Setup
- Using the Application

Setup

Introduction

The purpose of this application is to receive data from sushi sensors, using the e-RT3 as a data acquisition module. The e-RT3 will send sushi data to a cloud database, InfluxDB Cloud, and will monitor the data using a visualization software, Grafana.

Overview

Materials

SD card (SanDisk 32GB Ultra SDHC)
Router (CyberTAN Technology WR214E Wireless-B Broadband Router)
PC
- e-RT3 setup
CPU module (F3RP70-2L)
- Power modules (F3PU10-0S, F3PU16-0S, F3PU20-0S, F3PU26-0S, F3PU30-0S, F3PU36-0S)
- Base modules (F3BU04-0N, F3BU06-0N, F3BU05-0D, F3BU09-0N, F3BU013-0N, F3BU16-0N)
Sushi setup (please follow manuals to configure)
- Vibration/pressure/temperature sensor
- Gateway
- Phone with "Sushi Sensor App" and "Key Card Editor"

Software

InfluxDB Cloud
- Time-based database, cloud
Node-RED (e-RT3)
- Flow-based, low code development tool
Grafana (e-RT3)
- Visualization software
Helpful tools:
- Balena Etcher (PC)
  - Writes OS images
- Windows PowerShell
  - Command line
- WinSCP
  - File manager, SFTP, FTP
- PuTTY
  - Terminal emulator
- Github documentation
  - Web-based version control

Manuals

e-RT3
- IM 34M06M52-25E
Sushi
- IM 01W06C01-01EN
Gateway
- TI01W06A51-50EN
GA10
- IM 04L65B01-01EN

Wiring

The LoRaWAN Gateway, PC, and LAN1 of e-RT3 should be connected to the router. All wiring can be connected now or connected as the manual progresses.

Network Addresses

Default Addresses

e-RT3 Web-browser:
http://192.168.3.72

Gateway Web-browser:
http://192.168.2.1
Default username: admin
Default password: Yokogawa1

Router Web-browser:
http://192.168.1.1
*Default username: admin
*Default password: admin
*Check your router’s default settings

Final Addresses

e-RT3 Web-browser:
http://192.168.2.72

e-RT3 Grafana Server:
http://192.168.2.72:3000
Default username: admin
Default password: admin

e-RT3 Node-RED Server:
http://192.168.2.72:1880

Gateway Web-browser:
https://192.168.2.4/
Default username: admin
Default password: Yokogawa1

Gateway Node-RED Server:
https://192.168.2.4:1880

Router Web-browser:
http://192.168.2.1
*Default username: admin
*Default password: admin
*Check your router’s default settings

Procedure

1) SD Card Setup

Please image the following file onto an SD card. The CPU module supports SDHC 4GB to 32GB. The
recommended SD card is a SanDisk Memory Card.

a. Download balenaEtcher.
b. Un-zip and extract the “eRT3 + Sushi Sensor Application” folder to a desired location.
c. Login or create an account on the Yokogawa Customer Portal.
d. Please locate the latest SD card image and click on the file to download it. You can navigate
from the homepage to “[Useful Links]” “PLC/PAC & Embedded Controller” > “OS-free CPU
module OS image Download” > “Agree” > “OS image for F3RP70”
“rp70_OS_Image_R211.zip”.
OR click on the following link.

e. Use a SD card reader to insert the SD card into the PC.
f. Open balenaEtcher. Select the OS image. Select target (SD card). Select Flash!

g. You may now insert the SD card into the SD1 slot of the e-RT3.

2) Setting up the Cloud

a. Create an account on InfluxDB Cloud. Use the official InfluxDB website for more information.
b. Fill out the prompts for company and organization after logging in. My company name is “Yokogawa Corporation of America”. My organization name is “eRT3”. Make sure to select the correct region for the server. I selected “US East”.
c. Choose the “Free” plan.
d. Once you have reached the dashboard, go to “API Tokens”.

e. Select “Generate API Token” > “All Access API Token”. Enter a description for the API Token. Click “Save”.

f. Please copy and paste the custom API token into a safe location where you can access it in the future.
g. Navigate to “Load Data” > “Buckets” from the side bar.
h. Click on “Create Bucket” and name the bucket “sushiSensor”. If you are on the free version, keep the “Data Retention Preferences” as “30 days”. Click “Create”.

i. Please note down the server address located in your URL bar. You can learn more about your cloud region here. For example, if you live in the US East, please note down: https://us-east-1-1.aws.cloud2.influxdata.com/.

3) Configuring the Network

To configure the network and change IP addresses of various devices, the PC IP address will need to be within the same network as the device it is communicating with. This will involve changing the PC IP address various times using the following steps:

a. Go to your PC’s “Control Panel” > “Network and Internet” > “Network Connections”. Select the Ethernet connection to the router.
b. Double-click on “Properties”.
c. Double-click “Internet Protocol Version 4 (TCP/IPv4)”.

d. Select “Use the following IP address:”.
e. Change the PC IP address to one that is in the same network as the device’s address. For example, the default e-RT3 IP address is 192.168.3.72. Therefore, the PC address would need to be changed to 192.168.3.X (you may choose anything for X that has not already been assigned). Make the Subnet mask 255.255.255.0 (smaller network).

Changing the e-RT3 IP address

a. On the physical device itself, check to see if the e-RT3 is on Command Mode: 0 (the arrow should be pointing to 0). Make sure the e-RT3 is mounted to the Base module and is being powered through the Power Supply module. Refer to IM 34M06M52-25E for more details.

b. Using an ethernet cord, connect the PC directly to LAN1 of the e-RT3.
c. Change the PC IP address to one that is in the same network as the e-RT3 (Use Ex: 192.168.3.3).
d. This manual will be using Windows PowerShell as its terminal. If not already installed on your PC, please use the following link to install or find an alternative terminal.
e. Wait until the display on the e-RT3 lights up “OS” under “BAT”. This means that the operating system is now running. If you try to SSH before the “OS” appears, it will not work.
f. SSH into the e-RT3 using the following commands:

ssh ert3@192.168.3.72

When prompted for the password, type (note: the typing will be invisible):

user_ert3

g. (Optional): If you have used this SSH key before and it notifies you that remote host identification has changed, you may need to locate and open your “known_hosts” file (may be located in the .ssh folder) in a text editor and remove the all the lines that includes the IP address and the key (corresponding to 192.168.3.72). You can then save the “known_hosts” file and attempt to SSH again. This time you will be prompted to add and confirm a new host key to the “known_hosts” file. Type “yes” if prompted.
h. To make sure the DNS servers are specified, we need to change the resolved.conf file. Type the command:

sudo nano /etc/systemd/resolved.conf

When prompted for the password, type (note: it will be invisible):

user_ert3

i. Once the file opens in the terminal, uncomment the DNS and Fallback DNS lines by deleted the “#” and add the Google DNS server addresses as shown below. Please note there is a space between “8.8.8.8” and “8.8.4.4”:

[Resolve]
# Some examples of DNS servers which may be used for DNS= and FallbackDNS=:
# Cloudflare: 1.1.1.1 1.0.0.1 2606:4700:4700::1111 2606:4700:4700::1001
# Google:     8.8.8.8 8.8.4.4 2001:4860:4860::8888 2001:4860:4860::8844
# Quad9:      9.9.9.9 2620:fe::fe
DNS=8.8.8.8 8.8.4.4
FallbackDNS=8.8.8.8 8.8.4.4
#Domains=
#DNSSEC=no
#DNSOverTLS=no
#MulticastDNS=yes
#LLMNR=yes
#Cache=yes
#DNSStubListener=yes
#DNSStubListenerExtra=
#ReadEtcHosts=yes
#ResolveUnicastSingleLabel=no

j. On your keyboard, use the shortcut “ctrl + s” to save and then “ctrl + x” to exit the program.
k. Restart systemd-resolved:

sudo systemctl restart systemd-resolved

l. To change the IP address for LAN1, we need to change its network configuration file:

sudo nano /etc/systemd/network/10-eth0.network

If prompted for the password, type (note: it will be invisible):

user_ert3

m. Once the file opens in the terminal, change the text in the file so that it matches to the routes below:

[Match]
Name=eth0

[Network]
Address=192.168.2.72/24

[Route]
Gateway=192.168.2.1
Destination=192.168.2.0/24

[Route]
Gateway=192.168.2.1
Destination=0.0.0.0/0

n. On your keyboard, use the shortcut “ctrl + s” to save and then “ctrl + x” to exit the program.
o. Restart systemd-networkd:

sudo systemctl restart systemd-networkd

p. Turn the e-RT3 off and on. The e-RT3 and PC are now on the same network and the e-RT3 has internet access.
q. (Optional): Change the PC IP address to one that is in the same network as the e-RT3 (Ex: 192.168.2.3). Ping 192.168.2.72 to make sure the PC and the e-RT3 are now on the same network:

ping 192.168.2.72

If you can send and receive packets successfully, the setup is correct.

r. You may now remove the ethernet cord.

Changing the Gateway IP address

If further information is needed to set up the Gateway, please refer to TI01W06A51-50EN.
a. Using an ethernet cord, connect the PC directly to the Gateway.
b. Change the PC IP address to one that is in the same network as the Gateway (Ex: 192.168.2.3). The default IP address of the Gateway is 192.168.2.1.
c. Access the Gateway web-browser by entering “192.168.2.1” into the URL bar.
d. An error message may appear but click on “Advanced” > “Proceed to 192.168.2.1 (unsafe)”.

e. Login into the Gateway web-browser with the following default username and password:

Username: admin

Password: Yokogawa1

f. On the left side of the dashboard, select “Setup” > “Network Interfaces”.
g. Click the pen under “Options” to edit the IP address visible under the name “br0”.

h. Change the IP address to 192.168.2.4 and hit “Submit”.

i. Click on “Save and Restart” and press “OK” when prompted.
j. The Gateway IP address is now changed. Give the web-browser some time to restart. Access the web-browser now at https://192.168.2.4:1880/ in the URL bar.
k. You may now remove the ethernet cord.

Changing the router IP address

This procedure is for the CyberTAN Technology router WR214E. Please refer to your own router’s manual if different.

a. Using an ethernet cord, connect the PC directly to one of the ports of the router. Make sure the router is connected to the internet and is powered.
b. Reference your router’s manual to change its IP address. For example, the router in this setup, WR214E, has a default web-browser of http://192.168.1.1 and a default username and password: admin. You will need to change the PC IP address to one that is in the same network as the router temporarily (Ex: 192.168.1.2).
c. After changing the router IP address (to 192.168.2.1) and applying the changes within the web-browser, turn the router off and on.
d. Change the PC IP address to one that is in the same network as the router (Ex: 192.168.2.2). You can now access the router web-browser at http://192.168.2.1.
e. You may now remove the ethernet cord.

4) Wiring Connections

a. Please connect all devices to the same network via the router. The PC, LAN1 of the e-RT3, and Gateway can all plug into the router with an ethernet cord. The router needs to connect to an ethernet cable wired with internet connectivity. Make sure all devices are powered. The PC is only optional after the application is setup.

5) e-RT3 Dependencies

A couple of different dependencies will need to be built on the e-RT3: NVM (Node Version Manager), Node.js, npm, Node-RED, Grafana. To interact with the e-RT3, you will need to SSH into it and continue to work in PowerShell.

Installing NVM, Node.js, and npm

Please refer to this manual for more information on how to install NVM. NVM is used to install Node.js and npm while maintaining various Node.js versions.

a. SSH into the e-RT3 (using the steps shown in the “Configuring the Network” section of this manual.
b. Run the following command to download and install NVM:

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh | bash

c. Restart PowerShell by closing and opening it.
d. SSH into the e-RT3.
e. Verify that NVM was properly installed by running the following command:

nvm --version

f. Install the latest version of Node.js using the following command:

nvm install --lts

g. Verify that Node.js and npm were properly installed by running the following commands:

node --version

npm --version

Installing Node-RED

a. If needed, SSH into the e-RT3.
b. Run the following command to install Node-RED:

npm install -g --unsafe-perm node-red

c. Verify that Node-RED was properly installed by running the following command:

node-red --version

d. To start the Node-RED server, run the following command:

node-red

If the server is running and you want to access the command line, you will have to open a new terminal as the current terminal will be preoccupied with the server.

e. You may now access the Node-RED web-browser on the e-RT3 at http://192.168.2.72:1880/.

Installing Grafana

a. If needed, SSH into the e-RT3.
b. Update the packages list on the e-RT3:

sudo apt-get update

c. Run the following commands to install Grafana:

sudo apt-get install -y adduser libfontconfig1 musl

wget --no-check-certificate https://dl.grafana.com/oss/release/grafana_10.0.6_armhf.deb

sudo dpkg -i grafana_10.0.6_armhf.deb

d. To start the Grafana server, run the following command:

sudo systemctl start grafana-server

e. You may now access the Grafana web-browser on the e-RT3 at http://192.168.2.72:3000/

6) Boot

Once Node-RED and Grafana are installed, their servers must be manually started through the command line every time the e-RT3 is restarted. For convenience, it is recommended to automatically start the servers when the e-RT3 is booting.

Node-RED Boot

a. If needed, SSH into the e-RT3.
b. To add Node-RED to the systemd boot, we will be using PM2, a process manager for Node.js. Run the following commands:

npm install -g pm2

Please change the highlighted with the latest version of Node.js installed, if different. Use the command “node –version” to find the latest version.

pm2 start /home/ert3/.nvm/versions/node/v20.15.1/bin/node-red

pm2 save

pm2 startup systemd

The following command is given within the terminal and may change depending on your setup. If different than below, please follow the instructions for the path given in the terminal.

sudo env PATH=$PATH:/home/ert3/.nvm/versions/node/v20.15.1/bin /home/ert3/.nvm/versions/node/v20.15.1/lib/node_modules/pm2/bin/pm2 startup systemd -u ert3 --hp /home/ert3

c. Turn the e-RT3 off and on.
d. Verify that Node-RED is properly started at boot by accessing the web-browser (http://192.168.2.72:1880/) without any additional commands.

Grafana Boot

a. If needed, SSH into the e-RT3.
b. To add Grafana to the systemd boot, run the following commands:

sudo /bin/systemctl daemon-reload

sudo /bin/systemctl enable grafana-server

sudo /bin/systemctl start grafana-server

c. Turn the e-RT3 off and on.
d. Verify that Grafana is properly started at boot by accessing the web-browser (http://192.168.2.72:3000/) without any additional commands.

7) Grafana Dashboard Setup

We will be setting up the Grafana Dashboard and its connection to the InfluxDB Cloud server.

a. Access the Grafana web-browser at http://192.168.2.72:3000/
b. Login into the Grafana web-browser with the following default username and password:

Username: admin

Password: admin

c. You can set your own password or choose to keep it the same.
d. On the home page, click the hamburger icon on the left to access “Connections” > “Data sources”.
e. Click on “Add data source”.

f. Select the InfluxDB option.
g. Under “Query Language”, select “Flux”.
h. Under “HTTP”, fill in the InfluxDB Cloud server URL: https://us-east-1-1.aws.cloud2.influxdata.com/ If you live in a different region, use the appropriate server URL.

i. Under “Auth”, turn on “Basic auth” and “Skip TLS Verify”.

Depending on your company’s security policies, you may need to assign a CA certificate rather than skipping the TLS Verify.

j. In “InfluxDB Details”, name the “Organization” as “ert3”. Under “Token”, fill in the API token generated from the InfluxDB Cloud. Click on “Save & test”.

h. Click on the hamburger icon on the left again and navigate to “Dashboards”.
i. Select “New” > “Import”.
j. Upload the " eRT3_Sushi_Dashboard-1719517428740.json" JSON file from the ZIP-folder provided with this manual.
k. Under the “InfluxDB Cloud” dropdown, select “InfluxDB (default)” and click “Import”. You can now view the dashboard panels and change them as you wish. As the Node-RED flow is not configured and no data has been sent through the cloud database, all the panels should say “No data”.

8) Gateway Node-RED Setup

We will be setting up the Gateway Node-RED flow and its connection to the e-RT3 Node-RED flow. The purpose of this flow is to receive data from the LoRaWAN and send it to the e-RT3 in the form of a JSON object. It is important to note that firmware versions of the gateway older than 5.3.0 use Node-RED. 5.3.0 or later use the “GW Custom App Generator for Sushi Sensor”. This manual uses the Node-RED firmware version. These steps can also be replicated with the Custom App version by placing http://192.168.2.72:1800/sushidata in the URL box. You can learn more in the Gateway manual TI01W06A51-50EN.

a. Access the Gateway Node-RED web-browser at https://192.168.2.4:1880/. You can also access the web-browser from the Gateway web-browser by navigating to “Apps” and then clicking on “Launch Node-RED”.
b. Login into the Gateway Node-RED web-browser with the following default username and password:

Username: admin

Password: Yokogawa1

c. The dashboard will appear. Ensure all the blocks are in place and connected by dragging and dropping them from the left-hand bar:

d. Double-click on the orange function block and use the script below:

e. Exit the function block. Double-click on the yellow input block and match the inputs below:

The Gateway Node-RED server will be making a http post request to the e-RT3 Node-RED server.

f. Make sure to click on the “Deploy” button if any changes were made.

9) e-RT3 Node-RED Setup

We will be setting up the e-RT3 Node-RED flow and its connection to the Gateway Node-RED flow and the InfluxDB Cloud server. The purpose of this flow is the translate the JSON object received from the Sushi Gateway into readable sensor data and send it to the cloud database.

a. Access the e-RT3 Node-RED web-browser at http://192.168.2.72:1880/
b. Skip through the tutorial and instructions.
c. Click on the hamburger icon on the top-right side and select “Import”.
l. Click on “select a file to import” and upload the “eRT3_Sushi_Application_NodeRED_flow.json” JSON file from the ZIP-folder provided with this manual. Click on “Import”.
d. Switch to the new tab created by the file named “eRT3_Sushi_Application_NodeRED_flow.json”.
e. You will notice that an error is thrown for the influxdb blocks. To fix this error, we will need to add the “influxdb” library.

f. Click on the hamburger icon and click on “Manage pallete”. Switch to the “Install” tab and look up the module “node-red-contrib-influxdb”. Select the first option with this exact name. When selecting which node-red packages to install, it is important to consider the last time the package was updated.

g. Click on “install”.
h. Once it is done installing, click on “Close”.
i. Double-click on the brown influxdb block named “[v2.0] Cloud sushi” that has appeared.

j. Match the inputs below if different:

The bucket name must match what was named within the InfluxDB Cloud server. If you followed the manual exactly, use “sushiSensor”. Keep the “Measurement” name as “sushi” or you will have to change things later.

k. Click on the pencil icon to edit the influxdb node.
l. Fill in the InfluxDB Cloud server URL: https://us-east-1-1.aws.cloud2.influxdata.com/ If you live in a different region, use the appropriate server URL.
m. Make sure to input the InfluxDB Cloud API token:

n. Click on “Update” and “Done”.
o. (Optional): Double-click on the leftmost yellow http request block to see it matches the code below:

The URL must match the endpoint given in the Gateway Node-RED http request (“/sushidata”).

b. Double-click the “sushiTranslation” block and copy and paste the code from the “sushi_translation_script_on_ert3.txt” text file into the “On Message” entry. You can find this file in the ZIP-folder provided with this manual.
c. If you wish to send data to GA10, draw a line from the initial post request to the GA10 post request. You must have GA10 setup on your PC. Please refer to IM 04L65B01-01EN for more information on how to connect GA10 and Sushi. Match the inputs below if different:

p. Click on “Deploy” in the top-right corner. You should receive a successful deployment message. You have now completed setting up all the software.

q. To save the flow, click on the hamburger icon. Select “Export”.
r. Click on the “Local” tab. Name the JSON file whatever you wish and click on “Export to library”.

10) Using the Application

You have completed all the steps to set up the application. To collect data and visualize it, you need to turn on the Sushi sensors that are connected to the Gateway. Vibration, temperature, and pressure sensors work with this application. Please refer to the Sushi sensor and Gateway manuals to learn more, respectively: IM 01W06C01-01EN, TI01W06A51-50EN.

a. Turn on all Sushi sensors configured with the Gateway.
b. Monitor the Grafana “eRT3_Sushi_Dashboard” as data comes through by navigating through the hamburger icon. Click on “Dashboards”. Select the “eRT3_Sushi_Dashboard”.
c. You can select which devices you wish to monitor based on their EUIs. Click on the “eui” drop-down menu and select accordingly. If you select more than one EUI to display, the panels will repeat for each EUI.

d. Depending on the sensor you are monitoring, certain panels will display “No data”. For instance, if you are displaying a vibration sensor, the panels for temperature and pressure sensors will display “No data”. Depending on the parameters selected within the Sushi app, these will appear appropriately within the panels. In the image below, a vibration sensor with the parameters for only the X-Axis is being visualized:

e. If you wish to show/hide any panels, click on the drop-down arrows next to the title names:

f. If you do not wish to have a permanent monitor, you can remove the PC from the setup. The sensors will continue to collect data and send it to the cloud database and Grafana regardless of if the PC is present.

You can now visualize Sushi sensor data on Grafana using InfluxDB Cloud and the e-RT3.
Contact netra.gandhi@yokogawa.com for questions.

Using Power BI to visualize Edge data sent to Azure IoT Hub

Yokogawa-BrigitteZacharia — Tue, 04 Apr 2023 10:00:36 +0000

Introduction

Azure IoT Runtime enables you to collect information and execute commands on Edge devices remotely. When installed on e-RT3 Plus, the features of both e-RT3 Plus and Azure Runtime Environment can be utilized to perform various operations.

This is the last part of a five-part series that demonstrates how to use Azure Runtime Environment with e-RT3 Plus. The first two parts describe how to use Azure IoT Runtime environment to create and deploy Azure IoT Edge Python modules that read and upload data to an IoT Hub. In the first part we read data from a sample temperature module, in the second part we read data from an Analog Input Edge device.

The third part describes creating and deploying an Azure IoT Edge Python module that writes data to an Edge device.

Create an Azure IoT Edge Python module that writes data to an Edge device

The fourth part describes visualizing the data received at the IoT Hub using a web application created by using Azure App Service.

Using Azure App Service to visualize Edge data sent to Azure IoT Hub

In this article, we visualize the data received at the IoT Hub by using Power BI. To send the data to Power BI, an Azure Stream Analytics job is created which connects to the Power BI workspace. The data visualizations are then configured in the designer of Power BI.

The ultimate goal for this article series is to demonstrate how to:

Create a Python data collection module to gather data, process it and upload it to IoT Hub
Create a Python module for writing data to an Edge device
Visualize the collected data using Azure App Service and Power BI.

This article is based on the Microsoft tutorial Visualize real-time sensor data from Azure IoT Hub using Power BI. For more information about installing and deploying Azure IoT Runtime environment, refer to Deploying Azure Runtime Environment on e-RT3 Plus.

Hardware modules

The following table describes the hardware modules used in this article.

Module	Description
e-RT3 Plus F3RP70-2L (CPU module, Ubuntu 18.04 32-bit)	e-RT3 Plus controls the complete module set. It accesses each module from the CPU module to read and write data. The `armhf` architecture package runs on this device.
F3AD08-6R (Analog Input module)	The Analog Input module (AD module) converts the external analog data received to digital data.
F3DA04-6R (Analog Output module)	The Analog Output (DA module) module converts the digital data sent from the e-RT3 Plus to analog data.
F3BU05-0D (Base module)	This is the base for connecting each module. It takes care of the power supply and communication between the modules that are connected to it.
F3PU20-0S (Power module)	The Power module is connected on the Base module and is used for power supply.

The module configuration and wiring is the same as described in the previous article.

For more information on the hardware modules, refer to this page.

Note: The IoT Edge module development and device operations are performed on Windows 10 using Google Chrome as the web browser.

Prerequisites

Before trying to visualize the data in the IoT Hub using Power BI, you must complete the steps mentioned in the previous articles Create an Azure IoT Edge Python module to gather data from an Edge device and transmit it to the IoT Hub and Create an Azure IoT Edge Python module that writes data to an Edge device.

Workflow

To visualize the data sent to the IoT Hub in Power BI, you must complete the following:

Install Power BI
Create IoT Hub consumer group
Create Stream Analytics
Configure Stream Analytics settings
Configure and visualize data in Power BI

Installing Power BI Desktop

Power BI is a data analytics and reporting tool from Microsoft that helps you to visualize data. You can use it as a desktop application with Power BI Desktop or as a web service with Power BI service. In this demonstration, we use Power BI Desktop which is signed into a personal account.

Note: The steps described in this article are the same if you use Power BI Desktop with a corporate account.

Follow these steps to install Power BI Desktop on your computer:

Navigate to the official website and click Download free.

The Open Microsoft Store dialog box appears.
Click Open Microsoft Store.
In the Microsoft Store window that opens, click Get to download the installer.
Install Power BI and open it.
To sign in, in the upper-right corner, click Sign-in and follow the instructions displayed.

Creating IoT Hub consumer group

To create a IoT Hub consumer group from Azure Portal, follow the steps mentioned in the previous article. For this demonstration, we have created a consumer group with the name ert3-visualization-powerbi.

Note: Ensure that you save the name of the consumer group as you will need to use it later.

Creating Stream Analytics

To send data from the IoT Hub to Power BI, we create a Steam Analytics job in Azure Portal.

Follow these steps to create a stream analytics job:

Navigate to Azure Portal.
In the upper left corner of the page, click Create a resource.
In the search box, type Stream Analytics job and select the same from the search results.
Click Create.

Configure the mandatory details as shown in the following table.

Parameter	Value
Project details - Subscription	Select any existing subscription
Project details - Resource Group	Create a new Resource Group or select any existing Resource Group
Instance details - Name	Specify any unique name
Instance details - Region	Select any region as applicable
Instance details - Hosting environment	Select Cloud
Streaming unit details - Streaming unit	In this demonstration, as we are not using any special processing for the stream data, streaming unit 3 is sufficient.

Note: Stream Analytics usage is billed based on the start time and the number of streaming units. For more information on pricing, click here.

Click Review + create.
After verifying the configured information, click Create.

A message is displayed indicating that the Stream Analytics job is deployed.

Configuring Stream Analytics settings

Follow these steps to configure the settings for the Stream Analytics job:

Navigate to Azure Portal and open the Stream Analytics job that you created.

The overview page appears.

To add an input to the Stream Analytics job, perform these steps:

Click Add input and select IoT Hub from the drop-down list.
The IoT Hub pane appears.

Configure the parameters as shown in the following table.

Parameter	Value
Input alias	Specify any name to use as an alias
Subscription	Select the Azure subscription that you are using for the IoT Hub from which you want to visualize data
IoT Hub	Select the IoT Hub from which you want to to visualize data
Consumer group	Specify the consumer group created in Creating IoT Hub consumer group

Click Save.

The IoT hub is added to the input list and a message is displayed indicating that the connection test is successful.

To add an output to the Stream Analytics job, perform these steps:
1. From the overview page, click Add output and then select Power BI from the drop-down list.
  
  The Power BI - New output pane appears.
2. Select Authorize and follow the instructions displayed to sign in to your Power BI account.
3. After you sign in to Power BI, configure the parameters as shown in the following table.
  
  Parameter Value
  
  Output alias Specify any name to use as an alias
  
  Subscription Select your target group workspace
  
  Dataset name Specify a dataset name
  
  Table name Specify a table name
4. Click Save.
  The Power BI settings are added to the ouput list and a message is displayed indicating that the connection test is successful.
To configure the query for the Stream Analytics job, perform these steps:
1. On the left pane, under Job topology, select Query.
2. In the editor that appears, type the query as follows:
```
SELECT
messageID,
deviceID,
CAST([datetime] AS datetime) AS datetime,
ch1, ch2, ch3, ch4, ch5, ch6, ch7, ch8
INTO
[<Set output alias>]
FROM
[<Set input alias>]
```
3. Click Save query.
  
  A notification appears indicating that the settings are saved.
To start the Stream Analytics job, from the overview page, click Start.

After the job is started successfully, the job status changes from Stopped to Running.

Parameter	Value
Output alias	Specify any name to use as an alias
Subscription	Select your target group workspace
Dataset name	Specify a dataset name
Table name	Specify a table name

Configure and visualize data in Power BI

Follow these steps to visualize the data in Power BI:

Open Power BI Desktop.
From the menu bar, select Home > Get data >Power BI Datasets.

The Data hub page appears.
Select the dataset that you specified in step 3 of Configuring Stream Analytics settings.

The data set is loaded and the designer appears. The Filter, Visualizations, and Fields panes appear on the right.
To design the visualization chart, perform these steps:
1. To visualize the data in a particular channel, in the Fields pane, select the checkbox next to channel name.
2. In the Visualizations pane, select the Line graph icon. You can resize the chart screen displayed in the centre as necessary.
  
  The sum of values is displayed, since the horizontal axis is not yet configured.
3. On the Fields pane, select the datetime check box and drag it to the X-axis section in the Visualizations pane.
  
  The date and time is displayed on the horizontal axis.
4. In the Filters pane, expand datetime.
5. From the Filter type drop-down list, select Relative time.
6. From the Show items when the value drop-down list, select is in the last.
7. In the value box, type 5.
8. From the drop-down list, select minutes
  
  The chart is updated to display data from the last five minutes.
To refresh the data displayed on the chart click Update.

If the visualization is successful, the following chart is displayed.
To stop sending data, after the verification is complete, in Azure Portal, expand the Stream Analytics job summary and click Stop.

Note: The Stream Analytics billing is calculated based on operation time and streaming units.

Conclusion

The data that is sent to the IoT Hub is visualized in Power BI Desktop. This concludes the article series that demonstrates how to use Azure IoT Runtime to send data to IoT Hub from an Edge device, write data to an Edge device, and visualize the data in Power BI and Azure App Service.

References

Using Azure App Service to visualize Edge data sent to Azure IoT Hub

Yokogawa-BrigitteZacharia — Tue, 04 Apr 2023 09:40:59 +0000

Introduction

This is part four of a five-part series that demonstrates how to use Azure Runtime Environment with e-RT3 Plus. The first two parts describe how to use Azure IoT Runtime environment to create and deploy Azure IoT Edge Python modules that read and upload data to an IoT Hub. In the first part we read data from a sample temperature module, in the second part we read data from an Analog Input Edge device.

The third part describes creating and deploying an Azure IoT Edge Python module that writes data to an Edge device.

Create an Azure IoT Edge Python module that writes data to an Edge device

In this article, we create a web application that visualizes the data received at the IoT Hub using Azure App Services.

The ultimate goal for this article series is to demonstrate how to:

Create an Azure IoT Edge Python module to gather data, process it, and upload it to IoT Hub
Create an Azure IoT Edge Python module for writing data to an Edge device
Visualize the collected data using Azure App Service and Power BI.

This article is based on the Microsoft tutorial Visualize real-time sensor data from your Azure IoT hub in a web application. For more information about installing and deploying Azure IoT Runtime environment, refer to Deploying Azure Runtime Environment on e-RT3 Plus.

Hardware modules

The following table describes the hardware modules used in this article.

Module	Description
e-RT3 Plus F3RP70-2L (CPU module, Ubuntu 18.04 32-bit)	e-RT3 Plus controls the complete module set. It accesses each module from the CPU module to read and write data. The `armhf` architecture package runs on this device.
F3AD08-6R (Analog Input module)	The Analog Input module (AD module) converts the external analog data received to digital data.
F3DA04-6R (Analog Output module)	The Analog Output (DA module) module converts the digital data sent from the e-RT3 Plus to analog data.
F3BU05-0D (Base module)	This is the base for connecting each module. It takes care of the power supply and communication between the modules that are connected to it.
F3PU20-0S (Power module)	The Power module is connected on the Base module and is used for power supply.

The module configuration and wiring is the same as described in the previous article.

For more information on the hardware modules, refer to this page.

Note: The IoT Edge module development and device operations are performed on Windows 10 using Google Chrome as the web browser.

Prerequisites

Before trying to visualize the data in the IoT Hub using Azure App Service, you meet the following requirements:

Complete the steps mentioned in the articles Create an Azure IoT Edge Python module to gather data from an Edge device and transmit it to the IoT Hub and Create an Azure IoT Edge Python module that writes data to an Edge device.
Install Git for Windows on your computer as Git is used in the deployment of the web application.

Workflow

To create a web application that visualizes the data received at the IoT Hub using Azure App services, we must complete the following steps:

Adding a consumer group to the IoT Hub
Obtaining the IoT Hub service connection string
Creating web application
Deploying a web application
Verifying data visualization in the web application

Adding a consumer group to the IoT Hub

Adding a consumer group to the IoT hub endpoint to enables the web application to read data from it.

Follow these steps to add a consumer group to the IoT Hub:

Navigate to Azure Portal.
On the left pane, under the Hub settings category, click Built-in endpoints. The Built-in endpoints page appears.
In the Consumer Groups section, create a consumer group. > Note: Ensure that you save the name of the consumer group as you will need to use it later.

Obtaining the IoT Hub service connection string

The service connection string provides the required permissions for a service to read and write data to the IoT Hub's endpoints.

Follow these steps to obtain the service connection string for your IoT Hub:

Navigate to Azure Portal.
On the left pane, under the Security settings category, click Shared access policies.
The Shared access policies page appears.
Under the Manage shared access policies section, click iothubowner.

The iothubowner pane appears, displaying the Primary and secondary connection strings.
Copy either the Primary connection string or Secondary connection string.

Note: Ensure that you save the connection string as you will need to use it later.

Creating web application

To create a web application using Azure App Service, we must complete the following steps:

Create App Service resource
Configure App Service settings
Modify web application code

Creating App Service resource

Follow these steps to create an App service resource:

Navigate to Azure Portal.
In the upper left corner of the page, click Create a resource.
Select Web App.

The Create Web App page appears.

Configure the mandatory parameters as described in the following table.

Parameter	Value
Project details - Subscription	Select an existing subscription
Project details - Resource Group	Create a new Resource Group or select an existing Resource Group
Instance details - Name	Specify any name
Instance details - Publish	Select Code
Instance details - Runtime stack*	Specify Node 16 LTS
Instance details - Operating System	Select Windows
Instance details - Region	Select region
App Service Plan - Windows Plan (<region name>)	Create a new plan or select an existing plan.
App Service Plan - SKU	Select any item as necessary
App Service Plan - Size	Select any item as necessary

* - Since the web application is created based on the sample described in Visualize real-time sensor data from your Azure IoT hub in a web application, select Node 16 LTS for runtime stack.

Click Review + create.
After verifying the configured information, click Create.
A message is displayed indicating that the App Service is deployed.

Configuring App Service settings

After creating the App Service resource, we must configure it.

Follow these steps to configure the settings of the App Service:

Navigate to Azure Portal.
Select App Services, and then select your app.
On the left pane, under the Settings category, select Configuration.

The Configuration page appears.

In the Application settings tab, click + New application setting and perform these steps:

Add the application settings described in the following table.

Name	Value	Deployment slot and settings
EventHubConsumerGroup	Specify the consumer group name created in Adding a consumer group to the IoT Hub	Empty
IotHubConnectionString	Specify the IoT Hub connection string obtained in the previous step	Empty

Verify the added settings and click Save.

The settings are applied.

Note: If a dialog box appears, click Continue.

Click the General settings tab and specify the following details.
1. From the HTTP version drop-down list, select 2.0.
2. Select On for the Web sockets configuration.
3. Click Save.
On the left pane, under the Deployment category, click Deployment Center.
In the Settings tab, from the Source drop-down list, select Local Git and click Save.

The FTPS credentials tab changes to Local Git/FTPS credentials.
Click the Local Git/FTPS credentials tab.

The Git Clone Uri is displayed in the Local Git section. Since this is required when you deploy the web application, copy and save it for future use.
In the User scope section, specify the Username, Password and Confirm Password. These credentials will be required when you deploy the web application.

Note: You can also change the username and password in this User scope section.
Click Save.

Modifying web application code

The web application is created based on the sample code described in the tutorial Visualize real-time sensor data from your Azure IoT hub in a web application.

Note: In this demonstration, we have used the code with the tag number 193943.

Follow these steps to modify the web application code:

Run the following commands to clone the sample code with tag number 193943 and create a branch.
```
git clone https://github.com/Azure-Samples/web-apps-node-iot-hub-data-visualization.git -b 193943
cd web-apps-node-iot-hub-data-visualization
git branch master
git checkout master
```
The license is available here.

Note: If your computer is in an environment that requires proxies to connect to the internet, configure the system environment variables as described here.
Open the web-apps-node-iot-hub-data-visualization folder in Visual Studio Code and edit following files:

public/index.html

Edit the title of the chart as follows:

```html
<!-- Line 14 -->
<title>e-RT3 Plus &amp; AD08 Real-time Data</title>

<!-- Line 23 -->
<span>e-RT3 Plus & AD08 Real-time Data</span>
```

public/js/chart-device-data.js

The sample code is configured such that the temperature and humidity data collected from the sample module is represented on the vertical axis of the chart.

We modify it such that data from channel 1 and 2 of the Analog Input module is represented on the vertical axis of the chart.

Replace the parameters in the file `public/js/chart-device-data.js` according to the following table.

|Existing parameter|Modified value|
|---|---|
|temperatureData|ch1Data|
|humidityData|ch2Data|
|temperature, Temperature, Temperature (ºC)|ch1|
|humidity, Humidity, Humidity(%)|ch2|

scripts/event-hub-reader.js

The sample code is configured such that the date and time at which the IoT Hub received telemetry data is represented on the horizontal axis of the chart.

We modify it such that the date and time at which the Python IoT Edge data collection module fetches the channel data is represented on the horizontal axis of the chart.

Modify the file `scripts/event-hub-reader.js` as follows:

```javascript
// Line 28
events[i].body.datetime,
```

Deploying a web application

Follow these steps to deploy the web application:

Open Visual Studio Code.
From the Menu bar, select View > Terminal to open the terminal. > Note: Ensure that the current working directory in the terminal is web-apps-node-iot-hub-data-visualization.
Run the following commands to save the code changes to the local repository.
```
    git add
    git commit
```
Run the following command to add the remote repository.
```
git remote add webapp <Git_Clone_URI>
```
Here, Git_Clone_URI is the URI obtained in step 8 of Configuring App Service settings.
Run the following command to push the web application to the remote repository.
```
git push webapp master:master
```
If the operation is completed, the following messages are displayed on the terminal. This operation usually takes a while to execute.
```
remote: Deployment successful.
To ＜Git clone URI＞
* [new branch] master -> master
```
Navigate to Azure Portal.
On the left pane, under the Deployment category, click Deployment Center.

The Deployment Center page appears.
Select the Logs tab and verify whether the web application has been pushed.
If the web application is successfully deployed, on the left pane, click Overview and then click Start to start the web application.

Verifying data visualization in the web application

After starting the web application, you can open it by using the URL that is displayed on the same page.

The data collected by the IoT Edge module is displayed in real time.

The following figure shows the chart displayed when the data is sent every 2 seconds.

You can also modify the data collection interval for the Module Twin and observe the corresponding changes in the chart. The procedure to modify the interval for a Module Twin is described here.

The following figure shows the chart displayed when the data is sent every 5 seconds.

Note: Since this URL can be accessed by anyone who has the link, you must stop the web application after confirming the operation.

Conclusion

The data sent to IoT Hub by the Azure IoT Edge data collection module is visualized by using an Azure App Service web application. In the next article, we will use Power BI to visualize this data.

Appendix

Proxy settings

If your computer is in an environment that requires proxies to connect to the internet, you must add the HTTPS_PROXY to the system environment variables in your computer.

Follow these steps to add the HTTPS_PROXY to the system environment variables in your computer:

Click the Start icon and search for System environment variables.
From the search results, select "Edit the system environment variables".

The System Properties dialog box appears.
Click Environment Variables....

The Environment Variables dialog box appears.
In the System variables section click New....

The New System Variable dialog box appears.
Specify the Variable name and Variable value as described in the following table and then click OK.

Item name Setting information

Variable name HTTPS_PROXY

Variable value http://username:password@example.com:port/

The HTTPS_PROXY is added to the system environment variables in your computer.

Item name	Setting information
Variable name	HTTPS_PROXY
Variable value	`http://username:password@example.com:port/`

References

Create an Azure IoT Edge Python module that writes data to an Edge device

Yokogawa-BrigitteZacharia — Tue, 04 Apr 2023 09:15:02 +0000

Introduction

This is part three of a five-part series that demonstrates how to use Azure Runtime Environment with e-RT3 Plus. The first two parts describe how to create and deploy Azure IoT Edge Python modules that read and upload data to an IoT Hub. In the first part, we read data from a sample temperature module, and in the second part we read data from an Analog Input Edge device.

The aim of this article is to add a Azure IoT Edge Python writing module that generates waveform data according to the configuration received from the Module Twin and writes it to the Analog Output module. The Analog Input module which is connected to the Analog Output module receives the generated data. Subsequently, the Azure IoT Edge Python data collection module deployed in the previous article collects data from all channels of the Analog Input module and sends the data to the IoT Hub.

The ultimate goal for this article series is to demonstrate how to:

Create an Azure IoT Edge Python module to gather data, process it and upload it to IoT Hub
Create a Azure IoT Edge Python module for writing data to an Edge device
Visualize the collected data using Azure App Service and Power BI.

For more information about installing and deploying Azure IoT Runtime environment, refer to Deploying Azure Runtime Environment on e-RT3 Plus.

Hardware modules

The following figure shows the hardware modules used for this demonstration.

Note: The image shows connections to channels 1 and 2 only. However, in the actual test, channels 1 to 4 are connected.

The following table describes the hardware modules used in this article.

Module	Description
e-RT3 Plus F3RP70-2L (CPU module, Ubuntu 18.04 32-bit)	e-RT3 Plus controls the complete module set. It accesses each module from the CPU module to read and write data. The `armhf` architecture package runs on this device.
F3AD08-6R (Analog Input module)	The Analog Input module (AD module) converts the external analog data received to digital data.
F3DA04-6R (Analog Output module)	The Analog Output module (DA module) converts the digital data sent from the e-RT3 Plus to analog data.
F3BU05-0D (Base module)	This is the base for connecting each module. It takes care of the power supply and communication between the modules that are connected to it.
F3PU20-0S (Power module)	The Power module is connected on the Base module and is used for power supply.

For more information on the hardware modules, refer to this page.

Note: The IoT Edge module development and device operations are performed on Windows 10.

Prerequisites

Before performing the steps described in this article, you must complete the steps mentioned in the previous article. Additionally, you must meet the hardware requirements listed here.

Follow these steps to verify the versions of the installed software:

Run the following command to verify that Azure IoT Edge version 1.1.4 is installed.
```
$ iotedge --version
iotedge 1.1.4
```
For information about installing Azure IoT Edge, refer to Deploying Azure Runtime Environment on e-RT3 Plus.
Run the following commands to verify that Docker Desktop version 20.10 is installed.
```
$ docker --version
Docker version 20.10.18+azure-1, build b40c2f6b5deeb11ac6c485c940865ee40664f0f0
```
For information about installing Docker Desktop, refer to Setting up the Container Engine.

Workflow

To write data from e-RT3 Plus to the Analog Output module, we must complete the following steps:

Create Azure IoT Edge Python writing module
Deploy module on e-RT3 Plus
Verify module operation

Module creation

The Azure IoT Edge Python writing module writes data on channels 1 – 4 of the Analog Output module every 0.1 seconds. It can generate data for four types of waveforms.

Sine wave
Triangle wave
Square wave
Sawtooth wave

To create and push the Azure IoT Edge Python writing module module to Azure portal, you must complete the following steps:

Create module project
Modify module code
Build and push module

Create module project

The steps for creating a new Python module in Visual Studio Code are the same as described in the steps 2 - 4 of the previous article.

In this demonstration, we have created a project with the following details:

Solution name : Ert3WriteDA
Module template : Python Module
Module name : Ert3WriteDAModule
Target architecture : arm32v7

Modify module code

After creating the project, rewrite the contents of main.py as follows:

import os
import json
import math
import subprocess
import signal
import datetime
import ctypes
from azure.iot.device import IoTHubModuleClient
from azure.iot.device import Message

DESIRED_KEY = 'desired'
ERT3DAOUT_KEY = 'ert3daout'
FORM_KEY = 'form'
FORM_SIN_TAG = 'sin'
FORM_TRIANGLE_TAG = 'triangle'
FORM_SQUARE_TAG = 'square'
FORM_SAWTOOTH_TAG = 'sawtooth'
CYCLE_SEC_KEY = 'cycle_sec'
AMPLITUDE_KEY = 'amplitude'
OFFSET_KEY = 'offset'
STATUS_KEY = 'status'
CYCLE_SEC_DEFAULT = 60
AMPLITUDE_DEFAULT = 2000
OFFSET_DEFAULT = 0
CHTAG_KEYS = ['ch1', 'ch2', 'ch3', 'ch4', 'ch5', 'ch6', 'ch7', 'ch8']
FAM3DA_CHNUM = {'DA04': 4, 'DA08': 8}
UNIT = 0
SLOT = 3
INTERVAL_SEC = 0.1
LDCONFIGEXEC = 'ldconfig'
M3LIB_PATH = '/usr/local/lib/libm3.so.1'


class DaOut():
    def __init__(self, unit, slot, interval_sec):
        self.__unit = unit
        self.__slot = slot
        self.__interval_sec = interval_sec
        self.__libc = ctypes.cdll.LoadLibrary(M3LIB_PATH)
        self.__libc.getM3IoName.restype = ctypes.c_char_p
        self.__chnum = self.__get_m3da_ch_num()
        self.__counters = [0 for x in range(self.__chnum)]
        self.__configs = {CHTAG_KEYS[x]: {} for x in range(self.__chnum)}
        signal.signal(signal.SIGALRM, self.__signal_handler)
        signal.setitimer(signal.ITIMER_REAL, interval_sec, interval_sec)

    def __get_m3da_ch_num(self):
        namebytes = self.__libc.getM3IoName(
            ctypes.c_int(self.__unit), ctypes.c_int(self.__slot))
        num = 0
        if namebytes is not None:
            num = FAM3DA_CHNUM.get(namebytes.decode(), 0)
        return num

    def __write_m3da_ch_data(self, ch, data):
        short_arr = ctypes.c_short * 1
        ch_data = short_arr(data)

        self.__libc.writeM3IoRegister(
            ctypes.c_int(self.__unit),
            ctypes.c_int(self.__slot),
            ctypes.c_int(ch),
            ctypes.c_int(1),
            ch_data)

    def __da_form_out(self, ch, form, interval, cycle, ampl, offset, count):
        data = 0.0
        count = count % (cycle / interval)

        if form == FORM_SIN_TAG:
            data = ampl * math.sin(math.pi * 2 * interval * count / cycle)

        elif form == FORM_TRIANGLE_TAG:
            data = count * interval * ampl * 4 / cycle
            if ampl * 3 < data:
                data = data - (ampl * 4)
            elif ampl < data:
                data = (ampl * 2) - data

        elif form == FORM_SQUARE_TAG:
            data = ampl
            if ((cycle / interval) / 2) < count:
                data = - data

        elif form == FORM_SAWTOOTH_TAG:
            data = (count * interval * ampl * 2 / cycle) - ampl

        data = data + offset

        self.__write_m3da_ch_data(ch, round(data))

    def __signal_handler(self, signum, frame):
        for id in range(self.__chnum):
            if FORM_KEY in self.__configs[CHTAG_KEYS[id]]:
                self.__da_form_out(
                    id + 1,
                    self.__configs[CHTAG_KEYS[id]][FORM_KEY],
                    INTERVAL_SEC,
                    self.__configs[CHTAG_KEYS[id]].get(CYCLE_SEC_KEY, CYCLE_SEC_DEFAULT),
                    self.__configs[CHTAG_KEYS[id]].get(AMPLITUDE_KEY, AMPLITUDE_DEFAULT),
                    self.__configs[CHTAG_KEYS[id]].get(OFFSET_KEY, OFFSET_DEFAULT),
                    self.__counters[id]
                    )
                self.__counters[id] = self.__counters[id] + 1

    def set_condition(self, desired):
        if ERT3DAOUT_KEY not in desired:
            return {STATUS_KEY: False}

        for id in range(self.__chnum):
            if CHTAG_KEYS[id] not in desired[ERT3DAOUT_KEY]:
                self.__configs[CHTAG_KEYS[id]] = {}
                self.__counters[id] = 0
            elif not self.__configs[CHTAG_KEYS[id]] == desired[ERT3DAOUT_KEY].get(CHTAG_KEYS[id], {}):
                self.__configs[CHTAG_KEYS[id]] = desired[ERT3DAOUT_KEY].get(CHTAG_KEYS[id], {})
                self.__counters[id] = 0

        reported = {STATUS_KEY: True}
        return reported


def da_out():
    module_client = IoTHubModuleClient.create_from_edge_environment()
    module_client.connect()
    twin = module_client.get_twin()

    daout = DaOut(UNIT, SLOT, INTERVAL_SEC)
    reported = daout.set_condition(twin.get(DESIRED_KEY, {}))
    module_client.patch_twin_reported_properties(reported)

    while True:
        reported = daout.set_condition(
            module_client.receive_twin_desired_properties_patch()
        )
        module_client.patch_twin_reported_properties(reported)

    module_client.disconnect()


if __name__ == "__main__":
    subprocess.run([LDCONFIGEXEC])

    da_out()

Note:

Data is written to the Analog Output module every 0.1 seconds.

The library to access to Analog Output module is not included in the Azure IoT Edge Python writing module. Instead we bind the CPU module library for use.

Build and push modules

After rewriting the code in main.py and saving it, you must push the Azure IoT Edge Python writing module to the Container Registry.

Follow these steps to build and push the Azure IoT Edge Python writing module to the Container Registry:

On the left pane of the Visual Studio Code window, in the project folder, locate the deployment.template.json file.
Right-click the deployment.template.json file and select Build and Push IoT Edge Solution.

The Azure IoT Edge Python writing module is built and pushed to the Container Registry. The progress of the command execution and the execution result is displayed on the Terminal pane at the bottom of the window.

Note: Pushing the Azure IoT Edge Python writing module to Container Registry will fail if the proxy settings are not configured correctly. For more information about configuring the Visual Studio Code and Docker Desktop proxy settings refer Proxy settings

Deploy module

Except for the Module Twin settings, the deployment procedure for this module is the same as described in the previous article. You can configure the Module Twin settings according to the format described here.

Note: After deployment, verify that runtime status of the modules $edgeAgent, $edgeHub, Ert3D2cModule and the deployed Ert3WriteDAModule is displayed as running. Additionally, the IoT Edge Runtime Response must be displayed as 200-OK.

Verify operation

The module operation can be verified by performing the following:

Verifying the channel data
Modifying the Module Twin and observing the changes

Verifying channel data

You can verify the operation of the module by following the procedure described in the previous article.

If the values of channels 1 - 4 are displayed in the following format, it means that the waveform data generated in the e-RT3 Plus device is sent to the Analog Output module.

    {
        "body": {
            "messageID": 336,
            "deviceID": "test_ert3_f3rp70",
            "datetime": "2021-04-07T02:30:53.839398Z",
            "ch1": -358,
            "ch2": 827,
            "ch3": 501,
            "ch4": 413,
            "ch5": 0,
            "ch6": 2,
            "ch7": 5,
            "ch8": 1
        },
        "enqueuedTime": "2021-04-07T02:30:53.847Z",
        "properties": {}
    }

Modifying Module Twin

By modifying the properties of the Module Twin on the Azure portal, you can control the type of waveform generated for each output channel of the Analog Output module. You can also modify the properties of the waveform such as amplitude, wave period, and offset.

Modify the properties of the Module Twin according to the format described here. For more information about how to edit the Module Twin, refer to the previous article.

After configuring the Module Twin settings, you can verify the operation of the deployed Azure IoT Edge Python writing module by following the procedure described in the previous article.

Conclusion

The waveform data is sent by the Azure IoT Edge Python writing module to the Analog Output module according to the settings configured in Module Twin. In the next article, we will visualize the data sent to the IoT Hub.

Appendix

Module Twin

The following table describes the JSON parameters in the Module Twin that you can configure for generating waveforms with the Analog Output module.

JSON parameter	Description
1ch - 4ch	Channel for which the waveform settings are configured
form	Wave form
cycle_sec	Wave period
amplitude	Wave amplitude
offset	Wave offset

Let us consider a sample configuration for generating waveforms at the Analog Output module as shown in the following table.

Channel number	Wave type	Wave period (seconds)	Wave amplitude	Wave offset
Channel 1	Sine wave	60	2000	100
Channel 2	Triangle wave	120	1000	100
Channel 3	Square wave	30	500	0
Channel 4	Sawtooth wave	40	1500	500

The JSON code for configuring the Analog Output module to generate the waveforms as described in the table is as follows:

{
    "ert3daout": {
        "ch1": {
            "form": "sin",
            "cycle_sec": 60,
            "amplitude": 2000,
            "offset": 100
        },
        "ch2": {
            "form": "triangle",
            "cycle_sec": 120,
            "amplitude": 1000,
            "offset": -100
        },
        "ch3": {
            "form": "square",
            "cycle_sec": 30,
            "amplitude": 500,
            "offset": 0
        },
        "ch4": {
            "form": "sawtooth",
            "cycle_sec": 40,
            "amplitude": 1500,
            "offset": -500
        }
    }
}

References

Create an Azure IoT Edge Python module to gather data from an Edge device and transmit it to the IoT Hub

Yokogawa-BrigitteZacharia — Tue, 04 Apr 2023 07:56:56 +0000

Introduction

This is part two of a five-part series that demonstrates how to use Azure Runtime Environment with e-RT3 Plus. In the previous article, we created a Python module that sends data to the IoT Hub from a Simulated Temperature sensor module. In this article, we create a Python IoT Edge data collection module that gathers data from the Analog Input module (F3AD08-6R) and uploads it to the IoT Hub.

The ultimate goal for this article series is to demonstrate how to:

Create a Python data collection module to gather data, process it and upload it to IoT Hub
Create a Python module for writing data to an Edge device
Visualize the collected data using Azure App Service and Power BI

Hardware modules

The following figure shows the hardware modules used for this demonstration.

The following table describes the hardware modules used in this article.

Module	Description
e-RT3 Plus F3RP70-2L (CPU module, Ubuntu 18.04 32-bit)	e-RT3 Plus controls the complete module set. It accesses each module from the CPU module to read and write data. The `armhf` architecture package runs on this device.
F3AD08-6R (Analog Input module)	The Analog Input module converts the external analog data received to digital data.
F3BU05-0D (Base module)	This is the base for connecting each module. It takes care of the power supply and communication between the modules that are connected to it.
F3PU20-0S (Power module)	Used for power supply, the Power module is connected on the Base module.

For more information on the hardware modules, refer to this page.

Note: The IoT Edge module development and device operations are performed on Windows 10.

Prerequisites

The prerequisites to complete this article are the same as the prerequisites mentioned in the previous article. Additionally, you must meet the hardware requirements listed here.

Getting started

To send data from e-RT3 Plus to the IoT Hub and view the received data, we must complete the following steps:

Create Python IoT Edge module
Deploy module on e-RT3 Plus
Verify module operation

Module creation

To create a module for collecting data from the Analog Input module and sending it to the IoT Hub, we create a new Python project using Visual Studio Code and then modify the code to suit our requirements.

The Python IoT Edge data collection module must collect data from all the channels of the Analog Input module and send the collected data to the IoT Hub. Additionally, you must be able to configure the data collection frequency from the module twin.

Create new project

The steps for creating a new Python module in Visual Studio Code are the same as described in the steps 2 and 4 of the previous article.

In this demonstration, we have created a project with the following details:

Solution name : Ert3D2c
Module template : Python Module
Module name : Ert3D2cModule
Target architecture : arm32v7

Modify code

After creating the project, rewrite the contents of main.py as follows:

import os
import json
import subprocess
import signal
import datetime
import ctypes
from azure.iot.device import IoTHubModuleClient
from azure.iot.device import Message

DESIRED_KEY = 'desired'
ERT3ADD2C_KEY = 'ert3add2c'
INTERVALSEC_KEY = 'interval_sec'
STATUS_KEY = 'status'
FAM3AD_CHNUM = {'AD04': 4, 'AD08': 8}
UNIT = 0
SLOT = 2
DEFAULT_INTERVAL_SEC = 2.0
LDCONFIGEXEC = 'ldconfig'
M3LIB_PATH = '/usr/local/lib/libm3.so.1'
DEVICE_ID = os.environ['IOTEDGE_DEVICEID']
OUTPUT_NAME = os.environ['IOTEDGE_MODULEID'] + 'ToIoTHub'


class AdD2C():
    """
    Class implemented by each function that collects data from the AD module.
    """
    def __init__(self, module_client, unit, slot):
        """
        Contructor.
        """
        self.__module_client = module_client
        self.__unit = unit
        self.__slot = slot
        self.__message_no = 1
        self.__libc = ctypes.cdll.LoadLibrary(M3LIB_PATH)
        self.__libc.getM3IoName.restype = ctypes.c_char_p
        self.__chnum = self.__get_m3ad_ch_num()
        signal.signal(signal.SIGALRM, self.__signal_handler)

    def __get_m3ad_ch_num(self):
        """
        Decide on either AD08 or  AD04.
        """
        namebytes = self.__libc.getM3IoName(
            ctypes.c_int(self.__unit), ctypes.c_int(self.__slot))
        num = 0
        if namebytes is not None:
            num = FAM3AD_CHNUM.get(namebytes.decode(), 0)
        return num

    def __read_m3ad_ch_datas(self):
        """
        Obtain data from all channels of AD module.
        """
        short_arr = ctypes.c_short * self.__chnum
        ch_datas = short_arr()
        self.__libc.readM3IoRegister(
            ctypes.c_int(self.__unit),
            ctypes.c_int(self.__slot),
            ctypes.c_int(1),
            ctypes.c_int(self.__chnum),
            ch_datas)
        return ch_datas

    def __signal_handler(self, signum, frame):
        """
        Create message and send to edgeHub module.
        """
        bodyDict = dict(
            messageID=self.__message_no,
            deviceID=DEVICE_ID,
            datetime=datetime.datetime.utcnow().isoformat() + 'Z'
            )

        ch_datas = self.__read_m3ad_ch_datas()
        for index, ch_value in enumerate(ch_datas):
            bodyDict['ch' + str(index + 1)] = ch_value

        bodyStr = json.dumps(bodyDict)
        msg = Message(bodyStr, output_name=OUTPUT_NAME)
        self.__module_client.send_message(msg)
        print(bodyStr)

        self.__message_no = self.__message_no + 1

    def set_condition(self, desired):
        """
        Configure the data collection frequency settings.
        """
        if ERT3ADD2C_KEY not in desired:
            return {STATUS_KEY: False}

        interval_sec = desired[ERT3ADD2C_KEY].get(INTERVALSEC_KEY,
                                                  DEFAULT_INTERVAL_SEC)
        reported = dict()
        reported[ERT3ADD2C_KEY] = desired[ERT3ADD2C_KEY]
        if interval_sec < 0.0:
            reported[ERT3ADD2C_KEY][STATUS_KEY] = False
        else:
            signal.setitimer(signal.ITIMER_REAL, interval_sec, interval_sec)
            reported[ERT3ADD2C_KEY][STATUS_KEY] = True

        return reported


def send_ad_data():
    """
    Call each function.
    Monitor the changes in the module twin.
    """
    module_client = IoTHubModuleClient.create_from_edge_environment()
    module_client.connect()
    twin = module_client.get_twin()

    add2c = AdD2C(module_client, UNIT, SLOT)
    reported = add2c.set_condition(twin.get(DESIRED_KEY, {}))
    module_client.patch_twin_reported_properties(reported)

    while True:
        reported = add2c.set_condition(
            module_client.receive_twin_desired_properties_patch()
        )
        module_client.patch_twin_reported_properties(reported)

    module_client.disconnect()


if __name__ == "__main__":
    subprocess.run([LDCONFIGEXEC])

    send_ad_data()

Note: The code flow is explained as comments in the code. Additional information on the code is described in Python code details.

Build and push modules

After rewriting the code in main.py and saving it, you must push the Python IoT Edge data collection module to the Container Registry.

Follow these steps to build and push the Python IoT Edge data collection module to the Container Registry:

On the left pane of the Visual Studio Code window, in the project folder, locate the deployment.template.json file.
Right-click the deployment.template.json file and select Build and Push IoT Edge Solution.

The Python IoT Edge data collection module is built and pushed to the Container Registry. The progress of the command execution and the execution result is displayed on the Terminal pane at the bottom of the window.

Note: Pushing the Python IoT Edge data collection module to Container Registry will fail if the proxy settings are not configured correctly. For more information about configuring the Visual Studio Code and Docker Desktop proxy settings refer Proxy settings.

Deploy module

After building the Python IoT Edge data collection module and pushing it to the Container Registry, we must deploy it on the e-RT3 Plus device.

Follow these steps to deploy the Python IoT Edge data collection module on e-RT3 Plus from Azure Portal:

Open Azure Portal.
Navigate to the IoT Hub that you created.
On the left pane, under the Device management section, click IoT Edge.
From the device IDs that are displayed, select the ID of the target device on which you want to deploy the module.

The device information appears.
Click Set modules.
In the Container Registry Credentials section, ensure that the credentials are configured. For more information about configuring the credentials, refer to step 7 of Creating a new project.

Note: If the PythonModule and SimulatedTemperatureSensor modules configured in the previous article are displayed, remove them by clicking the Delete icon on the right.
To configure the settings of the Python IoT Edge data collection module, click +Add and then select IoT Edge Module from the drop-down list.

Configure the parameters of the IoT Edge module as described in the following table and click Add.

Setting	Information to be entered
Module name	The module name
Image URI	Image URI obtained from the repository of Container Registry. For information about how to obtain the image URI, refer to Image URI.
Restart Policy	Always (retain default settings)
Desired Status	Running (retain default settings)
Image Pull Policy	Empty (retain default settings)

Click the Container Create Options tab and specify the options in the editor as follows.

{
    "HostConfig": 
    {   
        "Binds": [
        "/usr/local/lib/libm3.so.1.0.1:/usr/local/lib/libm3.so.1.0.1"
        ],
        "Devices": [
        {
            "PathOnHost": "/dev/m3io",
            "PathInContainer": "/dev/m3io",
            "CgroupPermissions": "rwm"
        },
        {
            "PathOnHost": "/dev/m3sysctl",
            "PathInContainer": "/dev/m3sysctl",
            "CgroupPermissions": "rwm"
        },
        {
            "PathOnHost": "/dev/m3cpu",
            "PathInContainer": "/dev/m3cpu",
            "CgroupPermissions": "rwm"
        },
        {
            "PathOnHost": "/dev/m3mcom",
            "PathInContainer": "/dev/m3mcom",
            "CgroupPermissions": "rwm"
        },
        {
            "PathOnHost": "/dev/m3dev",
            "PathInContainer": "/dev/m3dev",
            "CgroupPermissions": "rwm"
        },
        {
            "PathOnHost": "/dev/m3ras",
            "PathInContainer": "/dev/m3ras",
            "CgroupPermissions": "rwm"
        },
        {
            "PathOnHost": "/dev/m3wdt",
            "PathInContainer": "/dev/m3wdt",
            "CgroupPermissions": "rwm"
        }
        ]
    }
}

For more information about Container Create Options, refer to the official documentation.

For information about the settings that can be configured, refer to the Docker documentation.

Note: The library information is specified in the Binds tag and the device information is specified in the Devices tag.

Click the Module Twin Settings tab and specify the settings as follows:
```
{
    "ert3add2c": 
    {
        "interval_sec": 2
    }
}
```
Here, we configure an interval of 2 seconds. This configures the Python module to collect data from all the channels of the Analog Input module in a time interval of two seconds.
Click Apply.
Click the Routes tab and configure the details as described in the following table.

NAME VALUE

yourModuleToIoTHub FROM /messages/modules/<yourModule>/outputs/* INTO $upstream

Here, we configure the routes to send data from the Python IoT Edge data collection module to the IoT Hub.

Note: If the routes configured in the previous article are displayed, remove them by clicking the Delete icon on the right.
Click Review + create and verify the configuration information.

In the upper-left corner of the screen, the message "Validation passed" is displayed.
After verifying the configuration information, in the lower-left corner of the page, click Create.

The Device settings page appears, and the module list with the status of each module is displayed.
Verify the following information:
- The IoT Edge Runtime Response must be displayed as 200 - OK.
- The runtime status of all the modules ([edgeAgent], [edgeHub], and the module you deployed[yourModule]) must be displayed as running.
Note: It usually takes some time to view the status of the deployment. If any of the above conditions are not satisfied, click Refresh to view the latest information.

NAME	VALUE
yourModuleToIoTHub	FROM /messages/modules/`<yourModule>`/outputs/* INTO $upstream

Verify module operation

To verify the module operation we must observe the following outcomes:

View telemetry data in IoT Hub

Verify that the telemetry data sent by the Python IoT Edge data collection module is received in the IoT Hub.
View changes in operation corresponding to Module twin updates

To verify that updates to the module twin are reflected in the messages received at the IoT Hub, we perform the following:
1. Increase the frequency of data collection and observe the corresponding change at the IoT Hub.
2. Set the frequency of data collection to zero. This stops data transmission.

View telemetry data

You can verify the operation of the Python IoT Edge data collection module in the same way described in the previous article.

If the format of the telemetry data received is as follows, the Python IoT Edge data collection module is running properly.

    {
        "body": {
            "messageID": 2034,
            "deviceID": "test_ert3_f3rp70",
            "datetime": "2021-01-26T03:50:58.445146Z",
            "ch1": -1,
            "ch2": 8,
            "ch3": 1,
            "ch4": 2,
            "ch5": 0,
            "ch6": 1,
            "ch7": 3,
            "ch8": 1
        },
        "enqueuedTime": "2021-01-26T03:50:58.476Z",
        "properties": {}
    }

Note: As an 8-channel F3AD08 is used, data from 8 channels is transmitted.

Update module twin and verify operation

We can update the module twin from Azure portal and verify that the module operation changes accordingly.

Follow these steps to update the module twin:

Open Azure portal and navigate to the IoT Hub that you created.
On the left pane, under the Device management section, click IoT Edge.
From the device IDs that are displayed, select the ID of the device on which you want to deploy the module.

The device information appears.
Click Set modules, select the IoT Edge module that you want to modify.

The IoT Edge module screen appears.
Click the Module Twin Settings tab.

The module twin information is displayed.
Update the module twin information in the text box as necessary. Verify the updated information and if there are no issues, click Review + Create.

Modify the frequency of data collection

Update the value of interval_sec to change the frequency of data collection. For example, we have updated the data collection frequency to 5 seconds as follows:

    {
    "ert3add2c": {
        "interval_sec": 5
         }
    }

Stop data collection

If the value of interval_sec is set to zero, then data collection is stopped and messages will not be sent to IoT Hub.

     {
     "ert3add2c": {
         "interval_sec": 0
          }
    }

The module is created and deployed with the updates. After the deployment is complete, verify the operation as follows:
1. If you increased or decreased the data collection frequency, the telemetry data received at the IoT Hub changes accordingly.
2. If you stopped data collection, no messages are received at the IoT Hub.

Note: If you want to restart data collection, change the value of interval_sec to any value greater than 1.

Conclusion

The data from the Analog Input module is received by the Python module deployed in the e-RT3 Plus device. Subsequently, the same data can be viewed in Azure IoT Hub, proving that the Python module created is functioning as expected. In the next article, we will create an IoT Edge module that writes data into an analog output module that is connected to the e-RT3 Plus device.

Appendix

Image URI

To obtain the image URI, follow these steps:

Open Azure Portal and navigate to the Container Registry.
On the left pane, under the Services category, click Repositories.
From the list of repositories displayed, click the ID of the repository that you want to deploy.
From the tags that are displayed, click the tag that you want to deploy.

Note: The tags are displayed in the format <version_number>-<target_device>.
Confirm that the target device in the tag is displayed as arm32v7.

The repository details are displayed. In the Docker pull command box, the content displayed after docker pull is the image URI.

Note: The image URI is in the format <registry name>.azurecr.io/<module name>:<tag>.

Python code details

The flow of the code is explained as comments in the Python code. Here, we describe additional details:

Libraries

The following libraries are imported:

ctypes

The ctypes library provides C compatible data types, and allows calling functions in DLLs or shared libraries. Since the functions of the e-RT3 Plus device run on C language, we import the ctypes library.
```
    import ctypes
```
IoTHubModuleClient

The IoTHubModuleClient library is part of the Azure IoT Hub SDK. It is a Synchronous module client that connects to an Azure IoT Hub or Azure IoT Edge instance. For more information on the IoTHubModuleClient library, click here.
```
from azure.iot.device import IoTHubModuleClient
```
Message

The Message library is part of the Azure IoT Hub SDK. It represents a message to or from IoTHub. For more information on the Message library, click here.
```
from azure.iot.device import Message
```
The library to access the Analog Input module is not included in the Python IoT Edge data collection module, instead it is bound to the CPU module (host) library and used.

Definitions

The following definition provides the value of the symbolic link of e-RT3 Plus within the Python IoT Edge data collection module.

M3LIB_PATH = '/usr/local/lib/libm3.so.1'

e-RT3 Plus functions

The Python module uses the following e-RT3 Plus functions:

getM3IoName

This function obtains the module ID by specifying the unit and slot ID.

The function definition is as follows:
```
char* getM3IoName (int unit, int slot);
```
readM3IoRegister

This function reads the data from an I/O register by specifying the unit, slot, channel from which you want to read data, number of channels from which you want to read, and data storage position.

The function definition is as follows:
```
int readM3IoRegister(int unit, int slot, int pos, int num, unsigned short *data);
```

Other functions

subprocess.run([LDCONFIGEXEC])

This function is used for binding /usr/local/lib/libm3.so.1.0.1 of F3RP70-2L to /usr/local/lib/libm3.so.1.0.1 of Python IoT Edge data collection module.

It generates a symbolic link of the library and is invoked at the beginning of the main function to enable the usage of the library within e-RT3 Plus when deploying the Python IoT Edge data collection module.

Payload format

The payload for sending data is created in the following format:

{
"body": {
    "messageID": 1,
    "deviceID": "{deviceID}",
    "datetime": "2020-01-01T00:00:00.000Z",
    "ch1": 123,
    "ch2": 234,
    "ch3": 345,
    "ch4": 456,
    "ch5": 567,
    "ch6": 678,
    "ch7": 789,
    "ch8": 890
    }
}

Here,

messageID is the ID of the message. The message ID is generated as a sequence of numbers starting from 1,

deviceID refers to the ID of the target device on which the Python IoT Edge data collection module is deployed,

datetime refers to the time at which the CPU module acquires the data,

ch1 - ch8 refers to the data acquired from each channel.

References

Deploying a sample Python module on an Edge Device installed with Azure IoT Edge Runtime

Yokogawa-BrigitteZacharia — Tue, 25 Oct 2022 05:48:20 +0000

Introduction

This is part one of a five-part series that demonstrates how to use Azure Runtime Environment with e-RT3 Plus. In this article, we explore how to deploy a sample Python module on e-RT3 Plus and Raspberry Pi 4 Model B and update the values from Azure Portal.

We deploy two IoT Modules; a Simulated Temperature Sensor module that generates sample temperature values, and a Python module that uploads the temperature values to the IoT Hub whenever the temperature value crosses a threshold. We can view the uploaded temperature values on Visual Studio Code.

This demonstration is based on two Microsoft tutorials:

However, it is customized to show the procedure for the ARM 32-bit device.

Environment

Supported devices（OS)

e-RT3 Plus F3RP70-2L（Ubuntu 18.04 32-bit）
Raspberry Pi 4 Model B （Ubuntu Server 18.04 32-bit）

The armhf architecture package runs on these devices.
The modules are developed on a computer installed with Windows 10.

Workflow

The following figure shows the workflow for deploying a sample python module on e-RT3 Plus using Azure IoT runtime.

Prerequisites

Azure Runtime environment must be installed on the e-RT3 Plus or the Edge device that you are using. For more information about deploying Azure Runtime Environment on e-RT3 Plus, refer to Deploying Azure Runtime Environment on e-RT3 Plus.

The following software must be installed and configured before you start creating the sample Python module.

Container Engine (Docker Desktop)
Python
Visual Studio Code

Setting up the Container Engine

You must set up a container engine on the computer in which you want to develop the Python module. In this demonstration, we install the Docker Desktop container engine.

Note: If your computer is in an environment that requires proxies to connect to the internet, you must configure the Proxy settings.

Follow these steps to set up Docker Desktop on your computer:

Download Docker Desktop. For Windows Home and other operating systems, select the corresponding OS from the menu on the left and continue the installation. Ensure that you enable Hyper-V backend during installation.

For more information on how to install Docker Desktop, refer to Install Docker Desktop on Windows.

Note: Ensure that your computer meets the system requirements specified on Install Docker Desktop on Windows before installing Docker Desktop.
After the installation is complete, launch Docker Desktop from the Start menu.
A notification appears, indicating that Docker Desktop is running and the container engine setup is complete. Ensure that Docker Desktop is successfully launched before proceeding.
Note:
1. As we are using a Linux container, ensure that the notification "Linux Containers Hyper-V backend is launching..." is displayed.
2. Docker Desktop may fail to launch if the computer load is high. In this case, retry launching.

Installing Python

Follow these steps to install Python on your computer:

Download the Python installer.

Note: To maintain compatibility with the Python module template, download Python 3.7.
Open the installer and click Install Now.
Python is installed on your computer.

Note: In the installer dialog box, select the Add Python 3.7 to PATH check box.

Setting up Visual Studio Code

Visual Studio Code is used for developing the IoT Edge module.

Follow these steps to install and configure Visual Studio Code:

Download the installer for Visual Studio Code from the official website.

Note: Ensure that you select a version that is compatible with your computer.
Start the installer and follow the steps in the installation wizard to complete the installation.
If the installation is completed successfully, a message is displayed indicating the same.
To install Azure IoT Tools and Python extension features, on the left pane, click Extensions and perform the following operations:
1. In the search bar, type Azure IoT Tools and click Install on the corresponding search result.
  
  Azure IoT Tools extension is downloaded and installed on Visual Studio Code.
2. In the search bar, type Python and click Install on the corresponding search result.
  
  Python extension is downloaded and installed on Visual Studio Code.
To sign in to Azure, in the Command Palette, type Azure: Sign In and select the corresponding search result.
Follow the instructions displayed to sign in.

If you have successfully signed in, your account information is displayed at the bottom of the Visual Studio Code window.
To select the IoT Hub, in the Command Palette, type Azure IoT Hub: Select IoT Hub and select the corresponding search result.
Follow the instructions displayed and select Azure subscription and IoT Hub.
To view the IoT Hub and connected devices, on the left pane, select the file explorer and expand AZURE IOT HUB.

The IoT Hub is displayed and the connected devices are listed under the Devices section.

Creating and deploying a module

To create, deploy and test an IoT Edge module, you must complete the following steps:

Create container registry
Create project
Add registry credentials
Select target architecture
Update module with custom code
Build and push module
Deploy Simulated Temperature Sensor module
Deploy Python module
Verify the module operation

Creating container registry

To store the docker images, any registry that is compatible with docker can be used. In this demonstration, we use the Azure Container Registry.

Note: Azure Container Registry is a paid service. For the pricing details, click here.

Follow these steps to create a container registry:

Sign-in to Azure Portal.
In the upper-left corner of the page, click + Create a resource.
From the left panel, select Containers.
From the list of displayed services, select Container Registry.
In the Information tab, specify the following settings information:
1. Subscription
2. Resource group
3. Registry name
4. Location
5. SKU >Note: SKU must be set to Basic.
Click the Review + create tab, and verify the setting information.
If the settings are configured correctly, click Create.

Creating a new project

Follow these steps to create a new Edge IoT solution in Visual Studio Code:

Open Visual Studio Code.
In the Command Palette, type Azure IoT Edge: New IoT Edge solution and select the same from the search results.

The folder selection box appears.
Specify the location where you want to save the project.
In the Provide a Solution Name box that appears in the Command Palette, specify a solution name.
From the Select Module Template drop-down list that appears in the Command Palette, specify Python Module.
In the Provide a Module Name box that appears in the Command Palette, specify a name for the module.

The module name specified is reflected within the template and is considered as the module name in IoT Edge. Here, we have specified the name PythonModule.

Note: The module name of IoT Edge can also be modified individually before building the images.
In the Provide Docker Image Repository box that appears in the Command Palette, specify the address of the Container Registry login server.

By default localhost:5000 appears.

To obtain the login server information, open Azure Portal and navigate to the Container registry that you created in previous step. On the left pane, under the Settings section, select Access keys. The login server information is displayed on the right.

Note: Ensure to save this information as it is required at a later stage.
Copy the Login server information <registry_name>.azurecr.io and paste it in the Command Palette in the format <registry_name>.azurecr.io/<module_name>.

The project template is created with the folder structure as shown below.

Adding registry credentials

Follow these steps to add the registry credentials:

To add the registry credentials, on the left pane, select the file explorer and open the .env file under the PythonModule folder.
```
.env
CONTAINER_REGISTRY_USERNAME_<registry name>=
CONTAINER_REGISTRY_PASSWORD_<registry name>=
```
In the CONTAINER_REGISTRY_USERNAME_<registry name> and CONTAINER_REGISTRY_PASSWORD_<registry name> fields, specify the username and password, i.e.: the Access keys of the Container Registry.
```
.env

CONTAINER_REGISTRY_USERNAME_<registry name>=<username>
CONTAINER_REGISTRY_PASSWORD_<registry name>=<password or password2>
```

Note: The password provided can be either password or password2.

Selecting target architecture

Follow these steps to select the target architecture:

Click the current architecture at the bottom of the Visual Studio Code window.

The Select Azure IoT Edge Solution Default Platform box appears in the Command Palette.
Since we are using the armhf architecture, select arm32v7.

The current architecture changes to arm32v7.

Note: If the current architecture is not displayed at the bottom of the window, type Azure IoT Edge: Set Default Target Platform for Edge Solution and select the same.

Updating module with custom code

Rewrite the contents of main.py and deployment.template.json by following the steps in Update Modules with Custom Code, and save the file.

The Python module routes temperature values to the IoT Hub only if the value exceeds the defined threshold.

Building and pushing the module

Follow these steps to build the created module and push the image to Container Registry:

From the menu bar, select View > Terminal to display the terminal pane.
Run the following command in the terminal to log in to the Container Registry.
```
docker login -u <username> -p <password or password2> <login server>
```
Note: If the required proxy settings are not configured then you will not be able to log into the Container Registry.
For details about proxy settings, refer to Proxy settings.
On the left pane, select File Explorer, right-click deployment.template.json, and select Build and Push IoT Edge Solution.

The module is built and if there are no errors the image is pushed to the Container Registry.
To verify that the images are pushed to the Container Registry, perform these steps:
1. Open Azure Portal.
2. Navigate to the Container Registry.
3. On the left pane, click Repositories.
The repositories page appears, displaying a list of images.
1. Verify that your image is present in this list.

Deploying Simulated Temperature Sensor module

The Simulated Temperature Sensor module generates sample temperature readings periodically which increase in value over time. This module is readily available for deployment in Azure Marketplace.

We will deploy this module and route the generated temperature values as input to the Python module.

Follow these steps to deploy the Simulated Temperature Sensor:

Open Azure Portal.
Navigate to the IoT Hub that you created.
From the left pane, under the Device management section, click IoT Edge.
On the right pane, click the device ID of the target device on which you want to deploy the module.
Click Set modules tab.

The Set modules on device page appears.
In the IoT Edge Modules section, click the Add drop-down menu, and select Marketplace Module.

The IoT Edge Module Marketplace page appears.
In the search box, type simulated temperature sensor and select the same from the search results.

The module is added to the IoT Edge Modules section with the Desired Status as running.
Click the Review + create tab.

The deployment settings in the standard JSON format is displayed.
In the upper-left corner of the page, ensure that the message Validation passed is displayed and then click Create.

The device details page appears, displaying the deployment status of the Simulated Temperature Sensor on the Modules tab.

Note: If the Edge device is in an environment that requires proxies to connect to the internet, the telemetry sent to IoT Hub may be blocked and not delivered even when the module is correctly deployed and working.
In such cases, you must configure the Proxy settings for $edgeHub module. For more information about the proxy settings refer to Configure proxy support.

Deploying Python module

After pushing the images to the Container Registry, we must deploy it on the Edge device.

Follow these steps to deploy the images from Azure Portal:

Open Azure Portal.
Navigate to the IoT Hub that you created.
From the left pane, under the Device management section, click IoT Edge.
On the right pane, click the device ID of the target device on which you want to deploy the module.
Click Set modules tab.
In the Modules tab, specify the details mentioned in the following table.

Setting items of module Information to be entered

NAME Registry name

ADDRESS Login server

USERNAME User name

PASSWORD Password or password2

To obtain the Registry name, Login Server details, username and password, refer to step 7 in Creating a new project.
To configure the settings of the IoT Edge module, click +Add and then select IoT Edge Module from the drop-down list.

Setting items of module	Information to be entered
NAME	Registry name
ADDRESS	Login server
USERNAME	User name
PASSWORD	Password or password2

Configure the parameters of the IoT Edge module as described in the following table and click Add.

Setting item of module	Information to be entered
IoT Edge Module Name	The module name
Image URI	Image URI obtained from the repository of Container Registry. For information about how to obtain the image URI, refer to Image URI.
Restart Policy	Always (retain default settings)
Desired Status	Running (retain default settings)

Click the Routes tab and configure the details as described in the following table.

NAME	VALUE
PythonModuleToIoTHub	FROM /messages/modules/PythonModule/outputs/* INTO $upstream
sensorToPythonModule	FROM /messages/modules/SimulatedTemperatureSensor/outputs/temperatureOutput INTO BrokeredEndpoint(‘/modules/PythonModule/inputs/input1’)

Click Review + create and verify the configuration information.

In the upper-left corner of the screen, the message "Validation passed" is displayed.
After verifying the configuration information, in the lower-left corner of the page, click Create.

The Device settings page appears, and the module list is displayed. It usually takes some time to complete the deployment.
Click Refresh.

The Module list is updated and you can see the module that you deployed to the Edge device added to the Modules list.

Verifying module operation

After viewing the telemetry messages received from the Edge device in the IoT Hub, we can confirm that the deployed module is working.

In this section, we describe how you can verify module operation by using Visual Studio Code.

Follow these steps to verify the reception of telemetry messages at the IoT Hub:

Open the project created in Creating a new project in Visual Studio Code.
On the left pane, right-click your Azure IoT Device, and select Start Monitoring Built-in Event Endpoint

The messages received at the IoT Hub are displayed in the OUTPUT tab.
To stop receiving messages, in the lower-left corner of the window, click Stop Monitoring built-in event endpoint.

Note: The maximum number of messages that can be sent in the case of the Simulated Temperature Sensor is 500. If you want to send more messages, you must restart the module by running the following command in the terminal:
      sudo iotedge restart SimulatedTemperatureSensor
You can restart the other module by replacing SimulatedTemperatureSensor with the required module ID.
      sudo iotedge restart <Module ID>

Editing Module Twin

By editing the module twin, the features of the sample module used can be edited without having to build or deploy the module again.
In this example, let us edit the module twin of the deployed Python module to change the threshold value of the temperature of the machine that notifies IoT Hub.

Follow these steps to edit the module twin from Azure Portal:

Open Azure Portal and navigate to the IoT Hub that you created.
From the Device management category, display the IoT Edge device.
Click the device ID of the target device.
Click Set modules.
From the list of IoT Edge modules, select the Python module that you deployed.

The Update IoT Edge Module page appears.
Click the Module Twin Settings tab.
In the editor, enter the TemperatureThreshold details in JSON format.
In the lower-left corner of the screen, click Apply.

The following example shows how to set the threshold value of the machine temperature to 40 degrees.
Click Review + Create.

The verification page appears, displaying the configured deployment in standard JSON format. You can view the updated temperature threshold here.

After confirming the deployment configuration and checking for the "Validation passed" message in the upper-left corner of the screen, click Create.

When you view the data generated by following the same procedure as Verifying module operation, you can see that the new threshold value has come into effect.

For example, if you changed the temperature threshold from 30 to 40, you will see that the Python module stops sending messages until the generated temperature values start exceeding 40℃. Subsequently, on the OUTPUT tab of Visual Studio Code, the messages are displayed only when the temperature values exceed 40℃.

Conclusion

As you have seen, the messages sent by the Edge device are received at the IoT Hub. Additionally, updates to the Twin Module in Azure Portal are reflected in the Edge device.

In our future articles, we will explore how to collect data using a Python module in Azure IoT Edge and send it to an IoT Hub.

Appendix

Image URI

To obtain the image URI, follow these steps:

Navigate to the images stored in the container registry as shown in step 4 of Build and Push modules.
Select the ID of the module you want to deploy.

The repository details are displayed.
From the Docker pull command box, the content displayed after docker pull is the image URI.

The image URI is in the format <registry name>.azurecr.io/<module name>:<tag>.

For example in the following image, the URI is <registry name>.azurecr.io/pythonmodule:0.0.1-arm32v7.

Proxy settings

If you are using a proxy server in your environment then you must configure the proxy settings for Visual Studio Code and Docker Desktop.

The proxy settings described in this section are an example of what was used for this demonstration. Depending on your environment, you must configure the proxy settings as necessary.

Visual Studio Code

Proxy settings for Visual Studio Code are required when logging in from Visual Studio Code to Azure.

Follow these steps to configure the proxy settings in Visual Studio Code.

Open Visual Studio Code.
From the menu bar, select File > User settings > Settings.

The Settings page is displayed.
To display the proxy settings, in the search box, type proxy.

The Http:Proxy search result is displayed.
In the Http:Proxy box, specify the proxy URL based on your environment. For example, http://username:password@example.com:port/

The proxy information is saved in the settings file in the following location.
C:\Users\username\AppData\Roaming\Code\User\settings.json

For environment details, contact your network administrator.

Docker

Proxy settings for Docker are required when logging in to Container Registry from the Visual Studio Code Terminal and pushing images.

For more information about configuring proxy settings in Docker, refer to the Docker official document.

Follow these steps to configure the proxy settings:

Use any editor to open the config.json file from the following folder location. C:\Users\username\.docker.

Note: If the file does not exist, create it.
Specify the proxy information in config.json by following the procedure mentioned in Configure the Docker Client.

If config.json is already created, there is a possibility that settings other than Proxy settings are specified in the file. In such cases, specify the configuration such that the label name is the same as the already existing labels.

The following shows the httpProxy and httpsProxy configurations used in this demonstration.

   {
        "HttpHeaders":
        {
            "User-Agent":"Docker-Client/19.03.13 (windows)"
        },
        "auths":
        {
            "<registry name 1>.azurecr.io":{},
            "https://index.docker.io/v1/":{},
            "<registry name 2>.azurecr.io":{}
        },
        "credStore":"desktop",
        "credsStore":"desktop",
        "proxies":
        {
            "default":     
            {
                "httpProxy":"http://username:password@example.com:port/",
                "httpsProxy":"http://username:password@example.com:port/"
            }
        },
        "stackOrchestrator":"swarm"
    }

As you can see, some of the configurations already existed. Here, we have added the proxies configuration.

To configure the proxy settings in Docker Desktop, perform these steps:
1. On the toolbar, click the arrow to open the tool tray
2. Right-click the Docker Desktop icon and select Settings.
3. Expand Resources and select PROXIES.
4. In the Web Server (HTTP) box, specify the proxy URL.
5. In the Secure Web Server (HTTPS) box, specify the proxy URL.
6. Click Apply & Restart.

The proxy settings are configured.

References

Installing AWS IoT Greengrass V2 in Ubuntu

Yokogawa-BrigitteZacharia — Thu, 25 Aug 2022 12:45:03 +0000

Introduction

This article describes the steps to install the latest version of Greengrass, called Greengrass V2, which was released in December 2020. Greengrass V2 supports new features such as open-source edge runtime, improved modularity, command-line interface, and improved fleet development features. This enables you to develop, deploy, and manage IoT applications locally on your Edge device.

For details about installing Greengrass V1, refer to this article.

Environment

Supported devices (OS)

e-RT3 Plus F3RP70-2L（Ubuntu 18.04 32-bit) Edge controller of Yokogawa.
Raspberry Pi 4 Model B （Ubuntu Server 20.04 32-bit）

The armhf architecture package runs on these devices.

Getting started

To install AWS IoT Greengrass V2 you must complete the fllowing steps:

Install jdk packages on the Edge device
Configure AWS authentication settings
Register IoT Core device on AWS and install core software in Edge device

Installing jdk packages on the Edge device

Run the following commands to install the required packages on the Edge device.

sudo apt update
sudo apt install unzip default-jdk

Note: If the Edge device is in proxy environment, configuring proxy settings is mandatory.

Configuring AWS authentication settings

Configure AWS authentication settings in the environment variables of Edge device. For details, refer to the AWS authentication settings.

Registering IoT Core device on AWS and installing core software in Edge device

Follow these steps to register an IoT Core device on AWS:

On the AWS IoT page, select Greengrass devices, and click Set up one core device.
In the Core device name box, specify the name of the core device.
Under Thing group, select Enter a new group name, and specify the name of the thing group in the Thing group name box.

Note: If you want to add your core device to an existing thing group, select Select an existing group, and specify the name of the thing group in the Thing group name box.
Download and install the core software in Edge device. Run the commands 1 and 2 in order in the Edge device to download and install the core software.

Note: If the Edge device is in proxy environment, refer to Installing core software in proxy environment.
After the installation is complete, click View core devices.
If the core device you created is displayed, registration and installation is successful. (It may take some time for the device to get displayed.)

Appendix

Installing core software in proxy environment

If you want to operate your Edge device in a proxy environment, configuring proxy settings is mandatory. The settings may differ based on the operating environment. You can follow the following steps as a reference to configure proxy settings that are specific to your environment.

Create a core software settings file called config.yaml and configure the proxy settings as follows:

config.yaml
services:
    aws.greengrass.Nucleus:
        configuration:
            mqtt:
                port: 443
            greengrassDataPlanePort: 443
            networkProxy:
                proxy:
                    url: "http://username:password@xxx.com:port/"

In step 4 of the Registering IoT Core device on AWS procedure, in command 2, delete --deploy-dev-tools true and add --init-config (path/to/config.yaml).

Modified command example:
```
sudo -E java \
-Droot="/greengrass/v2" -Dlog.store=FILE -jar ./GreengrassCore/lib/Greengrass.jar --aws-region ap-northeast-1 \
--thing-name SampleCore --thing-group-name SampleCoreGroup --component-default-user ggc_user:ggc_group \
--provision true --setup-system-service true --init-config (path/to/config.yaml)
```
Note: In a proxy environment, the option --deploy-dev-tools true is deleted as it will lead to deployment failure. If you want to deploy the CLI component manually, refer to Deploying Greengrass CLI.

Deploying Greengrass CLI

Greengrass CLI is the command-line interface (CLI) that enables you to develop and debug an application locally in the Edge device. If you have not included the --deploy-dev-tools true option when installing the AWS Core software, the Greengrass CLI (aws.greengrass.Cli) component is not deployed.

To deploy the Greengrass CLI component manually, perform these steps:

On the left pane, select Greengrass devices > Components.
On the Public components tab, select aws.greengrass.cli.

The aws.greengrass.cli page appears.
Click Deploy.
Select Create new deployment and click Next.
On the Specify target page, perform these steps:
1. In the Deployment information section, specify a name to identify the deployment.
2. In the Deployment target section, select Core device as the Target type, and then specify the target name in the Target name box.
3. Click Next.
From the Public components list, select the aws.greengrass.Cli check box and click Next.
Verify the component and advanced deployment configurations, and click Deploy.

The Deployment page appears, displaying the deployment status as "Completed"
Run the following command in the Edge device.
```
username@ubuntu:~$ /greengrass/v2/bin/greengrass-cli --version    
```
If the following output is generated, Greengrass CLI is deployed successfully.
```
Greengrass CLI Version: 2.1.0     
```

References

Transmitting messages between AWS IoT Core and Edge device

Yokogawa-BrigitteZacharia — Thu, 18 Aug 2022 06:29:42 +0000

Introduction

In this article, we describe how to register an Edge device in AWS IoT Core and transmit messages between an Edge device and AWS IoT Core. You can easily send messages without having to install runtime in the Edge device.

For details about installing AWS IoT Greengrass(V1) on an Edge device and connecting it to AWS, refer to this article.

Environment

Supported devices (OS)

e-RT3 Plus F3RP70-2L（Ubuntu 18.04 32-bit）
Raspberry Pi 4 Model B （Ubuntu Server 20.04 32-bit）

The armhf architecture package runs on these devices.

Getting started

You must configure the following settings to transmit messages between AWS and the Edge device:

Configure AWS settings
Configure Edge device settings

Configuring AWS settings

Configuring AWS settings involves the following steps:

Creating Edge device policy
Registering Edge device

Creating Edge device policy

To allow sending and receiving messages between the Edge device and AWS, we must create an Edge device policy.

Navigate to the Services page in AWS Console, and select Internet of Things on the left pane.
On the right pane, under Internet of Things, select IoT Core.

The AWS IoT page appears.
On the left pane, select Security > Policies, and then click Create Policy.

The Create Policy page appears.
In the Policy name box, type the name of the policy you want to create for the IoT device.

On the Policy statements tab, click JSON, and then enter the following JSON code to define the policy.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "iot:Publish",
                "iot:Receive"
            ],
            "Resource": [
                "arn:aws:iot:<region>:<account>:topic/test/topic"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "iot:Subscribe"
            ],
            "Resource": [
                "arn:aws:iot:<region>:<account>:topicfilter/test/topic"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "iot:Connect"
            ],
            "Resource": [
                "arn:aws:iot:<region>:<account>:client/test-*"
            ]
        }
    ]
}

In the code, replace <account> and <region> with your account and region names respectively.

NOTE: The region name is displayed in the upper-right corner of the AWS page.

To obtain the account name, refer to this procedure.

Click Create.

The policy for the IoT device is created.

Registering Edge device

On the left pane of the AWS IoT page, select All devices > Things, and then click Create things.
Create a Thing with the following settings.

Setting item Description

No. of things to be created Create one thing

Name of the thing Any (In this example, “testdevice”)

Device license Automatically creates new license
Select the policy you created in the previous section, and click Create things.
Download the following files:
- Device certificate
- Public key file
- Private key file
- Root CA certificate for Amazon trust services endpoint
Note: These files are required to enable message transmission between the Edge device and AWS IoT core. Therefore, you must save these files locally on your computer for further usage.
Click Done.

The Edge device is registered as an IoT device on AWS.

Setting item	Description
No. of things to be created	Create one thing
Name of the thing	Any (In this example, “testdevice”)
Device license	Automatically creates new license

Configuring Edge device settings

Configuring the Edge device settings involves the following steps:

Install required packages
Set up virtual environment
Install SDK
Prepare sample program

Installing required packages

Run the following commands to install the required packages:

sudo apt update
sudo apt install cmake python3-dev python3-venv

In case of Raspberry Pi, SDK installation is not possible with the default compiler. Therefore, you must run the following commands to install the gcc package.
```
sudo apt install build-essential
sudo apt install gcc-8
```
Note: If the Edge device is under proxy environment, configuring proxy settings is mandatory.

Setting up virtual environment

Run the following commands to create a virtual environment.

python3 -m venv venv
source venv/bin/activate

Installing SDK

Run the following commands to install the SDK.

For e-RT3
Install awsiotsdk.

(venv) username@ubuntu:~$ python3 -m pip install wheel
(venv) username@ubuntu:~$ python3 -m pip install awsiotsdk

For Raspberry Pi
Specify the compiler and install awsiotsdk.

(venv) username@ubuntu:~$ python3 -m pip install wheel
(venv) username@ubuntu:~$ CC=/path/to/gcc-8 python3 -m pip install awsiotsdk

Note: gcc-8 path can be verified by runnning the following command.
username@ubuntu:~$ which gcc-8
/usr/bin/gcc-8

Note: If encoding-related error occurs during installation, specify the locale and run the following command.
Example:
(venv) username@ubuntu:~$ LC_CTYPE=C.UTF-8 python3 -m pip install awsiotsdk

Preparing sample program

Save the AWS SDK2 sample program, pubsub.py, in the Edge device. The license is saved here.

Note: This article uses v1.7.1 of the code.

Place the certificates and the key files that you downloaded while registering the Edge device in the same directory where the sample program is placed.

Message transmission

To enable transmission of messages, run the following command to start the sample program in the Edge device:

(venv) username@ubuntu:~$ python3 pubsub.py --endpoint xxx.amazonaws.com --root-ca AmazonRootCA1.pem --cert xxx-certificate.pem.crt --key xxx-private.pem.key

In the arguments, specify the Root CA certificate, license, and key files that you downloaded while registering the Edge device. In addition, specify the endpoint in the command.

To obtain the endpoint, on the AWS IoT page, select Settings.

The endpoint is displayed in the Device data endpoint section.

Note: If the Edge device is under proxy environment configuring proxy settings is mandatory.

The following output indicates that message is sent successfully.

Connecting to xxx.amazonaws.com with client ID 'test-xxx'...
Connected!
Subscribing to topic 'test/topic'...
Subscribed with QoS.AT_LEAST_ONCE
Sending 10 message(s)
Publishing message to topic 'test/topic': Hello World! [1]
Received message from topic 'test/topic': b'"Hello World! [1]"'
Publishing message to topic 'test/topic': Hello World! [2]
Received message from topic 'test/topic': b'"Hello World! [2]"'
Publishing message to topic 'test/topic': Hello World! [3]

In addition to publishing the message in ‘test/topic’ topic, this program subscribes to the ‘test/topic’ topic and stops after sending and receiving messages ten times.

Verifying D2C message transmission

Follow these steps to verify if the message from the Edge device has reached the Cloud:

On the AWS IoT page, on the left pane, select MQTT test client.
On the Subscribe to topic tab, in the Topic filter box, type the name of the topic for which the message was sent from the Edge device.
Click Subscribe.

You are subscribed to the specified topic.

If the message is received, it is displayed at the bottom of the screen.

Verifying C2D message transmission

Follow these steps to send and verify C2D message transmission:

In the Edge device, run the following command to start the pubsub.py program without publishing the message.

(venv) username@ubuntu:~$ python3 pubsub.py --endpoint xxx.amazonaws.com --root-ca AmazonRootCA1.pem --cert xxx-certificate.pem.crt --key xxx-private.pem.key --message ""

To send the message from cloud to the Edge device, click the Publish to a topic tab.
In the Topic name box, type test/topic.
In the Message payload box, specify the message to be sent.
For example,
```
{
    "message":"Hello from AWS IoT console"
}
```
Click Publish.

If the following output is generated in the Edge device, the C2D message is transmitted successfully. The program stops after sending ten messages.

```bash
Received message from topic 'test/topic': b'{\n "message": "This is a message from AWS"\n}'
```

Appendix

Configuring proxy settings

If the Edge device is in proxy environment, run the pubsub.py program by using the following command.

(venv) username@ubuntu:~$ python3 pubsub.py --endpoint xxx.amazonaws.com --root-ca AmazonRootCA1.pem --cert xxx-certificate.pem.crt --key xxx-private.pem.key --proxy-host "proxy.example.com" --proxy-port xx

If user name and password are required, specify the same in the program as follows:

Before modification

pubsub.py
if (args.proxy_host):
proxy_options = http.HttpProxyOptions(host_name=args.proxy_host, port=args.proxy_port)

After modification

pubsub.py
if (args.proxy_host):
proxy_options = http.HttpProxyOptions(host_name=args.proxy_host, port=args.proxy_port, auth_type = http.HttpProxyAuthenticationType.Basic, auth_username="<username>", auth_password="<password>")

References

Cross compiling C/C++ programs for Ubuntu installed on Edge devices

Yokogawa-BrigitteZacharia — Thu, 04 Aug 2022 08:33:00 +0000

Introduction

Developing a large-scale program on an Edge device can take a considerable amount of time. To avoid this, in this document, we introduce a method for cross compiling C/C++ for a device on which Ubuntu is installed.

This enables you to develop a program using the Windows Subsystem for Linux (WSL) and deploy the program on Edge devices.

Environment

Supported device (OS)

e-RT3 Plus F3RP70-2L (Ubuntu 18.04 32-bit): An Edge controller from Yokogawa.

The armhf architecture package runs on this device.

Windows version

Windows 10 64-bit version 1909

Software version

Visual Studio Professional 2019 version 16.9.6

Getting started

Before you start building a cross-compile platform, you must install the libc6 package on e-RT3 Plus.

Run the following commands to install the package:

sudo apt update
sudo apt install libc6-dbg

Note: In some cases, you may need to configure the sudo and proxy settings.

WSL preparation

Installing WSL

Install WSL by following the steps described in the Microsoft official procedure.
Download Ubuntu 18.04 LTS from Microsoft Store.

Configuring WSL settings

Start WSL and login.

Run the following commands to install the required packages.

sudo apt update

# Install the build environment for armhf
sudo apt install crossbuild-essential-armhf

# Install other required packages
sudo apt install make rsync zip

You can verify that the installation is successful by running the following commands.

$ arm-linux-gnueabihf-gcc --version
arm-linux-gnueabihf-gcc (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ arm-linux-gnueabihf-g++ --version
arm-linux-gnueabihf-g++ (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Note: If the computer is in an environment that requires proxies to connect to the internet, you must configure the proxy settings.

Installing Visual Studio

The steps to install Visual Studio are described in the previous article.

Note: You need not download IntelliSense to execute the steps that are described in this article.

Creating and debugging projects

Creating and debugging a project involves the following steps:

Creating a project
Configuring project settings
Building and debugging the project

Creating a project

Follow these steps to create a project:

Start Visual Studio and create a new project.
Specify the project type as C++ and Linux, select Console Application, and create a project by specifying the project name.

Configuring project settings

From the menu bar, click Project > Properties.

For more information about configuring the project settings for a Linux WSL subsystem, refer to the official documentation.
From the Configuration drop-down list, select Debug.
From the Platform drop-down list, select x64.
On the left pane, select General.
On the right pane, under the Platform Toolset section, from the Platform Toolset drop-down list, select GCC for Windows Subsystem for Linux.
Click Apply
On the left pane, click Debugging.
From the Remote Debug Machine drop-down list, select the added device.
On the left pane, select C/C++.
On the right pane, under the General section, in the C Compiler and C++ Compiler boxes, type arm-linux-gnueabihf-g++.
On the left pane, select Linker.
On the right pane, under the General section, in the Linker box, type arm-linux-gnueabihf-g++.
Click Apply.
The project settings are configured.

Building and debugging the project

Follow these steps to build and debug the project:

From the Debug drop-down list, select x64.
Place a break point anywhere in the source code.
Build and run the program.
If the program stops at the break point, it indicates that the project is created successfully.

The output is displayed in the Linux console window.

Appendix

Accessing WSL files

WSL files are saved in the current Linux directory. To navigate to the directory, type \\wsl$ in the address bar of Windows Explorer.

Build project without debugging

You can build only the project without debugging it.

From the menu bar, select Build > {project-name}.

The output file is saved in the following location:

<Solution directory>\<Project directory>\bin\<Platform>\<Configuration>\

Using device-specific libraries

When you build a program using device-specific libraries, you must include the library and header files in the project.

Create the inc and lib folders in the project directory to store the header and library files respectively.
Open the project properties in Visual Studio.
On the left pane, select C/C++ > General.
On the right pane, in the Additional Include Directories box, specify the inc folder location.
On the left pane, select Linker > General.
In the Additional Library Directories box, specify the lib folder location.
In the Library Dependent File, specify the name of the library. For example, in case of libfoo.so, type foo.
Click Apply.

References

Using Visual Studio to set up a C/C++ remote development environment

Yokogawa-BrigitteZacharia — Wed, 03 Aug 2022 05:35:53 +0000

Introduction

In the previous articles, we have demonstrated a number of programs, mostly written in Python, that run on an Edge device. However, there are times when we need to write programs in C/C++. It is easy to set up a C/C++ remote development environment on an Edge device such as e-RT3 Plus or Raspberry Pi. In this article, we demonstrate how to set up the C/C++ environment on an Ubuntu installed Edge device using Visual Studio.

Environment

Supported devices (OS)

e-RT3 Plus F3RP70-2L (Ubuntu 18.04 32-bit) Edge controller of Yokogawa.
Raspberry Pi 4 Model B (Ubuntu Server 20.04 32-bit）

The armhf architecture package runs on these devices.

Software requirements

Visual Studio Professional 2022 version 17.2.6

Getting started

The first step towards setting up a Visual Studio environment on the device is to install the gdbserver package. To install the gdbserver package on the device, run the following commands.

sudo apt update
sudo apt install g++ make gdb gdbserver zip

Note: By default, this package is installed on e-RT3 Plus.

Installing Visual Studio

Follow these steps to install Visual Studio:

Install the tool set required to set up the Linux environment.

Open Visual Studio Installer and install Linux and embedded development with C++.

For more information about Linux development with C++ in Visual Studio 2022, refer to the official documentation.
Configure the settings for connecting Visual Studio and the device by performing these steps:
1. Start Visual Studio.
2. Open Tools > Options > Cross Platform > Connection Manager.
3. Click Add.
4. Enter the following details and click Connect.
  
  Items Value
  
  Host name IP address of device
  
  Port 22
  
  User name User name for device
  
  Authentication type Password
  
  Password User password
To download the header to be used in IntelliSense, click Connection Manager > Remote Header IntelliSense Manager, select the device that you added, and then click Download.

Items	Value
Host name	IP address of device
Port	22
User name	User name for device
Authentication type	Password
Password	User password

Creating and debugging a project

Creating and debugging a project involves the following steps:

Creating a project
Configuring project settings
Building and debugging a project

Creating a project

Follow these steps to create a project:

Start Visual Studio and create a new project.
Specify the project type as C++ and Linux, select Console Application, and create a project by specifying the project name.

Configuring project settings

Follow these steps to configure project settings:

From the menu bar, select Project > {project-name} Properties.
From the Configuration drop-down list, select Debug.
From the Platform drop-down list, select ARM.
On the left pane, select General.
On the right pane, from the Remote Build Machine drop-down list, select the added device.
Click Apply.
On the left pane, select Debugging.
From the Remote Debug Machine drop-down list, select the added device.
From the Debugging Mode drop-down list, select gdbserver, and then click Apply.

Building and debugging a project

Follow these steps to build and debug a project:

After selecting Debug for Configuration and ARM for Platform, place a break point anywhere in the source code to build and run the program.

If the program stops at the break point, it indicates that the project is created successfully.

The output is displayed in the Linux console window.

Appendix

Adding source files

Follow these steps to add source files:

Right-click the project icon, and select Add > New Item.

The Add New Item dialog box appears.
Specify the file name, select the file type and then click Add.

Library links

Follow these steps to add library links:

Open the Project properties.
On the left pane, select Configuration Properties > Linker > Input > Library dependencies, and then click Edit.
Enter the name of the library to be added.

The following image shows the scenario when libm is added.

Adding Include path

If you are using a header file that is available in a location other than the default location, you must configure the Include path settings.

From the Project Properties, navigate to C/C++ > General > Additional Include directories, and then click Edit.
Specify the directory path on device that includes the header file and click OK.

References

Communicating over Modbus TCP with Node-RED and storing data in InfluxDB

Yokogawa-BrigitteZacharia — Wed, 20 Jul 2022 05:19:53 +0000

Introduction

In the previous article, we learnt how to establish Modbus communication using Python and store the data in InfluxDB.
This is the last article of the 3-part series, where we explore how to establish Modbus communication using Node-RED and store the data in InfluxDB. As Node-RED contains a flow-based editor, it enables visual programming to connect and configure various nodes to establish Modbus communication easily.

System positioning

The following figure shows the setup required to establish Modbus connection with e-RT3 Plus.

The flow involves 2 events:

Data collection

Edge device collects data from the external device over Modbus TCP using Node-RED.
Data storage

Edge device stores the collected data in InfluxDB.

Supported devices

This example can be performed on the following devices:

e-RT3 Plus F3RP70-2L1 (Ubuntu 18.04 32-bit)
Raspberry Pi 4 Model B (Ubuntu Server 20.04 32-bit)

The armhf architecture package runs on these devices.

Getting started

To get started, you must follow these steps to retrieve and store data in the Edge device over Modbus TCP communication.

Installing Node.js and Node-RED
Installing custom nodes on Node-RED
Configuring InfluxDB settings
Configuring Modbus server
Creating data collection flow
Creating data storage flow

Installing Node.js and Node-RED

Note: If the Edge device is in proxy environment, configuring proxy settings for Linux and npm is mandatory.

Follow these steps to install Node.js and Node-RED on the Edge device:

To install Node.js run the following command.

curl -fsSL https://deb.nodesource.com/setup_14.x | sudo -E bash - 
sudo apt-get install -y nodejs

You can verify installation of Node.js by using the following command.
```
node -v
```
If Node. js is installed successfully then the version of the Node will be displayed as shown below.
```
v14.17.1
```

Run the following command and install Node-RED.

sudo npm install -g --unsafe-perm node-red

If the command is executed successfully, the following output is displayed.

+ node-red@2.2.0
added 286 packages from 375 contributors in 142.031s

Installing custom nodes on Node-RED

To communicate via Modbus TCP protocol, the following custom nodes must be installed on Node-RED:

node-red-contrib-influxdb
node-red-contrib-modbus

Follow these steps to install the custom nodes on Node-RED:

Run the following command in the Edge device and start Node-RED.
```
node-red
```
After starting Node-RED, in the address bar of your Internet browser, specify the following URL, <IP_ADDRESS_OF_EDGE_DEVICE>:1880 and open Node-RED.
In the upper-right corner of the screen, click the Menu icon and select Manage palette.

The User Settings dialog box appears.
On the Install tab, in the search box, type node-red-contrib-influxdb.

The search results appear.
Click Install to install the corresponding node.

Note: Perform steps 4-5 to install the node-red-contrib-modbus node.

Configuring InfluxDB settings

You can follow the steps described in this article to install InfluxDB. (Installation of Python package is not required.) And then, follow the steps described in this section to create a database with its name as noderedDB.

Configuring Modbus server

In this article, we will start the Modbus server on the computer to simulate a device such as a sensor that sends data to the Edge device. To do so, start the program described in this link.

Creating data collection flow

Let us create a flow to collect data from the Modbus server every 5 seconds. In this example, we collect two time-series data values stored in the holding register of the server.

Follow these steps to create the data collection flow:

From the left menu, drag-and-drop Modbus-Read node and double-click it to configure its properties.
Configure the properties of the node as follows.

Setting name Setting value

Name read_holding_register

FC FC 3: Read Holding Registers

Address 2

Quantity 2

Poll Rate 5 seconds
Next to the Server box, click the Edit icon.
Configure the properties of the Modbus server as follows and then click Add.

Setting name Setting value

Name serverPC

Type TCP

Host IP address of the computer on which Modbus server is running

Port 5020
After configuring the settings, click Done.
To check if the data is collected successfully, perform these steps:
1. Drag-and-drop the debug node.
2. Connect the debug node to the upper connection of the read_holding_register, and then click Deploy.
3. Click the debug tab on the right pane. If the collected data is displayed as shown in the image below, it is successful.

Setting name	Setting value
Name	read_holding_register
FC	FC 3: Read Holding Registers
Address	2
Quantity	2
Poll Rate	5 seconds

Setting name	Setting value
Name	serverPC
Type	TCP
Host	IP address of the computer on which Modbus server is running
Port	5020

Creating data storage flow

Now that we are able to collect data through the Modbus server, we proceed to configure a data storage flow to store the collected data in InfluxDB.

Follow these steps to create the data storage flow:

From the left menu, drag-and-drop the function node and double-click it to configure its properties.

Select the On Message tab and write the following program to process the array of collected data into the standard JSON format.

const data = msg.payload
payload = {
"holding_register_2": data[0],
"holding_register_3": data[1]
}
msg["payload"] = payload
return msg;

Click Done.
Drag-and-drop the influxdb out node to store data in InfluxDB and double-click it to configure its settings.
Configure the properties of the node as follows.

Setting name Setting value

Name To_noderedDB

Measurement nodered_measurement
Next to the Server box, click the Edit icon.
Configure the properties of the InfluxDB server as follows and then click Add.

Setting name Setting value

Name noderedDB

Host localhost

Port 8086

Database noderedDB
After configuring the settings, and click Done.
Connect the nodes as shown below and click Deploy.
After waiting for a while, run the following commands in the Edge device and verify the contents of the database:
1. Launch InfluxDB.
```
influx 
```
  The following output is displayed if InfluxDB is launched successfully.
```
Connected to http://localhost:8086 version 1.8.10
InfluxDB shell version: 1.8.10
```
2. Run the following command to select the noderedDB database.
```
> USE noderedDB
```
  If the database selection is successful, the following output is displayed.
```
Using database noderedDB
```
3. Run the following command to view the stored data.
```
> SELECT * FROM "nodered_measurement" LIMIT 3
```
  Storage is successful if data is stored as shown below.
```
name: nodered_measurement
time holding_register_2 holding_register_3
---- ------------------ ------------------
1643782593448526154 93 24
1643782598683263519 99 22
1643782603803351763 100 20
```

Setting name	Setting value
Name	To_noderedDB
Measurement	nodered_measurement

Setting name	Setting value
Name	noderedDB
Host	localhost
Port	8086
Database	noderedDB

Conclusion

As demonstrated in the article, it is possible to create visual data applications with minimal programs. In conjunction with Node-RED and InfluxDB, e-RT3 Plus is able to collect and store data from external data sources by using Modbus TCP communication.
You can learn more about visualizing the collected data on Grafana from our previous article.

Additional information

npm proxy settings

If the device is in proxy environment, run the following command to configure the proxy settings of npm.

sudo npm -g config set proxy http://{username}:{password}@{proxy-server-url}:{port}
sudo npm -g config set https-proxy https://{username}:{password}@{proxy-server-url}:{port}

References

Using Python to store collected data in InfluxDB and visualizing data in Grafana

Yokogawa-BrigitteZacharia — Thu, 07 Jul 2022 09:04:06 +0000

Introduction

In the previous article, we learnt how to collect data with Modbus TCP communication using Python. This is the 2nd article of the 3-part series, where we explore how to store collected data in InfluxDB and visualize the data in Grafana.

Supported devices

The steps in this demonstration can be performed on the following devices.

e-RT3 Plus F3RP70-2L1 (Ubuntu 18.04 32-bit)
Raspberry Pi 4 Model B (Ubuntu Server 20.04 32-bit)

The armhf architecture packages run on these devices.

System positioning

The following figure shows the setup required to collect, store, and visualize data on the Edge device over the Modbus TCP communication.

The flow involves four events:

Data collection

Data is collected from an external device by using Modbus TCP communication.
Data storage

The collected data is stored in Influx DB on the Edge device.
Visualization

The data stored in InfluxDB is visualized using Grafana.
Monitoring

The results are monitored in a computer by accessing Grafana in the Edge device.

Prerequisites

To achieve the objective of this article, you must install Grafana and InfluxDB on the Edge device.

Installing Grafana

Follow these steps to install Grafana on the Edge device.

Install the required packages.

sudo apt install -y apt-transport-https
sudo apt install -y software-properties-common wget
wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -

Add repositories and install Grafana.

echo "deb https://packages.grafana.com/oss/deb stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list
sudo apt update
sudo apt install grafana

For more information on installing Grafana, refer to Install on Debian/Ubuntu | Grafana Labs.

Start the service and verify the status.

sudo systemctl daemon-reload
sudo systemctl start grafana-server
sudo systemctl status grafana-server

If the program is executed successfully, the following output is displayed.

* grafana-server.service - Grafana instance
Loaded: loaded (/usr/lib/systemd/system/grafana-server.service; disabled; ven
Active: active (running) since Fri 2021-12-24 07:38:20 UTC; 14s ago
Docs: http://docs.grafana.org
Main PID: 2756 (grafana-server)
Tasks: 12 (limit: 2366)
CGroup: /system.slice/grafana-server.service
`-2756 /usr/sbin/grafana-server --config=/etc/grafana/grafana.ini --p
Dec 24 07:38:32 ubuntu grafana-server[2756]: t=2021-12-24T07:38:32+0000 lvl=info
Dec 24 07:38:32 ubuntu grafana-server[2756]: t=2021-12-24T07:38:32+0000 lvl=info

Note: If the Edge device is in a proxy environment, configuring proxy settings is mandatory.

Installing InfluxDB

Follow these steps to install InfluxDB on an Edge device.

Install the package that is used to operate InfluxDB from Python.
```
python3 -m pip install influxdb
```

Add repositories and install InfluxDB.

curl -s https://repos.influxdata.com/influxdb.key | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/influxdb.gpg
export DISTRIB_ID=$(lsb_release -si); export DISTRIB_CODENAME=$(lsb_release -sc)
echo "deb [signed-by=/etc/apt/trusted.gpg.d/influxdb.gpg] https://repos.influxdata.com/${DISTRIB_ID,,} ${DISTRIB_CODENAME} stable" | sudo tee /etc/apt/sources.list.d/influxdb.list
sudo apt update
sudo apt install influxdb

Start the service and verify the status.

sudo systemctl unmask influxdb.service
sudo systemctl start influxdb
sudo systemctl status influxdb

If the program is executed successfully, the following output is displayed.

* influxdb.service - InfluxDB is an open-source, distributed, time series databa
Loaded: loaded (/lib/systemd/system/influxdb.service; enabled; vendor preset:
Active: active (running) since Fri 2021-12-24 07:51:02 UTC; 6s ago
Docs: https://docs.influxdata.com/influxdb/Process: 4191 ExecStart=/usr/lib/influxdb/scripts/influxd-systemd-start.sh (co
Main PID: 4192 (influxd)
Tasks: 9 (limit: 2366)
CGroup: /system.slice/influxdb.service `-4192 /usr/bin/influxd -config /etc/influxdb/influxdb.conf`
Dec 24 07:51:01 ubuntu influxd-systemd-start.sh[4191]: ts=2021-12-24T07:51:01.75
Dec 24 07:51:01 ubuntu influxd-systemd-start.sh[4191]: ts=2021-12-24T07:51:01.75

For more information about installing InfluxDB, refer to the official documentation.

Data Collection

Collect data from the device with a Python program that uses PyModbus. In this example, start the Modbus server on the computer instead of the device.

Note: For more details about data collection using PyModubs, refer to the previous article.

Configuring Modbus server

Follow these steps to configure Modbus server:

Start the Modbus server on the computer.

To rewrite the values periodically, modify the program updating_server.py (You can access the license here).

In this example, we store two time-series data values in the holding register and Boolean data in the coil.

Modify the following code in the updating_write function.

import random
def updating_writer(a):
context = a[0]
slave_id = 0x00

# Write to coil 1
fx = 1
address = 0x01
values = [bool(random.getrandbits(1))]
log.debug("coil1: " + str(bool(random.getrandbits(1))))
context[slave_id].setValues(fx, address, values)

# Writing to holding register 2
fx = 3
address = 0x02
values = context[slave_id].getValues(fx, address, count=1)
values[0] += random.randint(-10, 10)
if values[0] > 100:
values[0] = 100
elif values[0] < 0:
values[0] = 0
log.debug("holding2: " + str(values[0]))
context[slave_id].setValues(fx, address, values)

# Writing to holding register 3
fx = 3
address = 0x03
values = context[slave_id].getValues(fx, address, count=1)
values[0] += random.randint(-5, 5)
if values[0] > 50:
values[0] = 50
elif values[0] < 0:
values[0] = 0
log.debug("holding3: " + str(values[0]))
context[slave_id].setValues(fx, address, values)

Rewrite the following code at the end of the updating_server.py program.

# Before change
StartTcpServer(context, identity=identity, address=("localhost", 5020))
# After change
StartTcpServer(context, identity=identity, address=("", 5020))

Install the following package and start the program.

python -m pip install twisted pymodbus
python updating_server.py

Data collection in Edge device

Follow these steps to collect data in the Edge device:

Run the following client.py program in the Edge device to collect the server data every 5 seconds.

Ensure that you replace the with the IP address of the computer on which the Modbus server is running.

client.py

#!/usr/bin/env python
from pymodbus.client.sync import ModbusTcpClient as ModbusClient
import time

# Connect to Modbus server
mbclient = ModbusClient('<IP_ADDRESS_OF_SERVER_PC>', port=5020)
mbclient.connect()
while True:
try:
    # Read value of coil 1
    rr = mbclient.read_coils(1, 1)
    co1 = rr.bits[0]

    # Read values from holding registers 2 and 3
    rr = mbclient.read_holding_registers(2, 2)
    hr2 = rr.registers[0]
    hr3 = rr.registers[1]

    print("coil1: {0}, holding2: {1}, holding3: {2}".format(co1, hr2, hr3))

    #Sleep for 5 seconds
    time.sleep(5)

except KeyboardInterrupt:
    # disconnect
    mbclient.close()
    break

except Exception as e:
    print(e)

If the program is executed successfully, the following output is displayed.

username@ubuntu:~$ python3 client.py
coil1: False, holding2: 89, holding3: 12
coil1: True, holding2: 85, holding3: 13
coil1: True, holding2: 78, holding3: 14
...
# Close with Ctrl+C

Note: Although this program enables easier collection of periodic data, the period in which the data is collected is not accurate. If you want to collect data in more accurate periods, check other methods of data collection.

Data storage

Store the data that is collected from Edge device in InfluxDB.

Configuring the database

Follow these steps to configure the database:

Start InfluxDB
```
sudo systemctl start influxdb
```

Create a database with the name as sampleDB.

username@ubuntu:~$ influx
Connected to http://localhost:8086 version 1.8.10
InfluxDB shell version: 1.8.10
> CREATE DATABASE sampleDB
> exit

Storage in database

Follow these steps to store the collected data in the database that you created.

Refer to the InfluxDB sample program and add the code to store data to the client.py program that was created in the step Data collection in Edge device.

After adding, the code is updated as follows.

client.py

#!/usr/bin/env python
from pymodbus.client.sync import ModbusTcpClient as ModbusClient
from influxdb import InfluxDBClient
from datetime import datetime
import time

# Connect to Modbus server
mbclient = ModbusClient('<IP_ADDRESS_OF_SERVER_PC>', port=5020)
mbclient.connect()

# Connect to database
dbclient = InfluxDBClient(host='localhost', port=8086, database='sampleDB')
while True:
try:
    # Read value from coil 1
    rr = mbclient.read_coils(1, 1)
    co1 = rr.bits[0]

    # Read values from holding registers 2 and 3
    rr = mbclient.read_holding_registers(2, 2)
    hr2 = rr.registers[0]
    hr3 = rr.registers[1]

    print("coil1: {0}, holding2: {1}, holding3: {2}".format(co1, hr2, hr3))

    # Write to database
    json_body = [
        {
            "measurement": "sample_measurement",
            "time": datetime.utcnow(),
            "fields": {
                "coil_1": co1,
                "holding_register_2": hr2,
                "holding_register_3": hr3
                }
        }
    ]

    print("Write points: {0}".format(json_body))
    dbclient.write_points(json_body)

    # Skip for 5 seconds
    time.sleep(5)
    except KeyboardInterrupt:

    #Disconnect
    mbclient.close()
    dbclient.close()
    break

    except Exception as e:
    print(e)

Note: Data is written to the SD card everytime data is stored in the database. If you want to reduce the frequency with which data is written to the SD card, use an appropriate storage medium such as RAM disk or network disk, and specify them as storage destinations in the InfluxDB settings.

Start the program and wait for a few seconds.
```
python3 client.py
# Close with Ctrl+C
```

Run the following commands in the Edge device and display the contents of the table.

Launch InfluxDB.

~/modbus$ influx

The following output is displayed if InfluxDB is launched successfully.

Connected to http://localhost:8086 version 1.8.10
InfluxDB shell version: 1.8.10

Run the following command to select the sampleDB database.
```
> USE sampleDB
```
If the database selection is successful, the following output is displayed.
```
Using database sampleDB
```

Run the following command to view the stored data.

> SELECT * FROM "sample_measurement" LIMIT 3

Storage is successful if data is stored as shown below.

name: sample_measurement
time coil_1 holding_register_2 
olding_register_3
---- ------ ------------------ ------------------
1641876882000000000 false 23 13
1641876887000000000 false 16 14
1641876892000000000 true 15 13

Visualization and monitoring

Use Grafana to visualize the data that is stored in InfluxDB.

Run the following command in Edge device and start Grafana.
```
sudo systemctl start grafana-server
```
In the address bar of your Internet browser, specify the following URL:

<IP address of the device>:3000
In the Username and Password boxes, type admin, and click Log in.

Both the username and password has been configured to admin. You can change them as necessary.

Database registration

Follow these steps to register the sampleDB database that was created in the step Configuring the database, in Grafana.

On the left pane, click the gear icon, navigate to Configuration > Data Sources, and then click Add data source.
Select InfluxDB.

The settings page appears.
Enter the following details and click Save & test.

Setting name Setting value

URL http://localhost:8086/

Database sampleDB

Setting name	Setting value
URL	http://localhost:8086/
Database	sampleDB

Visualizing time-series data

In this example, we show you how to create a dashboard and display the collected time-series data in graphs such that:

Values of holding register 2 are displayed in line graphs as raw data
Values of holding register 3 are displayed as the average of values every minute

Follow these steps to create the dashboard:

On the left pane, click the + icon, select Create > Dashboard, and then click Add a new panel.
Configure the graph settings for the holding register 2 as shown in the screen below, and click Apply.

Note: You can click the Edit icon and enter following query to configure query settings.
SELECT "holding_register_2" FROM "sample_measurement" WHERE $timeFilter
Repeat steps 1-2 to create a bar graph for holding register 3 (holding_register_3) and configure the settings as shown in the image below.

Note: You can also configure the query settings with the following query command.
SELECT mean("holding_register_3") FROM "sample_measurement" WHERE $timeFilter GROUP BY time(1m)

Visualizing Boolean data

Follow these steps to display the latest status of coil 1.

Click Add panel to add a panel.
Select Stat from the right-side menu and configure the settings as shown in the image below.

Note: You can also configure query settings with the following query command.
SELECT "coil_1" FROM "sample_measurement" WHERE $timeFilter

When all the operations are completed, the following screen is displayed.
Click Save dashboard to save the dashboard.

Conclusion

This article described how to collect data through Modbus communication using Python, store the data in InfluxDB, and visualize data in Grafana on the Edge device.

This article is useful when you want to visualize data locally, without having to save the data in external systems.

DEV Community: Yokogawa Technologies Solutions India

Data Visualization on the e-RT3 using Node-RED, InfluxDB Cloud, and Grafana

Table of Contents

Setup

Introduction

Overview

Materials

Software

Manuals

Wiring

Network Addresses

Default Addresses

Final Addresses

Procedure

1) SD Card Setup

2) Setting up the Cloud

3) Configuring the Network

Changing the e-RT3 IP address

Changing the Gateway IP address

Changing the router IP address

4) Wiring Connections

5) e-RT3 Dependencies

Installing NVM, Node.js, and npm

Installing Node-RED

Installing Grafana

6) Boot

Node-RED Boot

Grafana Boot

7) Grafana Dashboard Setup

8) Gateway Node-RED Setup

9) e-RT3 Node-RED Setup

10) Using the Application

Using Power BI to visualize Edge data sent to Azure IoT Hub

Introduction

Hardware modules

Prerequisites

Workflow

Installing Power BI Desktop

Creating IoT Hub consumer group

Creating Stream Analytics

Configuring Stream Analytics settings

Configure and visualize data in Power BI

Conclusion

References

Using Azure App Service to visualize Edge data sent to Azure IoT Hub

Introduction

Hardware modules

Prerequisites

Workflow

Adding a consumer group to the IoT Hub

Obtaining the IoT Hub service connection string

Creating web application

Creating App Service resource

Configuring App Service settings

Modifying web application code

Deploying a web application

Verifying data visualization in the web application

Conclusion

Appendix

Proxy settings

References

Create an Azure IoT Edge Python module that writes data to an Edge device

Introduction

Hardware modules

Prerequisites

Workflow

Module creation

Create module project

Modify module code

Build and push modules

Deploy module

Verify operation

Verifying channel data

Modifying Module Twin

Conclusion

Appendix

Module Twin

References

Create an Azure IoT Edge Python module to gather data from an Edge device and transmit it to the IoT Hub

Introduction