DEV Community: Luca Cipriani

Using Arduino's New Oplà Kit to Read Remote Sensors and Display the Data in Your Home

Luca Cipriani — Wed, 19 May 2021 12:26:10 +0000

(part one is here and part two is here)

A tutorial using the Oplà Kit to have 2 different IoT devices communicate with each other. This is the third part of the articles about IoT, Heroku and Arduino.

Recently, I received the new Arduino Oplà Kit for testing. To try it out, I decided to create an IoT project with multiple environmental sensors that remotely collect data, then report that data over the cloud and back to my home IoT device. It sounds complicated, but the setup was easy, and it allows for many advanced customizations in the future.

In this article, I'll walk through how simple it is to create a similar setup—the architecture, devices, and code. Where you take it from there is up to you!

The Project

To get started, I created a straightforward project to capture the outside temperature, display it on a device in my home, and compare it to the inside temperature. I also wanted to make sure I created something customizable and powerful, where I could add my own logic to allow for more complicated projects in the future.

Note: For a quick background on IoT terms (Thing, Variable, Sketch, and so on), check out this overview.

Our Architecture

The architecture is simple but very powerful. We will leverage the Arduino IoT Cloud REST and Realtime API to allow two different devices to communicate with each other. In the middle, we'll host a custom Node.js application on Heroku. We'll also use the code my friend Francesco Stasi wrote to help us move the data. The code is entirely open source, with an MIT license, and is available at my GitHub repo. Francesco and I will keep working on the code, if there is interest.

The architecture looks like this:

Our first Thing (Thing 1) is connected via Wi-Fi. It reads data (such as temperature) from the environment and sends the data to the Arduino IoT Cloud. By creating our own Node.js application hosted on Heroku, we are able to get the data from Arduino IoT in near real-time by using the MQTT/WebSockets API.

Once we have the data streaming to our application, we can manipulate the incoming information as needed. For simplicity, we'll simply republish the same values from Thing 1 on Thing 2. (Note: You can only do this if Thing 1 and Thing 2 have identical property names.)

This is all simpler than it sounds. Let’s see how to build it!

What we need:

1x Oplà Kit (The kit contains 1x MKR1010 and one year of free service for Arduino IoT.)
Arduino IoT Cloud account (included in the Oplà Kit)
Heroku account (the free hobby account should be fine)
1x Arduino MKR1010 (a basic Arduino board with Wi-Fi, included in the Oplà Kit)
1x MKR ENV Shield (environmental sensors)

Creating Thing 1: the External Sensor

First, open the Arduino IoT cloud dashboard and create an Outdoor sensor Thing with just one variable named "temperature."

As you can see in the image below, the type of the variable is “Temperature sensor (°c).” In the sketch, it is represented as a CloudTemperatureSensor. The property does not need to be set; it is just a sensor, not an actuator, so we do not need any kind of user interaction. To avoid sending too much data to the cloud, I set a threshold of 0.2 C.

Then, I created a new device with my MKR Wi-Fi 1010 and attached it to this Thing.

The whole setup looks like this:

Remember to add the network parameters before uploading your sketch. If you don't, you will get an error message.

Next, I added the MKR ENV Shield on top of the MKR Wi-Fi 1010 and created a simple sketch to send the temperature online. Here is the code of the whole Sensors Sketch.

Besides the print to the serial, there are really only two other important lines: ArduinoCloud.update() to send the data to the Arduino IoT Cloud, and the ENV.readTemperature() method, which allows us to measure the temperature and put it in a variable. Then, that variable will be sent to the cloud.

void loop() {
  ArduinoCloud.update();
  // Your code here 
  temperature = ENV.readTemperature();
  delay(200);
}

After uploading your sketch to the board, you should see the following in the serial monitor:

We are registering the temperature every 200ms but only sending it to the cloud if it changes at least 0.2 Celsius. Everything else is recorded and visible locally, though not sent to our cloud application. Feel free to improve the sketch and add your own remote properties.

The In-house Device to Visualize Data

The whole project came to my mind because I just received the new Arduino Oplà Kit for testing. The kit includes many sensors (more or less the same as the MKR ENV Shield), five LEDs, and five capacitive buttons (touch sensors). More importantly, it has a useful battery-powered screen to display data. To power the Oplà Kit, I bought some Samsung 18650 batteries from RS Components.

Because we have such a powerful kit, I decided to create three different visualizations on the screen.

Button 1 displays the indoor temperature (internalTemperature property) of the Oplà Kit.
Button 2 displays the outdoor temperature from the MKR Wi-Fi 1010 with the ENV Shield (what I called the Outdoor Board device and the Outdoor Sensors Thing).
Button 3 shows the difference between the internal and external temperatures.
LED 1: the LED turns blue if the difference is negative (outside is colder than inside), and red otherwise.

Setting Up the Kit in Arduino IoT Cloud

To set up the kit, first be sure to unplug your board; otherwise, it will not work. Then, configure your MKR1010. I named it CarrierBoard as seen in the image here:

After setting up the board, I created a new Thing named OplaKit, then I added two different properties: internalTemperature and temperature. The other board fetches the temperature property, while the internalTemperature comes from the Oplà Kit itself.

The internal Temperature is:

Type: Temperature
User interaction: Disabled (read only)
Send Values: On Change (threshold: 0.1 C)

Note that the temperature property is exactly the same as the property in the Outdoor Sensors Thing.

Creating the Sketch and Adding Interaction

The sketch running in the Oplà Kit seems like magic, but everything is extremely simple. We need to detect when a button is pressed, and change the values accordingly on the screen.

First, we need to add the proper libraries to the sketch to control the LEDs and touch buttons.

// Arduino_MKRIoTCarrier - Version: Latest 
#include &lt;Arduino_MKRIoTCarrier.h>
#include &lt;Arduino_MKRIoTCarrier_Qtouch.h>

Then, we need to include the cloud library (automatically done) and tell the system if we are using the case or not. (It will calibrate the sensitivity of the touch buttons.) Then, I set a variable changeDisplayCounter to switch between multiple screens.

#include "thingProperties.h"
MKRIoTCarrier carrier;
bool CARRIER_CASE = true;
int changeDisplayCounter = 0;

The loop is simple. We check each button to see if it's been pressed before displaying data on the screen.

void loop() {
 ArduinoCloud.update();
 // Your code here
 carrier.Buttons.update();
 if (carrier.Button1.onTouchDown()) {
   Serial.println("Touching Button 1");
   changeDisplayCounter = 0;
   printMeasure(internalTemperature, "Int. Temp");
 }
 // Checks if new data are available
 if (carrier.Button2.onTouchDown()) {
   Serial.println("Touching Button 2");
   changeDisplayCounter = 1;
   printMeasure(temperature, "Ext. Temp");
 }
 if (carrier.Button3.onTouchDown()) {
   Serial.println("Touching Button 3");
   changeDisplayCounter = 2;
   printMeasure(temperature - internalTemperature, "Temp. diff");
 }
 internalTemperature = carrier.Env.readTemperature();
 calcDiff(temperature, internalTemperature);
 delay(200);
}

Finally, at the end, we always read the internalTemperature from the kit, and calculate the difference with the external temperature.

The rest of the sketch contains details on the functions:

void onTemperatureChange() {
 // Do something
 calcDiff(temperature, internalTemperature);
}
void calcDiff(float temp1, float temp2){
 if(temp1 >= temp2) {
   //( led index , red , green , blue )
   carrier.leds.setPixelColor(0, 10, 180, 10);
   // return true;
 }
 else {
   carrier.leds.setPixelColor(0, 10, 10, 180);
   // return false;
 }
 carrier.leds.show();
}
void printMeasure(float temp, String name){
 int colors[] = {ST77XX_RED, ST77XX_BLUE, ST77XX_YELLOW};
 int index = 0;
 Serial.println(name);
 Serial.print(temp);
 Serial.println(" C");
 // Reset screen
 carrier.display.fillScreen(ST77XX_BLACK);
 index = changeDisplayCounter % 3;
 carrier.display.setTextColor(colors[index]);
 carrier.display.setCursor(30, 100);
 carrier.display.setTextSize(2);
 carrier.display.println(name);
 carrier.display.setTextSize(4);
 carrier.display.setCursor(50, 120);
 carrier.display.print(temp);
 carrier.display.setTextSize(2);
 carrier.display.print(" C");
}

Here you can find a full copy of my Oplà Kit Sketch.

As of now, the second sketch will not receive any kind of value for the variable temperature because that data lives in the MKR Wi-Fi 1010. Though it is sent to Arduino IoT Cloud, it is not yet sent back to our Oplà Kit. We can solve this by creating our own Node.js application to relay the data.

Creating a Custom Node.js Application on Heroku

To create our custom application, we can use two different kinds of Arduino IoT APIs: one is the REST API, and the other is a real-time MQTT/WebSocket API. The latter is not documented yet, but luckily a JavaScript client is available. You can find the two JavaScript modules here: one is the Arduino IoT Js Client, and the other is the Arduino IoT Js lib.

To interact with the APIs, we will need an API token. You can find yours by clicking the Integrations button at the top of the screen:

Be sure to keep your API key handy. We will use it in a moment.

My friend, Francesco Stasi, is an amazing developer. (You can find him on GitHub.) He helped create the application that proxies a property of a first Thing to another property with the same name in a second Thing.

The whole software is really small, and you can find it in my own Js-IoT-Proxy GitHub repository. The software is designed to be developed locally and deployed to Heroku. The main part of it is visible in this snippet of code available on the repo.

// Register a callback for every property in the object to keep the values
updatedObject.keys(thingProperties).forEach((propertyName) => {
   console.log(`${thingId} - register callback for ${propertyName}`);
   ArduinoIoTCloud.onPropertyValue(thingId, propertyName, (newValue) => {
     console.log(`${thingId} - property ${propertyName} changed`);
     thingProperties[propertyName].value = newValue;
     // Invoke the proxy callback
     proxyThingProperty(thingId, propertyName, newValue);
   });
 });
 return {
   // Get the array of known properties names
   getPropertiesList() {
     return Object.keys(thingProperties);
   },
   // Get the value of a property
   getPropertyValue(propertyName) {
     return typeof thingProperties[propertyName] !== "undefined"
       ? thingProperties[propertyName].value
       : null;
   },
   // Set the value for a property
   setPropertyValue(propertyName, newValue) {
     console.log(`${thingId} - writing ${propertyName}, value: ${newValue}`);
     return ArduinoIoTCloud.sendProperty(thingId, propertyName, newValue);
   },
 };

As you can see, if the two properties have the same name, the devices communicate with each other. The logic is visible in the proxy.mjs file.

Deploying the Application

I have been a Linux Engineer for quite a long time. However, when I need something robust that works out of the box without any maintenance, I prefer to use Heroku.

To deploy the application, I used a very handy feature I recently discovered: the GitHub automatic deploy feature. Every time I merge to main, the app immediately shows up in my Heroku panel.

We are only missing a few things to make all this work. We need to create and store different environment variables containing the following:

The Arduino API app ID (IOT_CLIENT_ID)
The Arduino API Secret Key (IOT_CLIENT_SECRET)
The Thing ID of the OutDoor Sensors (THING_ID_FROM)
The Thing ID of the OplaKit (THING_ID_TO)

We can do all of this by using the Heroku settings panel in our dashboard.

Finally, we can deploy the app and check the logs. Now, every time the temperature changes in the ENV Shield, you can see it being sent to the Oplà Kit in real-time.

Here is how everything looks like on the device itself. You can see the internal temperature by clicking Button 1.

With Button 2, we get the external temperature.

And, finally, with Button 3, we see the temperature difference. The LED will turn blue if the external temperature is less than the internal. Otherwise, it will be red.

To record the temperature, I created a dashboard in Arduino IoT Cloud by directly adding the Things to our dashboard.

If you want your system to be even more advanced and interactive, you can also install the Arduino IoT Remote app.

Resources

To simplify the system-creation process for you, I collected all the information you need.

Ideas for the Future

That's it! Now you have a fully functional IoT sensor that records outside environment variables, relays them over the cloud, and reports them to a second device in your home. In the future, I plan on further enhancing this by creating my own mobile or web application with a custom interface. (It should be simple with the JavaScript client we used already.) And then, on top of that, I'd like to store all the data points in my own Heroku Postgres instance, and share them via Heroku's Dataclips.

How to Build a Robust IoT Prototype In Less Than a Day - Part 2

Luca Cipriani — Mon, 29 Mar 2021 09:30:29 +0000

(part one is here)

How to Build a Robust IoT Prototype In Less Than a Day - Part 2

Welcome back to our second article about creating a robust, bidirectional IoT prototype in less than a day using Arduino, Heroku, and Node-RED. If you missed the first part, we covered setting up Node-RED, adding security, and deploying to Heroku. In this article we'll look at creating our embedded system on the Arduino boards, connecting them to our Node-RED instance, customizing Sketches, and creating a flow that allows our devices to talk to each other. Let's get started.

What is Arduino IoT Cloud?

To connect the physical devices (boards) to the internet we will use Arduino IoT Cloud, a device manager and data visualization tool. After setting up an account at https://create.arduino.cc, we can create our first digital twin.

A digital twin is a logical online representation of a system. In Arduino, this is referred to as “Things” or “Apps”. We need to create two different Apps—one for the sensors, and another for the actuators. Then, we can associate a physical device (one of our MKR1010 boards) to it.

Here is the scenario: We want one of the devices to read the environmental luminosity (the sensor device), and the LED Matrix on the other MKR1010 (the actuator device) to display ON if the value of the luminosity is over a certain limit, OFF otherwise.

First, we need to create a new Thing and give it a name. I will call mine EnvSensors. Then, we need to associate a device. To do this, click “Configure a new device”.

Before configuring the device, be sure to unplug any shield on the MRK1010 and plug it to your PC with a USB-micro cable. After installing the device, you can assign it a name. I will call it SensorsBoard.

In the first phase, the system will recognize the board; it should find an MKR1010. Then, it will upload a sketch to the board to automatically put an SSL certificate in the secure element for future encrypted communications. This process takes a few minutes, so be patient.

Once finished, we can continue creating our Thing. We need to create at least one property, luminosity, to start gathering data from our device.

The property variable name will be Luminosity. The Type will be Illuminance. We will also set a min value of 0 for the obscurity, and a max of 1000 for high brightness. Then we set the permissions as “Read Only” because we can only read from sensors; we cannot write values to them. Next, we want to send luminosity values periodically, so we set the frequency to one-second intervals.

Finally, click “Add Property” and check to make sure the SensorsBoard is set as the associated device.

Now we can click “Edit Sketch”, which will open a web IDE. Arduino IoT Cloud automatically creates a template to send the data as we specified before—we just need to read the data from the sensor.

To read the sensors from the board, we have to put the MKR ENV shield on top of the MKR WIFI 1010. Then, we can plug the USB cable again in the board and modify the sketch a bit before the final upload.

On the software side, we need to add a library to read the values from the sensor. The following line is enough:

#include <Arduino_MKRENV.h>

Then, to avoid trying to read data before the initialization, we continue looping until the library loads and communicates with the sensors:

 if (!ENV.begin()){
    Serial.println("Failed to initialize MKR ENV shield!");
    while(1);
}

And here is where the magic happens! The automatically-generated file named thingProperties.h has already declared the luminosity variable for us:

float luminosity;
void initProperties(){
  ArduinoCloud.setThingId(THING_ID);
  ArduinoCloud.addProperty(luminosity, READ, 1 * SECONDS, NULL);
}

Remember, never change thingProperties.h, it is automatically generated and can fall out of sync if you change it manually.

The interesting part is we can use the variable luminosity as a normal variable. It will be sent automatically to the Arduino Cloud every second, as we specified in the interface.

To read the data we just need a few lines in our loop() function:

void loop() {
  ArduinoCloud.update();
  luminosity = ENV.readIlluminance();
  delay(50); //avoid reading the sensor too many times
}

We are almost done, but how do we connect the board to the WiFi network? We specify the network credentials in the secret tab:

Here is the whole sketch.

Upload this sketch to your board.

Now it's time to verify that everything works as expected. So let's create a new simple dashboard in Arduino IoT Cloud:

https://create.arduino.cc/iot/dashboards

I named mine Sensors and Actuators. Now we'll add a new widget. In the Things tab, there's an easy way to create a dashboard with just one click. I selected my thing EnvSensors.

After placing the widget you should see the dashboard showing real-time luminosity data from the sensor:

Great! Now let’s create a new Thing and configure a new device. Let's call this new Thing EnvActuators and name the new device ActuatorsBoard. I suggest removing the SensorsBoard from your USB device. Avoid plugging in the LED matrix until after you configure the device.

The new Thing will have two variables: “display_text” to show online the same value the LED matrix is actually displaying, and a “switch” variable to tell us if the luminosity has reached a certain level. So we need to change the text on the display.

Display Test variable can be set as:

Name: Display Text

Variable: display_text

Type: Character String

Permission: Read and Write

Update: When the value changes

History: OFF

Name: LED Switch

Variable: led_switch

Type: ON/OFF (Bool)

Permission: Read and Write

Update: When the value changes

History: ON

After we click “Edit Sketch”, we have a new template to start with. There are two functions at the end of the sketch allowing the device to react when a property changes. Those functions are named CallBacks, and are called every time a property changes.

The following code allows you to change the value displayed in the LED Matrix every time the switch changes value from “true” to “false” and back.

void onLedSwitchChange() {
  if(led_switch){
    display_text = "on";
  }else{
    display_text = "off";
  }
}

While the main loop code is:

void loop() {
  ArduinoCloud.update();
  // Your code here
  MATRIX.beginDraw();
  MATRIX.clear();
  MATRIX.stroke(80, 80, 100);
  MATRIX.text(display_text, 0, 1);
  MATRIX.endDraw();
}

So every time the display_text variable changes, the LED Matrix clears and a new string is written on it.

You can find the entire code in my EnvActuators sketch.

Now we can upload the sketch to our MRK 1010 with the RGB Shield and add the Thing to the dashboard.

You should have a dashboard that looks similar to the image below:

Every time you change the led_switch toggle, the display text will change on both the display_text widget and on your physical device.

Interact with the Arduino Create IoT APIs

In order to create an application allowing the two different devices to speak to each other while adding our custom logic, we need to use the Arduino Create IoT REST APIs.

There are two kinds of APIs for Arduino IoT Cloud:

A fully-documented REST Arduino IoT Cloud API
A realtime API (only available for JavaScript as of the time of this writing), available on GitHub.

The system is throttled so you can't make more than 10 req/s for the REST API, but that's good enough for our use case.

In our first article, we learned how to deploy Node-RED on Heroku. This time we need to connect it to Arduino IoT Cloud by using an API token.

To obtain an IoT API token, go to the Things page and create one at the end of the page:

Be sure to keep your credentials in a safe place. (You won't be able to recover your client secret; you'll only be able to generate a new one.)

Connecting to Node-RED

If you followed the first article of this series, you should be able to access Node-RED from your Heroku Apps page. After accessing the app, you'll be able to see the Arduino blocks on the left panel and place them.

Double-clicking on a property will open a connection panel as shown below:

And clicking on the pencil will allow you to add the Arduino IoT Cloud API credentials.

Then, click “Add”. Now we can connect our luminosity property.

After clicking “Deploy” at the top right of the interface, we can begin to see the data flowing. Important: Nothing happens if you do not deploy your flow!

We are finally getting the property in Node-RED. The Luminosity I read is about 650 Lumen:

Design a Flow and Deploy It

Our flow is running on a Heroku Dyno. A Heroku Dyno is an application container with an ephemeral filesystem. This means your application will lose any data stored on a disk. Unfortunately, by default, Node-RED stores the flows and credentials on the filesystem.

There are many different strategies to avoid this, but they all require the development of a Node-RED configuration handler plugin. I created a simpler but effective solution. If we store the flows and the credentials in our source code, then they can be deployed on a new Dyno every time without losing our configuration. So I added the parameter flowFile: 'data/flows.json' in the settings.js file. Please see the previous article for more details.

Let’s run the application locally (in our node-red-private repo with the iot-proto branch).

$ heroku local

Now log into Node-RED to start creating the first flow. We should add an input property, a switch, and an output property. The input is connected to the luminosity sensor, while the output is connected to the LED Switch. In the middle, we'll place a switch block to control our LED value.

Our flow should look like this:

There are five components. The green ones are the Arduino properties, and include:

An input Arduino Property block (connected to the luminosity)
A logical switch, with the output 1 if the luminosity is above 300 and the output 2 if the luminosity is lower

A set block to activate the switch

A similar block to deactivate it
An output Arduino Property block bounded to the LED Switch property

Don't forget to click “Deploy” once you are done.If you're feeling lazy, you can import my simple Arduino IoT Cloud Node-RED Flow.

Now we have to commit our code for the flows.

$ git add data
$ git commit -a -m ‘Saving the flow'
$ git push heroku iot-proto:master

And finally, we have the app in production!

We now have the RGB Matrix displaying ON if the luminosity is more than 300, and OFF if it less than 300.

We now have a fully-functional, bi-directional IoT prototype up and running in just a few hours, and ready for any projects you can dream up.

Conclusion

Here is a quick recap of what we learned:

How to set up boards in Arduino IoT Cloud
How to set up properties for the Thing
How the Sketch is automatically built from the system, and how to customize it. Sketches are available at:
1. Sensor Sketch
2. Actuator Sketch
How to create a dashboard
How to use the Arduino IoT Cloud API
How to deploy Node-RED to Heroku
How to make the Heroku website persistent (with encrypted credentials)

All of those steps allowed us to:

Create a custom logic application on Node-RED
Have two different devices talk to each other

By leveraging Node-RED, you can also add many integrations with third-party services.

Potential Improvements

In the future, I'd like to add proper persistence by using a Redis or Postgres storage supported by the Heroku Database & Data Management System. This will require writing a small js plugin to allow Node-RED to store the flows and configurations in a database. I found an old repo by Atsushi Kojo containing a PoC. The code is old but I will probably update it in the future. The repo is named node-red-heroku.

After I finish that setup, it will be possible to use a nice Heroku feature, the Deploy to Heroku button. This will allow a one-click setup of everything we've seen so far.

What’s Next?

In the next article, we'll create a working prototype of an entire custom Node.js application by using the Arduino Real Time Js Client working inside Heroku and interacting with Arduino IoT Cloud.

How to Build a Robust IoT Prototype In Less Than a Day - Part 1

Luca Cipriani — Thu, 14 Jan 2021 13:53:01 +0000

The IoT world is powerful, but also quite complex. There are many protocols and vendors and the landscape is fragmented. [As Matt Turck has already shown in 2018.]

Creating stable and effective prototypes is becoming essential for every web or embedded developer. However, to create a prototype, you have to deal with:

the hardware
the connectivity
the embedded software
the IoT vendor API
the security and deployment of your custom application.

Even for a basic prototype, this process could take weeks. In this article, I’ll show you how to build a robust IoT prototype in only a few hours by leveraging the power of Arduino, Heroku, and Node-RED.

The Project

Many IoT projects focus only on reading data from sensors. However, for real prototypes, you need bi-directionality of the data, from the IoT sensor node to the cloud and back from the cloud to the actuators. For all of you with more knowledge in web development and a little less in embedded systems, here are the definitions of sensor and actuator.

A sensor is an input device, reading some physical dimension and converting it to either a digital or analog value. This value is then read by a microcontroller. If you want to measure temperature, you would use a temperature sensor.
An actuator is an output device, acting on the physical world and changing the status of something physical. A switch or an LED are both actuators—you can turn them ON or OFF.

If you want to know more, read this article about the difference between sensors and actuators.

The overall architecture of most IoT projects might look something like this:

So let’s create a sample app that will allow you to quickly create prototypes. We’ll use the following components:

Arduino boards as the IoT Node.
Arduino compatible shields as the sensors and actuators.
Arduino IoT Cloud account as the IoT server.
Node-RED for business logic (running in Heroku and with source code in GitHub)
A JavaScript custom application for the UI and mobile use case.

In our sample app, we’ll create a scenario where one Arduino board sends data from a sensor. (We will use luminosity, temperature, and humidity sensors.) The other device will show some information on the LED matrix as soon as it gets the information.

What you need:

2x Arduino MKR WiFi 1010 ($32.10)
1x Arduino RGB Shield ($44.90)
1x Arduino MKR ENV Shield ($34.40)
An Arduino IoT Cloud Maker Account ($6.99/month)
A GitHub Account (free)
A Heroku Hobby Account (free, or $7/month to avoid your app going to sleep)

Let’s Get Started

As a first step, while we wait for the hardware to arrive, we can set up Node-RED. Node-RED is an incredibly powerful low-code programming tool for event-driven applications based on Node.js. It allows you to easily and graphically wire together flows between hardware devices, then deploy those flows with just one click. Node-RED is built on Node.js. (I will assume you are already familiar with Node.js. If not, then you can learn about and install it at https://nodejs.org/.)

Installing Node-RED

TL;DR: If you want to quickly test the application you can skip this part and jump to the section Wrapping It Up which has a link to a quick-start version of this project.

However, if you want to understand how to build this yourself and eventually update your work to the latest version of Node-RED, please follow the detailed instructions below.

It’s a little bit of work to install Node-RED and get it set up, secured, and working, so let’s walk through the steps. To start, be sure you have the latest stable version. For our purpose, we are going to use git.

git clone git@github.com:node-red/node-red.git node-red-private
cd node-red-private
## Let's move the latest stable version as of 2020-07-01
git checkout 1.1.0
## And create our branch for experimenting
git checkout -b iot-proto
## Install deps and build the app
npm install
npm run build
## Start the app
npm start

Now Node-RED should be up and running, usually at http://localhost:1880. Open it, and you should see an interface similar to the following:

This is a good start, but there's no security, and everything is running locally. This setup is not suitable for serious prototyping, and we do not want to behave like this comic.

https://donthitsave.com/comic/2016/07/15/it-works-on-my-computer

Adding Credentials to Node-RED

To add security to Node-RED it’s best to protect the interface (static assets), the visual editor, and the Admin API. To do this, follow the official guide about Securing Node-RED. To add the credentials, we need a way to create a password and to obtain a hash for it. Node-RED has a simple tool for this called node-red-admin. It can be installed via:

$ npm install --save-dev node-red-admin
## Here is the dir where we will put our configurations
$ mkdir data

Let’s also install the Arduino plugin in the same directory.

$ cd data
$ npm install @arduino/node-red-contrib-arduino-iot-cloud

Now let’s create a password. The script will ask for a password interactively.

# ./node_modules/node-red-admin/node-red-admin.js hash-pw
Password: 
$2b$08$xGtfBswU4BF53FymNK9oae/Oz56d35W4/xxxxxxxxxxxx9cbC

The string given back is the hash we can use in the settings.js file. A proper settings file can be found on the Github repo of Node-RED. Let's start with that file and adapt it to our needs. First, we have to uncomment the part about adminAuth and change it to something like this:

adminAuth: {
       type: "credentials",
       users: [{
           username: process.env.NODERED_USERNAME || "admin",
           password: process.env.NODERED_PASSWORD_HASH,
           permissions: "*"
    }]
},

This allows us to use environment variables to set the hash of the password. (we just created it with the hash-pw command of node-red-admin), and a username (with fallback to “admin”).

The same will apply to another section of the file:

httpStaticAuth: {
        user: process.env.NODERED_USERNAME,
        pass: process.env.NODERED_PASSWORD_HASH
    },

And

credentialSecret: process.env.NODERED_CREDENTIAL_SECRET,

My complete settings file is available at [[ xxxxx ]]

The final important step is to set the port.

    uiPort: process.env.PORT || 1880,

Be sure to save your work in a proper directory. Usually, Node-RED saves the data into $HOME/.node-red, but this will not work well with our setup in Heroku, nor is useful to have the source code revisions for our work. So I added...

     flowFile: 'data/flows.json',

And

     userDir: 'data/',

The full file settings.js without comments is less than 40 lines.

module.exports = {
    // the tcp port that the Node-RED web server is listening on
    uiPort: process.env.PORT || 1880,
    mqttReconnectTime: 15000,
    serialReconnectTime: 15000,
    debugMaxLength: 1000,
    flowFile: 'data/flows.json',
    credentialSecret: process.env.NODERED_CREDENTIAL_SECRET,
    userDir: 'data/',
    adminAuth: {
       type: "credentials",
       users: [{
           username: process.env.NODERED_USERNAME || "admin",
           password: process.env.NODERED_PASSWORD_HASH,
           permissions: "*"
    }]
    },
    httpStaticAuth: {
        user: process.env.NODERED_USERNAME || "admin",
        pass: process.env.NODERED_PASSWORD_HASH
    },
    functionGlobalContext: {
    },
    exportGlobalContextKeys: false,
    logging: {
        console: {
            level: "info",
            metrics: false,
            audit: false
        }
    },
    editorTheme: {
        projects: {
            enabled: false
        }
    }
}

Setting Up Environment Variables

To declare the needed variables, you can create a local .env file like this one:

NODERED_PASSWORD_HASH='yourpassword-hash-from-the-node-red-admin-output'
NODERED_USERNAME='admin'
NODERED_CREDENTIAL_SECRET='averylongsecretofyourchoice'

Now let’s load those variables and start Node-RED again.

$ export $(cat .env | xargs)
$ npm run build
$ npm start

As soon as we see the Node-RED interface again, we will be prompted with a password.

Our server is now secure, but still running locally. And after the login, we should be able to see the new Arduino plugin available in the left palette.

The Arduino IoT Cloud plugin for Node-RED allows Node-RED to read data from the sensors, send a payload back to the board, inject a value in one of our Arduino IoT Cloud properties, and access the historical values of a property. A property is the logical web representation of either a sensor or an actuator.

Now it’s time to make everything simpler and use Heroku to deploy Node-RED to our staging environment.

Installing Heroku CLI

Heroku is a PaaS vendor that makes it easy and fast to deploy apps to the cloud. We’ll use the Heroku CLI to set up and deploy to Heroku from our local machine. (The best way to install and understand how this works is by following the official tutorial on how to install Heroku CLI.) Once installed, in the same directory of our Node-RED instance, run

$ heroku login -i
heroku: Enter your login credentials
Email [l.cipriani@hey.com]:             
Password: **************
Two-factor code: ******
Logged in as l.cipriani@hey.com

Then create a new app.

$ heroku create
Creating app... done, ⬢ arcane-castle-06470
https://arcane-castle-06470.herokuapp.com/ | https://git.heroku.com/arcane-castle-06470.git

And finally give it a better, memorizable name:

    $ heroku apps:rename node-red-arduino
    Renaming arcane-castle-06470 to node-red-arduino... done
    https://node-red-arduino.herokuapp.com/ | https://git.heroku.com/node-red-arduino.git
    Git remote heroku updated
     ▸    Don't forget to update git remotes for all other local checkouts of the app.

From now on, we’ll let Heroku handle the heavy work with deploying and running the app, as it supports Node.js and grunt out of the box.

To run Node-RED locally with Heroku, mimicking 100% what will happen in production, you can run...

    $ heroku local

This will open the app at http://localhost:5000. However, we want to run and deploy our app on Heroku. This is just a matter of setting our vars and then pushing to the Heroku git endpoint.

    ## This will set the proper variables in Heroku. You can skip this if you want to use the Heroku Web UI to set your variables
    $ cat .env | xargs heroku config:set
    ## Now deploy our app!
    $ git push heroku iot-proto:master

And see it online!

Wrapping It Up

We now have a secure Node-RED instance running on Heroku and ready to hook up to our Arduino boards.

I’ve also built a quick way to reproduce the steps I’ve shown in detail. Full instructions are available at my node-red-iot repo on Github.

Here’s a recap of what we learned today:

The most common IoT Architectures
Setting up Node-RED locally
Adding security to Node-RED
Add environment variables and manage settings
Install Heroku CLI
Deploy our Node-RED instance to Heroku

Next Steps

In the next article, I’ll show you how to create our embedded system on the Arduino boards, connect them to our Node-RED instance, then create and save a flow to our Heroku instance that is resilient to restarts. Finally, I’ll show you how to create a custom application on top of this setup using JavaScript.