DEV Community: Marek Sadowski

Visual Recognition App with Node-RED

Marek Sadowski — Sat, 01 Jun 2019 06:22:00 +0000

This blog post demonstrates how one can make custom AI service engaged in one's solution effortlessly under 10 minutes, learn a new node-flow based code development with Node-Red.

To make it happen you will learn about Watson Visual Recognition (it is a seasoned - commercial - industry grade AI service available via IBM Cloud), Watson Studio for fast machine learning, Node-Red - the javascript based graphical node-flow language to use the created Visual Recognition classifier in the on-line mode.

The why

I am a full stack developer always looking for great technologies to utilize in modern systems. Since I founded a robotics startup I have been always amazed with the use of AI powered visual recognition services for drones and robots. In this blog post I am sharing with you how it is easy now to make AI doing stuff for you and your systems/robots/drones. Back 10 years ago I tried to use autonomous robots in a swarm to detect armed intruders - the task seemed to me impossible then to tackle it without huge financial resources. Today, the solution can be delivered after a day long hackathon. _Man, I wish I knew then what I am going to share with you now ;) _

In addition United Nations, Red Cross and IBM among other organizations support CallForCode.org hackathon to help people affected by disasters. Using drones with image recognition might impact and improve well being of all those affected. I will show the code for detecting houses affected by wild fires, floods, etc. So why not take this call, contribute to the cause and help others!

Using Watson Visual Recognition in Node-Red

Watson Visual Recognition is the trained AI based service that can tell you what it sees on the picture with the pretrained classifiers. It can detect colors, people, faces, some objects, foods, etc. You are going to really like it.

Node-Red is an open source project and a programming tool for wiring together hardware devices, APIs and online services in new and interesting ways. It provides a browser-based editor that makes it easy to wire together flows using the wide range of nodes in the palette that can be deployed to its runtime in a single-click (*repeating after the nodered.org site)

So let's start with IBM Visual Recognition - the last time I checked on May 30th 2019 - you are able to call the service free of charge with a IBM Cloud lite account about thousand times a month. And you can get yourself a free lite account (just providing your email) by simply clicking on my personalized link (if you use this URL, I will be told that this blog is a good one) : https://ibm.biz/Bd2CUa

Just a side note - you can easily prepare a working Proof of Concept prototype for investors while just using the IBM Cloud Lite Account.

After you sign up, log in and just open the Catalog and select from AI category the Watson Visual Recognition service, and create it. After that you need to also create the Node-RED Starter service. Both services would work in the free tier!

When Node-RED service is provisioned, connect it with the previously created Watosn Visual Recognition service. You would need to restage the Node-RED service after you connect them together. When you set the service up for running you can create the first flows and use the following Node-RED lab: https://github.com/watson-developer-cloud/node-red-labs/tree/master/basic_examples/visual_recognition

*a small hint: at the bottom of the Node-RED lab page there is a link to a full JSON of the required Node-RED flow. Just copy the flow's JSON, import the JSON in the flow editor from the clipboard, and you would be good to start this service in matter of 1 minute! Do not forget to use "DEPLOY" button to install the new or changed flows.

Your service will look like this:

Now you can try your test service in your web browser thru this simple web application (just use the link with /reco extension) - for example: http://my-node-red-app.mybluemix.com / reco

Customizing the AI service with the custom classifier

If you want to add something new - create a custom classifier - you need to provide 1,000s of pictures for positive detection, and as I was told - about a half of this number with "negative" pictures - ie. pictures that do not represent the chosen classification.

The very basic example would start to work with just dozen pictures. The perfect example is provided by John Walicki, who trained a custom classifier to detect burnt houses in the pictures taken by drones. You can use one of his labs to do such a drone classifier - the labs on creating a classifier to detect houses demolished by wild fires, or flooded are presented in this github based labs: https://github.com/IBM/drones-iot-visual-recognition

I created a short video showing how it is being done step by step (yet the video is speeded up 2-3 times): https://youtu.be/kW7cjuWuPS0

In order to train the Watson Visual Recognition you will use IBM Watson Studio. In order to recap what is required to train the Visual Recognition service is to provide individual pictures or group them together in the zip files - one for positive, and one with the negative picture set. In the free tier you can get yourself 2 classifications like this.

Now in oder to view it - just expand your existing Node-RED flow with an additional function node and fill it out with the following code more details are in the step 3 in the link here - https://github.com/blumareks/ai-visrec-under-10min

Summary

With the created custom classifier you are able to equip your drone, or software with the ability to detect survivors and victims of the disasters. The power of Watson Visual Recognition could be used for good, in the CallForCode.org hackathon and beyond that.

What solution are you going to implement with it? Let me know!

Follow me on Twitter @blumareks

Monitor response times of cloud Java microservices with Databases for MongoDB

Marek Sadowski — Thu, 21 Mar 2019 01:15:00 +0000

One of my customers asked me to help them with a simple dashboard concept that would display series of availability and response times of the CRUD (Create, Read, Update, Delete) operations of the various data services exposed on the IBM Dedicated Cloud. By coincidence it also became a good example on how you can use CRUD operations on those data services accessed from a Java Open Liberty server on a local machine, as well as bound in the cloud, through the VCAP configuration file.

This tutorial instructs you on how to check the response times of services hosted on IBM Cloud. This is a simple tool that can also show you how to do the basic CRUD operations in your Java application/microservice on strategic datasources like Cloudant, IBM Message Hub, MongoDB, Elastic Search, and more.

This particular how-to post is an example on using *Databases for MongoDB * that is hosted and managed on IBM Cloud. The approach shown here is to generate a JSON response file with the report from accessing CRUD operations and provide the time needed to access each and every operation, in addition to the total operation time. The example of the response file you will try to create can be seen below.

{
"service": "mongodb",
"operations": [{
"type": "create",
"response_time": 30,
"response_code": 200,
"desc": {
"visitor id": "594ddeee34a639002645674d"
}
},
{
"type": "read",
"response_time": 25,
"response_code": 200,
"desc": {
"visitor id": "594ddeee34a639002645674d"
}
},
{
"type": "update",
"response_time": 49,
"response_code": 200,
"desc": {
"visitor id": "594ddeee34a639002645674d"
}
},
{
"type": "delete",
"response_time": 28,
"response_code": 200,
"desc": {
"deleted visitor id": "594ddeee34a639002645674d"
}
}
],
"response_code": 200,
"desc": "operations implemented CRUD/CRUD"
}

Learning objectives

After completing this tutorial, the reader will know how to:

* Track service response times
* Run Java Open Liberty server locally and also in the cloud
* Connect Java Open Liberty server to the exposed service

Prerequisites

Before you start you will need the following software on your machine:

* Git
* Cloud Foundry CLI
* Maven

You'll also need:

* An IBM Cloud - cloud.ibm.com account (note that we'll be using non-lite tier services)
* An IDE. In this example I use Eclipse with IBM Cloud plugin to deploy the application to the cloud.

Estimated time

Doing the entire lab takes about 20 minutes to 60 minutes for those who would set the environment up for the first time.

Steps

Step 1. Clone the sample app

Clone the https://github.com/blumareks/BluemixTestDashboard GitHub repository.

git clone https://github.com/blumareks/BluemixTestDashboard

Change the directory to where the sample app for MongoDB is located. In order to do it use the instructions below:

cd BluemixTestDashboard/GetStartedJavaMongoDb

That's good enough for now. We'll next provision a few resources on IBM Cloud and get back to the cloned repo.

Step 2. Create the cloud services

Log in to the IBM cloud.ibm.com console.
Search the \ **Catalog* * for **Databases for MongoDB** and `Liberty for Java `.

Create both the Liberty for Java sample app and the Databases for MongoDB service.
Once the two are created, bind the MongoDB service with the Liberty for Java app.
1. Click the *Connections* tab of the Liberty for Java service, and then click the *Create connection* button.

Select the **Databases for MongoDB** service.

Click the *Connect* button.

This action will initiate a restaging process of the Liberty for Java service.

Step 3. Connect JVM system properties for TLS/SSL to Databases for MongoDB

Go to the **Service Credentials** tab in the

Databases for MongoDB

service.
From the menu, select **View Credentials**.

The connection credentials from the VCAP file will look like this:

{
"connection": {
"cli": {
"arguments": [
[
"-u",
"ibm_cloud_[characters removed]",
"-p",
"[characters removed]",
"--ssl",
"--sslCAFile",
"b226140c-3543-11e9-837a-ae06e730b715",
"--authenticationDatabase",
"admin",
"--host",
"replset/4ffb04cb-473b-4895-879e-becf8fb4ac40-0.d7deeff0d58745aba57fa5c84685d5b4.databases.appdomain.cloud:31676,4ffb04cb-473b-4895-879e-becf8fb4ac40-1.d7deeff0d58745aba57fa5c84685d5b4.databases.appdomain.cloud:31676"
]
],

"bin": "mongo",
"certificate": {
"certificate_base64": "[characters removed]",
"name": "[characters removed]"
},
"composed": [
"mongo -u ibm_cloud_[characters removed] -p [characters removed] --ssl --sslCAFile b226140c-3543-11e9-837a-ae06e730b715 --authenticationDatabase admin --host replset/4ffb04cb-473b-4895-879e-becf8fb4ac40-0.d7deeff0d58745aba57fa5c84685d5b4.databases.appdomain.cloud:31676,4ffb04cb-473b-4895-879e-becf8fb4ac40-1.d7deeff0d58745aba57fa5c84685d5b4.databases.appdomain.cloud:31676"
],
"environment": {},
"type": "cli"
},
"mongodb": {
"authentication": {
"method": "direct",
"password": "[characters removed]",
"username": "ibm_cloud_[characters removed]7"
},
"certificate": {
"certificate_base64": "[characters removed]",
"name": "b226140c-3543-11e9-837a-ae06e730b715"
},
"composed": [
"mongodb://ibm_cloud_[characters removed]:[characters removed]@4ffb04cb-473b-4895-879e-becf8fb4ac40-0.d7deeff0d58745aba57fa5c84685d5b4.databases.appdomain.cloud:31676/ibmclouddb?authSource=admin",
"mongodb://ibm_cloud_[characters removed]:[characters removed]@4ffb04cb-473b-4895-879e-becf8fb4ac40-1.d7deeff0d58745aba57fa5c84685d5b4.databases.appdomain.cloud:31676/ibmclouddb?authSource=admin"
],
"hosts": [
{
"hostname": "4ffb04cb-473b-4895-879e-becf8fb4ac40-0.d7deeff0d58745aba57fa5c84685d5b4.databases.appdomain.cloud",
"port": 31676,
"protocol": "mongodb"
},
{
"hostname": "4ffb04cb-473b-4895-879e-becf8fb4ac40-1.d7deeff0d58745aba57fa5c84685d5b4.databases.appdomain.cloud",
"port": 31676,
"protocol": "mongodb"
}
],
"path": "/ibmclouddb",
"query_options": {
"authSource": "admin"
},
"scheme": "mongodb",
"type": "uri"
}
},
"instance_administration_api": {
"deployment_id": "crn:v1:bluemix:public:databases-for-mongodb:us-south:a/[characters removed]:[characters removed]::",
"instance_id": "crn:v1:bluemix:public:databases-for-mongodb:us-south:a/[characters removed]:[characters removed]::",
"root": "https://api.us-south.databases.cloud.ibm.com/v4/ibm"
}
}
Scroll down, and notice that the connection is SSL-enabled. If you do not have a credential file, create one using the defaults. Copy over the entire json. You will need the following parts:

"certificate": {
"certificate_base64": "[characters removed]",
"name": "b226140c-3543-11e9-837a-ae06e730b715"
},

URL: the first link from "composed": [
"composed": [
"mongodb://ibm_cloud_[characters removed]:[characters removed]@4ffb04cb-473b-4895-879e-becf8fb4ac40-0.d7deeff0d58745aba57fa5c84685d5b4.databases.appdomain.cloud:31676/ibmclouddb?authSource=admin",
"mongodb://ibm_cloud_[characters removed]:[characters removed]@4ffb04cb-473b-4895-879e-becf8fb4ac40-1.d7deeff0d58745aba57fa5c84685d5b4.databases.appdomain.cloud:31676/ibmclouddb?authSource=admin"
],

and from the same "composed" you can take the secondary URL (the second one from hosts)
"hosts": [
{
"hostname": "4ffb04cb-473b-4895-879e-becf8fb4ac40-0.d7deeff0d58745aba57fa5c84685d5b4.databases.appdomain.cloud",
"port": 31676,
"protocol": "mongodb"
},
{
"hostname": "4ffb04cb-473b-4895-879e-becf8fb4ac40-1.d7deeff0d58745aba57fa5c84685d5b4.databases.appdomain.cloud",
"port": 31676,
"protocol": "mongodb"
}
],

We'll need to set several JVM system properties in our application to ensure that the client is able to validate the TLS/SSL certificate presented by the server:

  \* `javax.net.ssl.trustStore`: The path to a trust store containing the certificate of the signing authority.
  \* `javax.net.ssl.trustStorePassword`: The password to access this trust store.

Note that the trust store is typically created with the keytool command-line program provided as part of the JDK. For example:

keytool -importcert -trustcacerts -file -keystore -storepass

Go back to the **Databases for MongoDB** console and get the SSL certificate encoded_64 available on the **Credentials** page. You will need to **decode64 it**. I used https://www.base64decode.org service to do it. When you decode the copied certificate, then you can use it in the next step.
Save the SSL certificate in a file:

cat > mongodbcert.crt

The content should look like:

-----BEGIN CERTIFICATE-----
LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0
[... removed couple lines ...]
VSVElGSUNBVEUtLS0tLQo
-----END CERTIFICATE-----

Use the keytool command to create the MongoDB keystore for your system. Note that you may want to consider changing the password to something more appropriate.

keytool -importcert -trustcacerts -file ./mongodbcert.crt -keystore ./mongoKeyStore -storepass aftereight

When done, place the mongoKeyStore in this path of your cloned repository:

GetStartedJavaMongoDb/src/main/resources/mongoKeyStore

You might want to comment the lines for in the IBM Cloud connection, and switch it to a local store thru uncommenting the following lines in the file MongoDbVisitorStore.java cloned from github in the following location: BluemixTestDashboard/GetStartedJavaMongoDb/src/main/java/wasdev/sample/store

Furthermore for the local Liberty and MongoDB tests you want to define the MongoDB URL in the local file here: /BluemixTestDashboard/GetStartedJavaMongoDb/src/main/resources/mongo.properties . For that use the previously captured elements from json:

URL
Part of the host2;
add &ssl=true at the end of the URL.

It should look somehow like that:

mongo_url=mongodb://ibm_cloud_[characters removed]:[characters removed]@4ffb04cb-473b-4895-879e-becf8fb4ac40-0.d7deeff0d58745aba57fa5c84685d5b4.databases.appdomain.cloud:31676,4ffb04cb-473b-4895-879e-becf8fb4ac40-1.d7deeff0d58745aba57fa5c84685d5b4.databases.appdomain.cloud:31676/ibmclouddb?authSource=admin&ssl=true

From the project root directory, run Maven (ensure it's installed by running mvn --version) in a terminal:

mvn install

The document after the mvn install is going to be locally stored at this location:

/your-path-to-the-target/GetStartedJavaMongoDb/target/TestJavaMongo-1.0-SNAPSHOT/WEB-INF/classes/mongoKeyStore

Or on IBM Cloud (after a cf push command):

/home/vcap/app/wlp/usr/servers/defaultServer/apps/myapp.war/WEB-INF/classes/mongoKeyStore

A typical application will also need to set several JVM system properties to ensure that the client presents a TLS/SSL certificate to the MongoDB server:

* javax.net.ssl.keyStore: The path to a key store containing the client's TLS/SSL certificates.
* javax.net.ssl.keyStorePassword: The password to access this key store.

Step 4. Running locally on Open Liberty with Databases for MongoDB in the IBM Cloud

In order to test and run the service locally, you might need to install the JAXRS-2.0 feature on the local server. I am using Eclipse IDE for that purpose. The server configuration screen with added JAXRS-2.0 looks like the image below:

The source of the local Liberty server managed by Eclipse IDE looks like this:

localConnector-1.0
servlet-3.1
jaxrs-2.0

After building the project with the mvn install command, you are able to run it on the above configured local server.

The following image shows the results after calling the service locally:

There is a round-trip time of about five seconds for the first call (we needed to create the DB and run everything remotely), which is not very impressive. But the next step would show running the same service in IBM Cloud, and the response times will be at least 10x better. Below are my results, but please keep in mind that the following calls were faster because our services were already provisioned and a connection established.

{

"service": "mongodb",

"operations": [{

        "type": "create",

        "response\_time": 386,

        "response\_code": 200,

        "desc": {

            "visitor id": "5aec8faf17683ad076f4fc58"

        }

    },

    {

        "type": "read",

        "response\_time": 303,

        "response\_code": 200,

        "desc": {

            "visitor id": "5aec8faf17683ad076f4fc58"

        }

    },

    {

        "type": "update",

        "response\_time": 579,

        "response\_code": 200,

        "desc": {

            "visitor id": "5aec8faf17683ad076f4fc58"

        }

    },

    {

        "type": "delete",

        "response\_time": 164,

        "response\_code": 200,

        "desc": {

            "deleted visitor id": "5aec8faf17683ad076f4fc58"

        }

    }

],

"response\_code": 200,

"desc": "operations implemented CRUD/CRUD"

}

Step 5. Running Open Liberty with Databases for MongoDB in the IBM Cloud

There are at least two ways you can deploy and run this service in the cloud. You can either manually adjust the Cloud Foundry files and run it with the Cloud Foundry CLI, or use the IBM Cloud connector for Eclipse. In this tutorial, we use the Cloud Foundry CLI.

Download CLI from IBM Cloud.
1. Go to the Liberty service.
2. Selecting the **Getting started** tab.
3. Download the Command Line Interface:
As soon as the CLI is installed, you can run the version command:

ibmcloud --version
ibmcloud version 0.6.6+d4d59ab5-2018-03-20T07:49:59+00:00

Set your access point. (Note that the API access point depends on the cloud location in US, EU, AU, and so on.)

ibmcloud api https://api.

ibmcloud login -u

After you log in, you can interactively select the org and space to use.

ibmcloud target --cf

Now edit the Cloud Foundry configuration file manifest.yml. The file should have the name of the application (your Liberty service) and the bound Databases for MongoDB service.

In this example it looks like this:

applications:
- name:
random-route: true
path: target/TestJavaMongo.war
memory: 256M
instances: 1
name: test-java-mongodb
host: test-java-mongodb

Update the location of the MongoDB keystore in the cloud in the java class file: MongoDbVisitorStore.java
You can now build the app for the cloud with Maven:

mvn clean
mvn install

After the mvn commands complete without errors, you are ready to push your app to thr cloud. It will take a few minutes — just enough to get a cup of coffee. :-)

ibmcloud app push

Consider looking at the logs of the document — on the dashboard of the service (see the **Logs** tab), or use the CLI in the terminal:

ibmcloud app logs

But check this out - the CRUD operation in the IBM Cloud is 100x faster than the first call, and 10x faster than the remote call from your local machine.

{

"service": "mongodb",

"operations": [{

        "type": "create",

        "response\_time": 12,

        "response\_code": 200,

        "desc": {

            "visitor id": "5aecb5b0705978002421614a"

        }

    },

    {

        "type": "read",

        "response\_time": 18,

        "response\_code": 200,

        "desc": {

            "visitor id": "5aecb5b0705978002421614a"

        }

    },

    {

        "type": "update",

        "response\_time": 11,

        "response\_code": 200,

        "desc": {

            "visitor id": "5aecb5b0705978002421614a"

        }

    },

    {

        "type": "delete",

        "response\_time": 8,

        "response\_code": 200,

        "desc": {

            "deleted visitor id": "5aecb5b0705978002421614a"

        }

    }

],

"response\_code": 200,

"desc": "operations implemented CRUD/CRUD"

}

Summary

I hope you enjoyed reading this tutorial! My next tutorial will be about adding an app to read, parse, and store the results. Stay tuned, and please follow me on Twitter @blumareks or on Medium for more updates!

This blog post was originally posted at https://blumareks.blogspot.com/

Serverless - from "Hello World" to exposing REST Api in under 10 minutes

Marek Sadowski — Sun, 10 Mar 2019 20:21:49 +0000

So you've been hearing a lot of buzz about Serverless tech, but what exactly is the serverless or cloud functions? Serverless has become the new style of coding, and it might be perfect to offload your mobile apps, and container based systems without incurring unnecessary costs. Read more here

Hack Raspberry Pi to tell you a joke

Marek Sadowski — Tue, 05 Mar 2019 23:16:36 +0000

This blog post is about enabling joking skill to the Raspberry Pi - now you can easily and fast build a basic "fun" skill and add it to your personal assistant. And this is the DIY personal assistant that is based on the basic IoT device - Raspberry Pi computer.