DEV Community: Thomas Südbröcker

SIMPLE NODE.JS SERVER EXAMPLE USING THE WATSON ASSISTANT API V2

Thomas Südbröcker — Sun, 08 Jan 2023 11:13:50 +0000

This blog post is about a simple example to use the Watson Assistant API v2 with the Node.js SDK to get a Watson Assistant sessionID and send a message to Watson Assistant using this sessionID. Here is the GitHub project watson-assistant-simple-node-js-server-example.

The objective is only to start a chat session and return the values we get from Watson Assistant API v2 methods. The simple Node.js server has implemented following REST endpoints :

The project contains examples for using Postman.

The blog post is structured in:

Connect to an assistant in the right environment
Implement a simple endpoint
Use the example
Summary

CONNECT TO AN ASSISTANT IN THE RIGHT ENVIRONMENT

Therefor we will first connect to our Watson Assistant service instance with the environmentID. Because we need to know to which environment we want to connect our server.

A small diagram with simplified dependencies. In my case I connected to the draft environment.

The GitHub project also contains a bash script to list the environments using cRUL.

The following code shows an extract of a return value of a bash script execution.

{"environments":[
    {"name":"draft",
     "description":"My Example Assistant",
     "environment":"draft",
     "assistant_id":"df8d00f8-e9cc-XXXXXXXX",
     "orchestration":{"search_skill_fallback":false},
     "environment_id":"3d6d1cd6-29fd-XXXXXXXX",
     "session_timeout":120,
      ...
}

The relevant code in server.js file to connect to the right environment of the Watson Assistant service.

const assistant = new AssistantV2({
    version: e_version,
    authenticator: new IamAuthenticator({
      apikey: e_apikey,
    }),
    serviceUrl: e_serverUrl,
  });

IMPLEMENT A SIMPLE ENDPOINT

When we are connected to the right assistant with the right environment, we can use this connection and we mostly just reuse the code given in the Watson Assistant API v2 documentation for the Node.js SDK in the implementation of our REST endpoint.

app.get('/getsession', (req, res) => {   
    assistant.createSession({
        assistantId: e_environmentID
    })
    .then(createSessionresult => {
        console.log(JSON.stringify(createSessionresult.result, null, 2));
        sessionID = JSON.stringify(createSessionresult.result, null, 2);
        res.statusCode = 200;
        res.write(sessionID);
        res.end();
    })
    .catch(createSessionError => {
        console.log(createSessionError);
        res.statusCode = 500;
        res.write(createSessionError);
        res.end();
    });  
});

This is a simplified diagram of the given dependencies of our scenario.

USE THE EXAMPLE

Here are the steps you can follow to run this example on your local machine.

STEP 1: CLONE THE PROJECT TO YOUR LOCAL COMPUTER

git clone https://github.com/thomassuedbroecker/
watson-assistant-simple-node-js-server-example.git
cd
watson-assistant-simple-node-js-server-example/code/simple-server

STEP 2: CONFIGURE THE ENVIRONMENT VARIABLES

Here we need an environmentID. We can find this ID in the Watson Assistant user interface, by navigating to the Environments page and selecting the environment you want to use (such as Draft or Live). Then you open the environment settings and copy the value from the appropriate field. (Contains the Link to the IBM Cloud documentation)

The image below shows the environment ID in the IBM Cloud UI.

We also need an API key and a base URL from the Credentials. The image blow shows the API key and the URL in the IBM Cloud UI.

Create an .env file

cat .env-template > .env

Configure the .env file to your needs

WATSON_ASSISTANT_VERSION='2021-11-27'
WATSON_ASSISTANT_API_KEY='XXXX'
WATSON_ASSISTANT_SERVICE_URL='https://api.us-south.assistant.watson.cloud.ibm.com'
WATSON_ASSISTANT_ENVIRONMENT_ID='XXX'

STEP 3: RUN LOCAL ON PORT 3010

npm install
npm start

STEP 4: OPEN BROWSER AND INVOKE THE GET SESSION ENDPOINT

It returns the newly created sessionID.

open http://localhost:3010/getsession

Example output:

{
"session_id": "37d13c72-1643-4add-bee5-574c6fd062dc"
}

STEP 5: IMPORT THE POSTMAN COLLECTION INTO POSTMAN

The images shows the import button in postman.

Select the /postman/watson-assistant-example.postman_collection.json in the project for the import.

STEP 6: AFTER THE IMPORT YOU HAVE A NEW COLLECTION INSIDE YOUR POSTMAN

The images shows the imported collection.

STEP 7: SEND A MESSAGE

Here again the simplified diagram of the given dependencies of our scenario.

STEP 7.1: GET A SESSIONID

Now we need to get the sessionID with GET request using the getsession endpoint. We copy the value of the sessionID to use it inside the next REST endpoint invocation.

STEP 7.2: INSERT THE `SESSION ID` INTO THE BODY OF THE `POST` REQUEST USING THE `SENDMESSAGE ENDPOINT`

This is the JSON format which is used in the POST request sendmessage body in Postman. Now we insert the copied sessionID into the body of the request inside postman.

{ "sendmessage" : {
  "sessionID" : "XXXX",
  "message_type": "text",
  "text": "Hello"
  }
}

Now we see the output of the sendmessage request in the expected JSON format.

SUMMARY

The usage of the Watson Assistant API v2 is very easy and good documented, but you need to understand the environment topic. At the moment the documentation talks about the assistantID to pass into an invocation as an example, this can cause a misunderstanding. Here a screen shot of the IBM Cloud documentation “createsession” created on 09.12.2022.

I hope this was useful for you and let’s see what’s next?

Greetings,

Thomas

watsonassistant, #nodejs, #ibmcloud, #rest, #javascript

Original blog post on www.suedbroecker.net SIMPLE NODE.JS SERVER EXAMPLE USING THE WATSON ASSISTANT API V2

RUN WATSON NLP FOR EMBED ON IBM CLOUD CODE ENGINE

Thomas Südbröcker — Wed, 04 Jan 2023 18:41:25 +0000

This blog post is about using the IBM Watson Natural Language Processing Library for Embed on IBM Cloud Code Engine and is related to my blog post Run Watson NLP for Embed on your local computer with Docker. IBM Cloud Code Engine is a fully managed, serverless platform where you can run container images or batch jobs.

The IBM Watson Libraries for Embed are made for IBM Business Partners. Partners can get additional details about embeddable AI on the IBM Partner World page. If you are an IBM Business Partner you can get a free access to the IBM Watson Natural Language Processing Library for Embed.

To get started with the libraries you can use the link Watson Natural Language Processing Library for Embed home. It is an awesome documentation and it is public available.

I used parts of the IBM Watson documentation in my Code Engine example and I created a GitHub project with some additional example bash scripting. The project is called Run Watson NLP for Embed on your IBM Cloud Code Engine.

The blog post is structured in:

Some simplified technical basics about IBM Watson Libraries for Embed
Some screen shots from a running example using IBM Cloud Code Engine
Execute the example on IBM Cloud Code Engine
Summary

1. SOME SIMPLIFIED TECHNICAL BASICS ABOUT IBM WATSON LIBRARIES FOR EMBED

The IBM Watson Libraries for Embed do provide a lot of pre-trained models you can find in the related model catalog for the Watson Libraries. Here is a link to the model catalog for Watson NLP, the catalog is public available.

There are runtime containers for the AI functionalities like NLP, Speech to Text and so on. You can integrate the container into your application and customize them, for example existing pre-trained models, but it’s also possible to use own created models. By separating the container runtime and the models to be loaded into the container, we are free to include the right models we need to integrate in our application to fulfil the needs of our business use case.

The resources are available in a IBM Cloud Container Registry. The image below shows a simplified overview.

The next gif shows a simplified overview of the dependencies and steps we need to execute when we using IBM Cloud Code Engine to run a custom Watson NLP runtime container. You can find the detailed instructions in the official IBM documentation installing running containers.

These are the simplified steps which are automated with bash scripting, which we see in the image above.

We create a custom Watson NLP container locally, by using the Watson NLP runtime image and select models for the example. Therefor we define a Dockerfile and we use the IBM entitlement key to access the runtime image and the models on the IBM Cloud Container Registry.
We use the IBM Cloud Container Registry to save and protect our custom container image. In the container image registry we create a namespace where we upload the image. The namespace name must be unique in the IBM Cloud region of the container registry.
We create a Code Engine project. The project must be unique in the IBM Cloud region of Code Engine. We also create a secret to access our IBM Cloud container registry and in this context the automation also need to create an IBM Cloud API key to do that task.
Now we configure a Code Engine application. For the configuration we use the container registry secret to load the container image from the IBM Cloud Container Registry and we configure an environment variable that we are going to accept the license when we run our custom Watson NLP container image.
To verify the running container image is working, we invoke a curl command to execute a SyntaxPredict from the Watson NLP API.

2. SOME SCREEN SHOTS FROM A RUNNING EXAMPLE ON CODE ENGINE

IBM Cloud Container Registry namespace and the uploaded container image:

The created IBM Cloud API key:

The created Code Engine project:

The created Code Engine application:

The created Code Engine Registry Access:

The created Envionment variable:

The mapped custom Watson NLP container image:

3. EXECUTE THE EXAMPLE ON IBM CLOUD CODE ENGINE

These are the steps you can follow along to run the example with custom Watson NLP container image on IBM Cloud Code Engine.

STEP 1: CLONE THE EXAMPLE PROJECT TO YOUR LOCAL COMPUTER

git clone https://github.com/thomassuedbroecker/watson-nlp-example-code-engine
cd watson-nlp-example-code-engine/code

STEP 2: SET NEEDED ENVIRONMENT VARIABLES IN THE `.ENV` FILE

cat .env-template > .env

Edit the .env file.

We need …

… an IBM_ENTITLEMENT_KEY to get the Watson NLP resources.
… an IBMCLOUD_APIKEY that the bash script can automate all tasks.
… an unique IBM Cloud Container Registry namespace name CR_NAMESPACE.
… to configure an email CE_EMAIL related to our container image registry access.
… an unique Code Engine project name CE_PROJECT_NAME.

# used as 'environment' variables
IBM_ENTITLEMENT_KEY="YOUR_KEY"
IBMCLOUD_APIKEY="YOUR_KEY"
CR_NAMESPACE="custom-watson-nlp-YOUR_NAME"
CE_EMAIL="YOUR_EMAIL"
CE_PROJECT_NAME="custom-watson-nlp-YOURNAME"

STEP 3: EXECUTE THE `RUN-WATSON-NLP-WITH-CODE-ENGINE.SH` BASH SCRIPT

The bash script execution can take several minutes.

sh run-watson-nlp-with-code-engine.sh

EXAMPLE OUTPUT AND THE LINKS TO THE RELATED FUNCTIONS IN THE BASH SCRIPT CODE

LINKS TO THE RELATED FUNCTIONS IN THE BASH SCRIPT CODE

EXAMPLE OUTPUT

# ******
# Create custom Watson NLP container image
# ******


# ******
# Connect to IBM Cloud Container Image Registry: cp.icr.io/cp/ai
# ******

...
Login Succeeded

# ******
# List model array content
# ******

Model 0 : watson-nlp_syntax_izumo_lang_en_stock:1.0.7
Model 1 : watson-nlp_syntax_izumo_lang_fr_stock:1.0.7

# ******
# Download the models
# ******

...

# 2. Put models into the file share
Archive:  /app/model.zip
  inflating: config.yml              
...

# ******
# Create container image
# ******

Image name: watson-nlp-runtime-with-models
[+] Building 0.1s (7/7) FINISHED                                                
 => [internal] load build definition from Dockerfile                       0.0s

...

# ******
# Upload image to IBM Cloud container registry 
# ******

...

# ******
# Configure IBM Cloud Registry
# ******

The region is set to 'us-south', the registry is 'us.icr.io'.

OK
Adding namespace 'custom-watson-nlp-tsued' in resource group 'default' for account XXXX Account in registry us.icr.io ...

OK
Logging 'docker' in to 'us.icr.io'...
Logged in to 'us.icr.io'.

OK
Container image: us.icr.io/custom-watson-nlp-tsued/watson-nlp-runtime-with-models:1.0.0
The push refers to repository [us.icr.io/custom-watson-nlp-tsued/watson-nlp-runtime-with-models]
2de7f9fb3378: Pushed 
...
9aaca8eae7c0: Pushed 
1.0.0: digest: sha256:8ad1b4bdd6a89b0686c48fc2325900018a1ddfec0ffb1689457b5c433300d61c size: 4290

# ******
# Create Code Engine project
# ******

**********************************
 Create Code Engine project: custom-watson-nlp-tsued
**********************************

...

Creating project 'custom-watson-nlp-tsued'...
ID for project 'custom-watson-nlp-tsued' is 'b879a212-XXXX'.
Waiting for project 'custom-watson-nlp-tsued' to be active...
Now selecting project 'custom-watson-nlp-tsued'.
OK
**********************************
 Configure IBM Cloud Container Registry Access (us.icr.io) for (custom-watson-nlp-tsued)
**********************************
**********************************
 Create Code Engine project: custom-watson-nlp-tsued
**********************************
API key: 
Creating API key cliapikey_custom-watson-nlp-tsued under 641ebfXXXXXXb45279e as XXXXX...
OK
API key cliapikey_custom-watson-nlp-tsued was created
Successfully save API key information to cli_key.json
Creating image registry access secret 'custom.watson.nlp.cr.sec'...
OK

# ******
# Create Code Engine application custom-watson-nlp-application
# ******

Creating application 'custom-watson-nlp-application'...
Configuration 'custom-watson-nlp-application' is waiting for a Revision to become ready.
Ingress has not yet been reconciled.
Waiting for load balancer to be ready.
Run 'ibmcloud ce application get -n custom-watson-nlp-application' to check the application status.
OK

Set CE_APPLICATION_NAME URL: https://custom-watson-nlp-application.wqtmqy9e03u.us-south.codeengine.appdomain.cloud

# ******
# Verify the custom image
# ******

...

{"text":"This is a test sentence.", "producerId":{"name":"Izumo Text Processing", "version":"0.0.1"}, "tokens":[{"span":{"begin":0, "end":4, "text":"This"}, 
...
"features":[]}], "sentences":[{"span":{"begin":0, "end":24, "text":"This is a test sentence."}}], "paragraphs":[{"span":{"begin":0, "end":24, "text":"This is a test sentence."}}]}

4. SUMMARY

It is easy to create a customize Watson Natural Language Processing Library for Embed container image and run it on IBM Cloud Code Engine.

I hope this was useful for you and let’s see what’s next?

Greetings,

Thomas

Original Blog Post: RUN WATSON NLP FOR EMBED ON IBM CLOUD CODE ENGINE

CUSTOM DOMAIN AND TLS CERTIFICATE FOR YOUR APPLICATION ON IBM CLOUD CODE ENGINE

Thomas Südbröcker — Wed, 04 Jan 2023 18:13:00 +0000

This blog post is about using the domain mapping functionality in IBM Cloud Code Engine to use custom domains and TLS certificates for your applications. The source code related to an example setup is in the GitHub project example WebApp build on Vue.js.

When we deploy an application to IBM Cloud Code Engine we get an IBM Cloud domain in following format **[CODE_ENGINE_APPLICATION].[CODE_ENGINE_PROJECT].[IBM_CLOUD_REGION].codeengine.appdomain.cloud** and a TLS certificate provided by Let’s encrypt. We are free to use this out-of-the-box for our application.

But now we are going to use a wild card certificate for our domain *.example.com. With a TLS certificate for that domain.

The blog post is structured in:

Used services or software
Simplified architecture overview
Example setup
Summary

1. USED SERVICES OR SOFTWARE

These are the services or software which are used in this example.

IBM Cloud
- IBM Cloud Code Engine is a fully managed, serverless platform where you can run container images or batch jobs.
- IBM Cloud Domain Name Registration

“IBM Cloud offers domain registration services complete with dedicated support staff, knowledgeable customer service, and reasonable prices, all delivered over a secure network.“

Open Source:
- Certbot

“Certbot is a free, open source software tool for automatically using Let’s Encrypt certificates on manually-administrated websites to enable HTTPS.“

Let’s encrypt

“Let’s Encrypt is a free, automated, and open certificate authority (CA), run for the public’s benefit. It is a service provided by the Internet Security Research Group (ISRG).“

Red Hat:
- Quay.io

“Quay.io is a registry for storing and building container images as well as distributing other OCI artifacts. The service is free for those who want to set up their own public repositories and available for a fee, if you want to create private repositories.“

2. SIMPLIFIED ARCHITECTURE OVERVIEW

The gif above shows a very simplified diagram with dependencies and steps related to the setup for an example application. The example application container is in my case saved in the public Quay.io container registry and the application container is based on the “Example WebApp build on Vue.js” GitHub project. We have in place a Code Engine project and the example web application is already deployed.

The following sections are a very simplified description of the steps in the simplified diagram above.

First, we buy a domain. In my case I bought the domain www.suedbroecker-example.com at the IBM Cloud Domain Name Registration for 10,00 Euro.
Then we use Certbot to create/request a TLS certificate at Let’s encrypt from our local computer. In my case I used a wildcard certificate ‘’ for the `.suedbroecker-example.com domain. I wanted to be flexible so that I can define new subdomains very easily and use this subdomain names for my new applications, for example with a naming like app01.suedbroecker-example.com, app02.suedbroecker-example.com` and so on.
Now Let’s encrypt provides a challenge we need to insert at our domain provider.
We need to insert the challenge to the domain provider in a TXT record. With that challenge Let’s encrypt can verify, if we are able to configure/own the given www.example.com domain using the challenge. Let’s encrypt will verify it and provide us the result with cert bot on the local computer. That can take sometimes several minutes.
When the verification was successful we get issued from Let’s encrypt a public TLS certifcate and private key which will be saved on our local computer by the cert bot application.
Now it’s time to start to configure the IBM Cloud Code Engine domain mapping. We add a new domain mapping and we insert the public TLS certificate and private key.
Now we need to configure in our domain provider the CNAME. In my case I configured the CNAME to map the wild card certificate ‘*’ to the link provided by Code Engine.
In my case I used a wild card ‘’ `.suedbroecker-example.com certificate, so I can configure a custom domain to point to the application. I configured the domain webapp.suedbroecker-example.com`.
Now we can invoke the domain webapp.suedbroecker-example.com with https. Our connection is now secured, and the browser can verify the TLS certificate is valid. The image below shows my running application on Code Engine using my TLS certificate.

The image below shows the example application.

3. EXAMPLE SETUP

These are the steps you can follow along to run the example application on IBM Cloud Code Engine. The GitHub project “Example WebApp build on Vue.js” contains the source code for this simple Vue.js web application.

3.1 GET THE SOURCE CODE

STEP 1: CLONE THE PROJECT

git clone https://github.com/thomassuedbroecker/vue-simple-webapp
cd vue-simple-webapp/code

STEP 2: SET THE NEEDED ENVIRONMENT VARIABLES IN THE `.ENV` FILE¶

cat .env-template > .env

Edit the .env file.

ASSISTANT_INTEGRATION_ID="YOUR_ID"
ASSISTANT_REGION="YOUR_REGION"
ASSISTANT_SERVICE_INSTANCE_ID="YOUR_SERVICE_INSTANCE"
ROOTFOLDER="YOUR_ROOTFOLDER" #Location on your local machine
MYPROJECT="YOUR_MYPROJECT" #IBM Cloud Code Engine project name
RESOURCE_GROUP="default" #IBM Cloud resource group
REGION="YOUR_REGION" #IBM Cloud region
CERT_EMAIL="YOUR_EMAIL" # If you want to create a TLS certification
CERT_DOMAIN="YOUR_DOMAIN" # If you want to create a TLS certification

You can follow the steps in the README.md of the “Example WebApp build on Vue.js” GitHub project. Then run the Vue.js application locally or as a container. When you run the application in a container a Nginx is used as the webserver. You are also able to deploy the application to IBM Cloud Code Engine. All of the options are automated with bash scripts.

3.2 REGISTER A DOMAIN

The following steps are showing, how to buy a domain at IBM Cloud.

STEP 1: REGISTER DOMAIN AT IBM CLOUD

Buy the domain

Verify your contact information

Configure the data sharing preferences

STEP 2: VERIFY YOUR DOMAIN IS READY TO USE

3.3 REQUEST/CREATE A TLS CERTIFICATE WITH `LET’S ENCRYPT AND CERTBOT`

In this context this blog post Generate Wildcard TSL certificate using Let’s Encrypt/Certbot was very useful. (Certificate authority CA)

STEP 1: INSTALL CERTBOT ON MACOS ¶

brew ``install` `certbot

STEP 2: EXECUTE TO CREATE-CERT.SH TO CREATE THE TLS CERTIFICATE

We are using the staging environment from Let’s encrypt. If you’re experimenting with different ACME clients, use the staging environment Let’s encrypt to avoid hitting rate limits.

The create-cert.sh does following automation:

Keep in mind the certificate will be valid for 90 days.

sh create-cert.sh

Output and interaction:

Insert your wild card domain: *.suedbroecker-example.com

Password:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Please enter the domain name(s) you would like on your certificate (comma and/or
space separated) (Enter 'c' to cancel): *.suedbroecker-example.com
Requesting a certificate for *.suedbroecker-example.com

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please deploy a DNS TXT record under the name:

_acme-challenge.www.suedbroecker-example.com.

with the following value:

n4H_yuDtBb6zXdZWg-XXXXXXX

Before continuing, verify the TXT record has been deployed. Depending on the DNS
provider, this may take some time, from a few seconds to multiple minutes. You can
check if it has finished deploying with aid of online tools, such as the Google
Admin Toolbox: https://toolbox.googleapps.com/apps/dig/#TXT/_acme-challenge.www.suedbroecker-example.com.
Look for one or more bolded line(s) below the line ';ANSWER'. It should show the
value(s) you've just added.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Press Enter to Continue

Insert the challenge in IBM Cloud Domain Name Registration in a TXT record:

Verify the challenge with the Google Toolbox. This task can take several minutes.

Now we can move on with the script and we see the downloaded certificates.

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/www.suedbroecker-example.com/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/www.suedbroecker-example.com/privkey.pem
This certificate expires on 2023-03-24.
These files will be updated when the certificate renews.

NEXT STEPS:
- This certificate will not be renewed automatically. Autorenewal of --manual certificates requires the use of an authentication hook script (--manual-auth-hook) but one was not provided. To renew this certificate, repeat this same certbot command before the certificate's expiry date.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Now you can insert the given locations for the cert and key and the script creates new files with different endings and it displays the cert and key we can easily copy and paste to the domain mapping.

Question: How to get .crt and .key files from? I just have .pem files answered here


Insert SOURCE_CERTIFICATE
/etc/letsencrypt/live/www.suedbroecker-example.com/fullchain.pem
Value: /etc/letsencrypt/live/www.suedbroecker-example.com/fullchain.pem

Insert SOURCE_KEY
/etc/letsencrypt/live/www.suedbroecker-example.com/privkey.pem
Value: /etc/letsencrypt/live/www.suedbroecker-example.com/privkey.pem

mkdir: /Users/thomassuedbroecker/Downloads/dev/vue-simple-webapp/simplewebapp/tls-cert
mkdir: /Users/thomassuedbroecker/Downloads/dev/vue-simple-webapp/simplewebapp/tls-cert/letsencrypt

Public cert:
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
Private key:
-----BEGIN PRIVATE KEY-----
...
-----END PRIVATE KEY-----

STEP 3: CONFIGURE THE DOMAIN MAPPING IN CODE ENGINE

Now we copy the certificate and key content into the domain mapping, and we configure the domain name. In my case I used the name webapp.suedbroecker-example.com, because I am free to choose the name and this related to the wild card certificate I use. In the image below you see the finally saved configuration in Code Engine domain mapping.

STEP 4: WE NEED TO INSERT THE LINK FROM CODE ENGINE INTO THE `CNAME`

In my case I mapped the wild card '*' to the link from Code Engine in the CNAME. It can take several minutes until you can use the mapped domain.

For details, please visit the Code Engine documentation custom domain configuration

STEP 5: OPEN THE DOMAIN URL

In my case it was webapp.suedbroecker-example.com.

4. SUMMARY

It’s awesome that Code Engine provides this functionality. With this functionality we are very flexible to configure our own domain. Maybe we can just use an IBM Cloud Domain Name Registration. You need to keep in mind you have to manage the certificate and update, because i will expire after 90 days. There are other blog posts about Code Engine and domain mapping out there like Custom domain for your serverless Code Engine app available.

I hope this was useful to you and let’s see what’s next?

Greetings,

Thomas

Original blog post: suedbroecker.net - CUSTOM DOMAIN AND TLS CERTIFICATE FOR YOUR APPLICATION ON IBM CLOUD CODE ENGINE

CONNECT TO A POSTGRESQL DATABASE, CREATE TABLES, INSERT DATA, AND USE A FILE FROM A GITHUB PROJECT WITH GO

Thomas Südbröcker — Thu, 03 Mar 2022 15:57:14 +0000

This blog post covers the topic connect to a PostgreSQL database, create tables, insert data and use a file from a GitHub project and do the implementation with GO. That blog post is also related to my last blog post called Connect to a PostgreSQL database using GO. You find the related code to my new blog post in that GitHub project.

The file to create the example tables and content is a part of the GitHub project called Multi-tenancy assets for IBM clients to build SaaS.

OBJECTIVE

Connect to a PostgresSQL database, create tables, insert data using a file from a GitHub project.

BASIC FLOW

Connect to the database
Get a file from a GitHub project with the SQL statements to create tables and insert data
Execute the SQL statement
Verify one value with a query

BASIC PROGRAMMING STEPS

Each step contains the link to the line in the source code inside the GitHub repository.

UNDERSTAND THE GITHUB URL FORMAT

As I said before, we use an example file from a GitHub project called Multi-tenancy assets for IBM clients to build SaaS. So, we need to get a bit familiar with the GitHub public APIs.

Example url we use: https://api.github.com/repos/IBM/multi-tenancy/contents/installapp/postgres-config/create-populate-tenant-a.sql

Mapping to the used GitHub API endpoint: https://api.github.com/repos/$NAME/$REPO/contents/$FILENAME

These are the related values of the example GitHub API endpoint above:

GitHub API: https://api.github.com/repos/
Name: “IBM/”
Repo: “multi-tenancy”
GitHub API: /contents/
Filename: “installapp/postgres-config/create-populate-tenant-a.sql”

For more details visit the GitHub public APIs documentation.

SOME USEFUL RESOURCES:

RUN THE EXAMPLE APPLICATION

These are the steps you need to follow to run the example on your local machine.

Note: You need a running PostgresSQL database somewhere

STEP 1: GIT CLONE

git clone https://github.com/thomassuedbroecker/go-access-postgres-example.git 
cd go-access-postgres-example

STEP 2: VERIFY THAT THE MOD FILE “GO.MOD”

cd gopostgressql 
ls

STEP 3: SET THE ENVIRONMENT VARIABLE

export DATABASE_URL= "postgres://username:password@localhost:5432/database_name"

Note: Don’t forget to insert your DATABASE_URL.

STEP 5: EXECUTE THE GO PROGRAM

go run .

SUMMARY

We touched in that simple example different essentials and useful topics in GO programming I would say (for example: base64, access HTTP endpoints, handle JSON and so on) and that makes it worth to take a note 😉

I hope this was useful for you and let’s see what’s next?

Greetings,

Thomas

Source: www.suedbroecker.net

go, #postgressql, #github, #buildlabs4saas

DEBUG A KUBERNETES OPERATOR WRITTEN IN GO

Thomas Südbröcker — Thu, 03 Mar 2022 15:48:13 +0000

In this blog post I want to share how to debug a GO Operator on your local machine on macOS. Adam de Leeuw and I verified it in different GO operator projects. Sometimes you find on Google information which uses the older Operator SDK. The following instructions worked for us in March 2022. 😉

The blog post is structured in four sections:

Prerequisites
Basics
Configure Visual Studio Code to debug the operator
Debug your GO operator

1. PREREQUISITES

You need to …

… ensure you have the Operator SDK installed (that also installs GO when you use brew for the installation)
… setup the Visual Studio Code extension for GO
… install DELVE debugger for GO
… setup the integration of the DELVE debugger for Visual Studio Code
… run an example GO operator 😉

2. BASICS

Please keep in mind some essentially basics when you run your operator locally with the make command:

make install run

That make install run command does …

… deploy the needed manifests (yaml's) to the Kubernetes cluster you use and create the connection.
… it starts the local GO application with the file called main.go (in the last step inside make file with the command go run ./main.go )

3. CONFIGURE VISUAL STUDIO CODE TO DEBUG THE OPERATOR

With that in mind we know how to debug.

First, you to run the make install run command to ensure you operator will find all need manifests configurations on your cluster.
Now you need to configure the DELVE GO debug in Visual Studio Code .vscode/launch.json configurations file in your workspace.

STEP 1: CREATE A `.VSCODE/LAUNCH.JSON` FILE IN VISUAL STUDIO CODE WITH A GO CONFIGURATION¶

The following short gif shows the creation of a .vscode/launch.json file. In that case it uses the folder where my workspace file is located and you see that go package was used as debug a configuration.

That’s the example result you saw in the gif.

"configurations": [   {      
  "name" :  "Launch Package" ,      
  "type" :  "go" ,      
  "request" :  "launch" ,      
  "mode" :  "auto" ,      
  "program" :  "${fileDirname}"    
} ]

STEP 2: NOW YOU CAN OPTIONAL CONFIGURE THE LOCATION WHERE YOU WANT TO START THE `MAIN.GO` FILE.

Just to ensure the right operator will be started. The gif below show how to customize the .vscode/launch.json file.

That’s the example result you saw in the gif.

{    // Use IntelliSense to learn about possible attributes.    
     // Hover to view descriptions of existing attributes.    
     // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387    "version" :  "0.2.0" ,    "configurations" : [      
  {         "name" :  "Launch Frontend Operator" ,        
                    "type" :  "go" ,        
            "request" :  "launch" ,
            "mode" :  "auto" ,
             "program" :  "/Users/thomassuedbroecker/Downloads/dev/multi-tenancy-frontend-operator/frontendOperator"      
} ] }

4. DEBUG YOUR GO OPERATOR

STEP 1: EXECUTE THE[ `MAKE INSTALL RUN`] FOR YOUR OPERATOR

You see the my example operator I used for my blog post and live stream Develop a simple operator to deploy a web application using the GO Operator SDK¶.

make install run

Example output

/Users/thomassuedbroecker/Downloads/dev/multi-tenancy-frontend-operator/frontendOperator/bin/controller-gen
rbac:roleName=manager-role crd webhook paths= "./..."` `output:crd:artifacts:config=config /crd/bases /Users/thomassuedbroecker/Downloads/dev/multi-tenancy-frontend-operator/frontendOperator/bin/kustomize` `build config /crd | kubectl apply -f - customresourcedefinition.apiextensions.k8s.io /tenancyfrontends .multitenancy.example.net configured /Users/thomassuedbroecker/Downloads/dev/multi-tenancy-frontend-operator/frontendOperator/bin/controller-gen` `object:headerFile= "hack/boilerplate.go.txt" 
paths= "./..." go  fmt` `./... go vet ./... go run . /main .go 1.6461312240985138e+09 INFO  controller-runtime.metrics   Metrics server is starting to listen   { "addr" :  ":8080" }
1.646131224098974e+09  INFO  setup  starting manager 1.646131224099315e+09  INFO  Starting server { "path" :  "/metrics" ,  "kind" :  "metrics" ,  "addr" :  "[::]:8080" }
1.646131224099315e+09  INFO  Starting server { "kind" :  "health probe" ,  "addr" :  "[::]:8081" }
1.6461312240993888e+09 INFO  controller.tenancyfrontend   Starting EventSource   { "reconciler group" :  "multitenancy.example.net" ,  "reconciler kind" :  "TenancyFrontend" ,  "source" :  "kind source: *v1alpha1.TenancyFrontend" }
1.6461312240994549e+09 INFO  controller.tenancyfrontend   Starting Controller    { "reconciler group" :  "multitenancy.example.net" ,  "reconciler kind" :  "TenancyFrontend" }
1.646131224200895e+09  INFO  controller.tenancyfrontend   Starting workers { "reconciler group" :  "multitenancy.example.net" ,  "reconciler kind" :  "TenancyFrontend" ,  "worker count" : 1}

STEP 2: STOP THE OPERATOR EXECUTION

STEP 3: ADD A BREAK POINT TO THE CONTROLLER AND START THE DEBUGGING FOR THE GO OPERATOR[¶]

The gif below shows how create a break point and start the debugging. You need to take a look in the debug console.

In the following image you see a screen shot of the running debugging.

Take a short look at

Run and debug
- Start the debug session with the just defined GO configuration for the Operator
Observe
- Variables
- Watch
- Stack
Debug console
- Show the output we would see, when we would have started operator with a make command

SUMMARY

It’s awesome to use the powerful DELVE debugger for GO inside Visual Studio Code when you build a GO Operator. 🙂

I hope this was useful for you and let’s see what’s next?

Greetings,

Thomas

Source: www.suedbroecker.net

operator, #go, #operatorsdk, #kubernetes, #delve, #buildlabs4saas

How to simply examine a JSON response from a Cloudant search in Java

Thomas Südbröcker — Thu, 01 Jul 2021 13:16:08 +0000

This is a short cheat sheet about, how to simply examine a JSON response from a Cloudant search in Java. I found different examples, but these examples were (more or less) older examples, where I missed some pieces and at the end for me the Java EE documentation was the best resource to realize it.

The JSON I wanted to examine, was a JSON with a nested JSON array and that array also contains a nested JSON.

JSON format of the Cloudant search response

Here you see is the JSON format I wanted to examine. As I said: It's a JSON with a nested JSON array and the array also contains a nested JSON.

{
  "total_rows": 1,
  "bookmark": "g2wAAAABaANkAClkYmNvcmVAZGI3LmJtLWNjLXVzLXNvdXRoLTExLmNsb3VkYW50Lm5ldGwAAAACYhAAAABiH____2poAkY_8AAAAAAAAGEAag",
  "rows": [
    {
      "fields": {
        "url": "https://haralduebele.github.io/2019/02/17/blue-cloud-mirror-dont-open-the-doors/",
        "customer": "your-blogs",
        "title": "Blue Cloud Mirror — (Don’t) Open The Doors!",
        "type": "articles",
        "authorName": "Harald Uebele"
      },
      "id": "7a5f27ce3ef66f895cb666e46ce45e55"
    }
  ]
}

This is the simply content I need from the JSON response of the Cloudant search:

AuthorName
Title
URL

Example in JavaScript format:

rows[0].fields.authorName
rows[0].fields.title
rows[0].fields.url

Java code and imports

Here is the Java code and the imports I used.

Imports

// JSON
import javax.json.Json;
import javax.json.JsonObject;
import javax.json.JsonArray;
import javax.json.JsonReader;
// Need to the string input
import java.io.StringReader;

Code

Steps:

Get Cloudant search response value
Create simple JSON object from search response value
Extract nested JSON array from simple JSON object
Get the first JSON object entry in the JSON array
Extract the nested JSON object from the first JSON object entry
Extract the relevant data from the nested JSON object.

// Cloudant search response value
System.out.println("-->log: search_response.toString: " + search_response.toString());

// Create simple json object from response value
JsonReader jsonReader = Json.createReader(new StringReader(search_response.toString()));
JsonObject object = jsonReader.readObject();
jsonReader.close();

// Extract nested json array from simple json object 
JsonArray rows = object.getJsonArray("rows");
System.out.println("-->log: rows: " + rows);

// Get the first json object entry in the json array 
JsonObject row_object = rows.getJsonObject(0);
System.out.println("-->log: row_object: " + row_object);

// Extract the nested json object from the first json object entry
JsonObject fields = row_object.getJsonObject("fields");
System.out.println("-->log:  fields: " + fields);

// Extract the relevant data from the nested json object
String url = fields.getString("url");
String authorName = fields.getString("authorName");
String title = fields.getString("title");
System.out.println("-->log: Author : " + authorName + " Title: " + " url: " + url);

Console output for the example

This is an example console output of the running application.

-->log: search_response.toString: {
  "total_rows": 1,
  "bookmark": "g2wAAAABaANkAClkYmNvcmVAZGI3LmJtLWNjLXVzLXNvdXRoLTExLmNsb3VkYW50Lm5ldGwAAAACYhAAAABiH____2poAkY_8AAAAAAAAGEAag",
  "rows": [
    {
      "fields": {
        "url": "https://haralduebele.github.io/2019/02/17/blue-cloud-mirror-dont-open-the-doors/",
        "customer": "your-blogs",
        "title": "Blue Cloud Mirror — (Don’t) Open The Doors!",
        "type": "articles",
        "authorName": "Harald Uebele"
      },
      "id": "7a5f27ce3ef66f895cb666e46ce45e55"
    }
  ]
}
-->log: rows: [{"fields":{"url":"https://haralduebele.github.io/2019/02/17/blue-cloud-mirror-dont-open-the-doors/","customer":"your-blogs","title":"Blue Cloud Mirror — (Don’t) Open The Doors!","type":"articles","authorName":"Harald Uebele"},"id":"7a5f27ce3ef66f895cb666e46ce45e55"}]
-->log: row_object: {"fields":{"url":"https://haralduebele.github.io/2019/02/17/blue-cloud-mirror-dont-open-the-doors/","customer":"your-blogs","title":"Blue Cloud Mirror — (Don’t) Open The Doors!","type":"articles","authorName":"Harald Uebele"},"id":"7a5f27ce3ef66f895cb666e46ce45e55"}
-->log:  fields: {"url":"https://haralduebele.github.io/2019/02/17/blue-cloud-mirror-dont-open-the-doors/","customer":"your-blogs","title":"Blue Cloud Mirror — (Don’t) Open The Doors!","type":"articles","authorName":"Harald Uebele"}
-->log: Author : Harald Uebele Title:  url: https://haralduebele.github.io/2019/02/17/blue-cloud-mirror-dont-open-the-doors/

Additional useful links :

Here are some additional links, which can give you an additional point of view.

Summary

Finally it's pretty easy to handle JSON in Java. I hope this was useful for you and let’s see what’s next?

Greetings,

Thomas

Run a Docker image as a Cloud Foundry App on IBM Cloud

Thomas Südbröcker — Wed, 13 May 2020 07:45:21 +0000

In that blog post I want to point out an awesome topic: “Run a Docker container image as a Cloud Foundry App on IBM Cloud”

Rainer Hochecker, Simon Moser and I had an interesting exchange about running a Docker image as a Cloud Foundry App on IBM Cloud.

The advantage with that approach is: you don’t need to instantiate a Kubernetes or OpenShift cluster. You can just run a single Docker image with your single application on IBM Cloud. That can be useful in different situations where you need to control the contents of your application, and the cloud foundry build-pack mechanism maybe restricts you.

IBM offers to run Cloud Foundry Apps on IBM Cloud and supports a set of build packs. But, by the fact IBM uses Cloud Foundry, you can also upload a Docker image as a Cloud Foundry application, it’s an officially supported feature. Yes there is no documentation related to that topic in the IBM Cloud documentation, but you can apply the Cloud Foundry documentation.

By the fact IBM uses , you can also try to upload a Docker image as a Cloud Foundry application, currenlty there is no restriction, but it's not offically supported. You not will find any documentation for that on IBM Cloud, but as IBM Cloud uses Cloud Foundry you can apply topics from the Cloud Foundry documentation.

One impact of that situation is, you don’t see the VCAP variables and you can’t use the out of the box binding for IBM Cloud services. You have to manage the bindings to your IBM Cloud services by yourself.

LET’S START WITH A SHORT GUIDE: HOW TO SETUP A CLOUD FOUNDRY APPLICATION USING A DOCKER IMAGE

When you follow the steps, you need to replace the name for the domain, port or hostname with your own settings. We will use in that example the [IBM Cloud Shell(https://cloud.ibm.com/docs/cloud-shell?topic=cloud-shell-getting-started)] on IBM Cloud and Node-RED as our Docker image.

The following images shows a simplified overview.

We push the Docker image and run it as a container in a Cloud Foundry app
We define a route with a port mapping to access the application from the internet.

STEP 1: LOGON TO IBM CLOUD AND OPEN THE IBM CLOUD SHELL.

The gif shows how to access the IBM Cloud Shell from the IBM Cloud UI.

STEP 2: CHANGE TO THE IBM CLOUD UI TO CREATE A CLOUD FOUNDRY SPACE IN YOUR REGION, IF YOU DON’T HAVE ONE.

The gif shows how to create a new space dev, in the existing organization thomas.suedbroecker , in the region Germany.

STEP 3: GO BACK TO THE IBM CLOUD SHELL AND SET THE CLOUD FOUNDRY ENDPOINT, ORGANIZATION AND SPACE. I MY CASE I USED FOLLOWING VALUES.

Endpoint: api.eu-de.cf.cloud.ibm.com
Organization: thomas.suedbroecker
Space: dev
Resource group: Default

ibmcloud target --cf-api api.eu-de.cf.cloud.ibm.com -o thomas.suedbroecker -s dev -g Default

STEP 4: VERIFY YOUR CLOUD FOUNDRY SETTINGS ON IBM CLOUD (IBM CLOUD DOCUMENTATION)

ibmcloud target

STEP 5: PUSH A DOCKER IMAGE FROM A CONTAINER REGISTRY.

Name: node-red
Image: node-red-docker:v10

In the IBM Cloud Shell you need to install the Cloud Foundry API with ibmcloud cf install and verify the API version ibmcloud cf -v.

ibmcloud cf push node-red --docker-image=docker.io/nodered/node-red-docker:v10  --no-start --no-route

Note: Of course you can also use a private container image registry like the IBM Cloud Image Container Registry , if you want.

In this case, you’d need understand the Cloud Foundry documentation:

CF_DOCKER_PASSWORD=YOUR-PASSWORD cf push APP-NAME --docker-image REPO/IMAGE:TAG --docker-username USER

The IBM Cloud Image Container Registry also contains the documentation how to do this for Cloud Foundry, here is the command.

export CF_DOCKER_PASSWORD=<apikey>
ibmcloud cf push appname -o <region>.icr.io/<namespace>/<image_repo> --docker-username iamapikey

STEP 6: GET THE GUID FROM THE CLOUD FOUNDRY APP INSTANCE. `A_GUID=APPLICATION GUID`

A_GUID=$(ibmcloud cf app node-red --guid|awk '/[0-9]/{print $1}')
echo $A_GUID

STEP 7: SET THE PORT TO ACCESS THE APPLICATION INSIDE THE DOCKER CONTAINER OF THE CLOUD FOUNDRY APPLICATION.

The port information you get from the Docker image description.

PORT: 1880

ibmcloud cf curl /v2/apps/$A_GUID -X PUT -d '{"ports": [1880]}'

STEP 8: IN THIS STEP WE CREATE A ROUTE TO THE CLOUD FOUNDRY APP, TO ACCESS LATER THE RUNNING APPLICATION FROM THE INTERNET.

The Domain name eu-de.mybluemix.net depends on the region you create the Cloud Foundry App. In that sample we using eu-de. (see IBM Cloud documentation )

Domain: eu-de.mybluemix.net
Hostname: node-red-tsuedbroecker

ibmcloud cf create-route dev eu-de.mybluemix.net --hostname node-red-tsuedbroecker

STEP 9: WE NEED TO MAP THE ROUTE TO THE CLOUD FOUNDRY APP , TO ACCESS THE RUNNING APPLICATION FROM THE INTERNET.

Application name: node-red
Domain: eu-de.mybluemix.net
Hostname: node-red-tsuedbroecker

ibmcloud cf map-route node-red eu-de.mybluemix.net --hostname node-red-tsuedbroecker

Now we extract the GUID for the created route. (R_GUID=route GUID)

R_GUID=$(ibmcloud cf curl "/v2/routes?q=host:node-red-tsuedbroecker" | sed -n 's|.*"guid": "\([^"]*\)".*|\1|p')
echo $R_GUID

STEP 10: NOW WE EXTRACT THE GUID FOR THE NEWLY CREATED ROUTE. (`R_GUID=ROUTE GUID`)

ibmcloud cf curl /v2/route_mappings -X POST -d '{"app_guid": "'"$A_GUID"'", "route_guid": "'"$R_GUID"'", "app_port": 1880}'

STEP 11: WE NEED TO UPDATE THE ROUTE MAPPINGS WITH OUR GUID’S. (`R_GUID` AND `A_GUID`)

ibmcloud cf curl /v2/route_mappings -X POST -d '{"app_guid": "'"$A_GUID"'", "route_guid": "'"$R_GUID"'", "app_port": 1880}'

STEP 12: START THE CLOUD FOUNDRY APPLICATION.

ibmcloud cf start node-red

STEP 13: VISIT THE CLOUD FOUNDRY APP IN THE IBM CLOUD UI AND INSPECT THE POSSIBILITIES.

In the gif you see, there is no build pack information and there are no environment variables.

STEP 14: NOW OPEN THE APPLICATION URL AND USE THE RUNNING NODE-RED INSTANCE.

The gif shows, how to access the application URL inside the Cloud Foundry App.

I hope this was useful for you and let’s see what’s next?

Greetings,

Thomas

A Node-RED Twitter-follower-flow

Thomas Südbröcker — Thu, 26 Mar 2020 09:18:46 +0000

In my last blog post I did an introduction "how use the starter kit for Node-RED on IBM Cloud" and in that new blog post I want to highlight the Node-RED Twitter-follower-flow I developed as an introduction to different topics I frequently face at hackathons. Here is the link to the "Twitter-follower-flow" GitHub project.

Node-RED is very good for prototyping, that is the reason why it is often used in hackathons. If you are new to Node-RED and you start to develop a Node-RED flow, you normally have following challenges:

How to ...

... define own REST endpoints to encapsulate an external API?
... automate the authentication to that external API?
... extract data from the external API?
... customize data and CRUD with databases?

The Node-RED flow of that project has the objective to provide an (little advanced) introduction to the first three topics above.
The CRUD (Create, Read, Update, Delete) topic is not covered here. Visit that CRUD example for more information.

The "Twitter-follower-flow" example ...
- ... uses open technologies (Node-RED is a Project of the OpenJS foundation
- ... creates no additional costs
- Uses a free runtime on IBM Cloud
- Uses the free Twitter developer API
- ... has following technical level:
- Beginner to intermediate
- Needs a basic knowledge of JavaScript and REST
- ... takes 30 - 45 min to setup the example from scratch
- Register on IBM Cloud
- Create a Node-RED instance on IBM Cloud
- Register at Twitter for a developer API Account
- Copy the existing Node-RED flow
- Configure the flow
- Run the flow

The YouTube video gives a 13 min introduction to the Twitter-follower-flow.

1. The UseCase

The UseCase "Extract the twitter follower list" addresses three topics listed above in the Node-RED Twitter-follower-flow.

Define own REST endpoints to encapsulate Twitter developer API calls.
Automate the authentication the two step authentication of the Twitter developer API.
Extract data from the Twitter follower list and build an own list.

2. Topics related to the Node-RED usage

The Twitter-follower-flow is an concrete example of following topics ...

... the implementation of Node-RED REST endpoints.

Create a very basic authentication to protect each Node-RED endpoint
Realize the two step authentication (also known as two-factor authentication) to access the Twitter developer API: 1. Basic authentication with a key and secret to request a bearer token from Twitter 2. Use the bearer token for authentication to use the Twitter API
Get the follower list from Twitter using the bearer token

... the usage of the implemented REST endpoints to automate following sequence to get the Twitter follower.

Get the authorization bearer token from Twitter to access the Twitter API
Get all followers from Twitter using a bearer token ( the challenge is: how implement the paging thought the followers list from Twitter )
Extract the names of the follower from the follower list

... the work with following Nodes in the Node-RED Twitter-follower-flow.

Function (Do small programming in javascript.)
Inject (Start a flow in Node-RED.)
HTTP in (Creates an HTTP end-point for creating web services.)
HTTP request (Sends HTTP requests and returns the response.)
HTTP response (Sends responses back to requests received from an HTTP Input node.)
Base64 (A function that converts the chosen property (default msg.payload) to and from base64 format.)
JSON (Converts between a JSON string and its JavaScript object representation, in either direction.)
Switch (Define a decision how to route the payload. Route messages based on their property values or sequence position.)

... the usage of flow variables to exchange values between functions in the Twitter-follower-flow.

3. Setup the Twitter-follower-flow example

That section contains six major steps to setup the Twitter-follower-flow example on IBM Cloud.

Setup Twitter developer application
Create a Node-RED instance on IBM Cloud
Install a additional Node to the Node-RED instance
Import the Node-RED flow
Configure the Node-RED flow
Introduction to the Node-RED flow (13 min video)

In the image you see the full flow.

Step 1: Setup Twitter developer application

You need a Twitter account and a Twitter developer application.
With your Twitter account can register for the Twitter developer API and create a Twitter developer application. That application provides the needed credentials to access the Twitter API, which is used in that example.

a. Link to add a Twitter developer application.

Here is a blog post with an example how to setup a Twitter application: How to Register a Twitter App in 8 Easy Steps

Here is a preview, how to get the credentials from your Twitter developer application:

b. Twitter API documentation `get-followers-list`

Step 2: Create a Node-RED instance on IBM Cloud

We use a Node-RED instance on IBM Cloud with an IBM Lite Account.

Create an IBM Cloud Lite Account just by register here.
Follow the steps in my blog post to setup a Node-RED instance on IBM Cloud

Step 3: Install a additional Node to the Node-RED instance

Install the node-red-node-base64 Node to the Node-RED instance. The gif shows the installation.

Step 4: Import the Node-RED flow

Import the flow from the flow.json file in flows folder of that project. The gif below shows how to import the flow in the Node-RED.

Step 5: Configure the Node-RED flow

We need to configure ...

... the Twitter authentication
... the Twitter username
... the Node-RED URL

a. Set Twitter API key and secret

Insert the values for the user and secret of the Twitter API credentials in the function set user and secret. The image shows an example, where you get the credentials information.

The mapping for the Node-RED flow:

user = Twitter API key
secret = Twitter API secret key
nodereduser = Your own definition to secure the Node-RED REST Endpoints
noderedpassword = Your own definition to secure the Node-RED REST Endpoints

Here is the source code for the set user and secret function.

var user = "USER";
var secret = "SECRET"
var nodereduser = "admin";
var noderedpassword = "notreallysecure"

flow.set("nodereduser", nodereduser);
flow.set("noderedpassword", noderedpassword);

msg.payload = user + ":" + secret;

return msg;

b. Set Twitter username

Define "username display name" you want get the follower list from, by inserting
in function set_basic_auth the "YOUR_TWITTER_DISPLAY_NAME".

// Set basic auth
flow.set("auth", msg.payload);
msg.payload = flow.get("auth");

// Init parameters:
// For more information visit get-followers-list:
// https://developer.twitter.com/en/docs/accounts-and-users/follow-search-get-users/api-reference/get-followers-list
flow.set("twitterCursor", "-1");
flow.set("twitterResultCount", "50");
flow.set("twitterSN", "YOUR_TWITTER_DISPLAY_NAME");

msg.nodereduser = flow.get("nodereduser");
msg.noderedpassword = flow.get("noderedpassword");

return msg;

c. Set Node-RED URL

Configure the HTTP request nodes. Replace the https://node-red-my-hackathon.mybluemix.net URL with your URL in each HTTP request node:

getTwitterFollower-Indicrect
getTwitterToken-Indicrect

The image shows how you enter the URL in the HTTP requests node getTwitterFollower-Indicrect.

That image shows HTTP requests nodes to be change.

Step 6: Introduction to the Node-RED flow

The YouTube video gives a 13 min introduction to the Twitter-follower-flow.

I hope this was useful for you and let’s see what’s next?

Greetings,

Thomas

Blog post also at www.suedbroecker.net

Write and execute a JUnit test for a Java microservice based on MicroProfile and run both in the OpenLiberty development mode

Thomas Südbröcker — Tue, 17 Mar 2020 17:28:02 +0000

That post has the focus on: how to develop a JUnit test for the Authors microservice from the Cloud Native Starter example and run both the Authors microservice and the JUnit test on OpenLiberty in the development mode.

That post hasn’t the objective to be a blueprint or a ‘how to guide’ for writing JUnit tests, JUnit test organization, test strategy and so on. The objective is to get technically started along one concrete microservice example from the Cloud Native Starter project.

The Authors microservice has one RESTful api endpoint called getAuthor. The endpoint provides one parameter for the Author name. The endpoint returns Author data in a JSON format.

Keep the end in mind: The gif shows a sample JUnit test execution for the Author microservice using OpenLiberty in the Visual Studio Code editor:

Let's start with: What do we need and how do we realize the implementation?

We need to ..
- invoke the REST endpoint of the Authors microservice with a REST Client. * transform the JSON response of the REST endpoint to an Author data class * handle different values to invoke the REST Endpoint parameter for the Author name to run tests with variations of the Author name.
- compare the actual response value with an expected value and document the result.
And how to ..
- setup a JUnit test for the development mode of the OpenLiberty server?
- convert JSON Data from a String to an Author Java instance with JSON-B?
- define a REST Client?
- configure a parameterized JUnit test?
- write the concrete parameterized JUnit test?
- execute the JUnit test?
- find the test results?

What are the tools and frameworks?

That are the Tools and frameworks of the sample project:
- IDE: Visual Studio Code with the Java Development Extension
- Server: Open Liberty
- Frameworks: Eclipse MicroProfile
- Java project organization: Apache Maven

1. How to setup a JUnit test for the development mode of the OpenLiberty server?

To setup and run JUnit tests with the OpenLiberty server in the development mode, we have to provide a test folder in the src folder of our Java project. The image below shows the folders of my sample project.

Content of the `com.ibm.authors` package for the Authors microservice

AuthorsApplication class represents the JAX-RS RESTful web application.
Author class represents the data structure we use for the Author.
GetAuthor class represents the REST API Endpoint.
HealthEndpoint class represents the support of readiness probes for Kubernetes.

Content of the `authortests` package for the JUnit test

AuthorJsonbAdapter class represents JSON-B adapter for a JSON-B mapping configuration.
AuthorTestClient class represents the REST Client of the Authors microservice.
Test_GetAuthors class represents the JUnit test which will be executed as the parameterized test run.

1.2 The `pom.xml` configuration for the JUnit and OpenLiberty

Dependencies

The junit-jupiter-api and the junit-jupiter-engine are the basics for the Unit tests. With the junit-jupiter-params depencency we can define later a parameterized test.

<!-- JUnit Test --> 
<dependency> 
 <groupId>org.junit.jupiter</groupId> 
 <artifactId>junit-jupiter-api</artifactId> 
 <version>5.6.0</version> 
 <scope>test</scope> 
</dependency> 
<dependency> 
 <groupId>org.junit.jupiter</groupId> 
 <artifactId>junit-jupiter-engine</artifactId> 
 <version>5.6.0</version> <scope>test</scope> 
</dependency> 
<dependency> 
 <groupId>org.junit.jupiter</groupId> 
 <artifactId>junit-jupiter-params</artifactId> 
 <version>5.6.0</version> 
 <scope>test</scope> 
</dependency> 
<!-- JUnit Test -->

Build plugins
- JUnit
- maven-surefire-plugin
- maven-failsafe-plugin

<!-- JUNIT --> 
<plugin> 
 <artifactId>maven-surefire-plugin</artifactId> 
 <version>2.22.2</version> 
</plugin> 
<plugin> 
 <artifactId>maven-failsafe-plugin</artifactId> 
 <version>2.22.2</version> 
</plugin> 
<!-- JUNIT -->

OpenLiberty
- liberty-maven-plugin

<!-- Enable liberty-maven plugin --> 
<plugin> 
 <groupId>io.openliberty.tools</groupId> 
 <artifactId>liberty-maven-plugin</artifactId> 
 <version>3.1</version> 
 <!-- libertyMavenConfiguration --> 
 <configuration> 
  <serverName>authorsDevJUnitServer</serverName> 
  <configFile>liberty/server.xml</configFile> 
 </configuration> 
 <!-- libertyMavenConfiguration --> 
</plugin> 
<!-- Enable liberty-maven-plugin -->

2. How to convert JSON Data from a String to an Author Java instance with JSON-B?

The response of our endpoint getauthor is a text in a JSON format, but we want use that data in an instance of an Author class.

In JSON-B we define a JsonbAdapter which does the conversion from JSON to an Author class instance. Therefor we @Override the operations adaptToJson and adaptFromJson. The operation adaptFromJson defines how to create an Author object from a JsonObject.

// Author data class 
import com.ibm.authors.Author; 
// JSON-binding 
import javax.json.bind.adapter.JsonbAdapter; 
import javax.json.JsonObject; 
import javax.json.Json; 

public class AuthorJsonbAdapter implements JsonbAdapter<Author, JsonObject> 
{ ... 
  @Override 
  public Author adaptFromJson(final JsonObject jsonObject) throws Exception { 
   final Author author = new Author()
   author.setBlog(jsonObject.getString("blog")); 
   author.setName(jsonObject.getString("name")); 
   author.setTwitter(jsonObject.getString("twitter")); 
   return author; 
  } 
}

In the following code of the Test_GetAuthors class you see how to utilize the AuthorJsonbAdapter to create a new JSON-B configuration. That JSON-B configuration is used to create a JSON-B object. The JSON-B object contains the implemented operation fromJson and knows how to create an instance of an Author class.

final JsonbConfig config = new JsonbConfig().withAdapters(new AuthorJsonbAdapter()); 
final Jsonb jsonb = JsonbBuilder.create(config); 
final Author author_json = jsonb.fromJson(response, Author.class);

3. How to define a REST Client?

The following code shows the interface class AuthorTestClient. That class contains the REST Client interface definition for the REST Endpoint of the Authors microservice. With the usage of the MicroProfile annotation @RegisterRestClient a RESTful Client will be created, when the interface is used in the JUnit test.

The expected return value of getAuthors response type is defined as a String.

import javax.ws.rs.Path; 
import javax.ws.rs.GET; 
import javax.ws.rs.Produces; 
import javax.ws.rs.core.MediaType; 
import javax.ws.rs.QueryParam; 
import org.eclipse.microprofile.rest.client.inject.RegisterRestClient; @Path("/getauthor") 
@RegisterRestClient 
public interface AuthorTestClient { 
  @GET 
  @Produces(MediaType.APPLICATION_JSON) 
  public String getAuthor(@QueryParam("name") String name); 
}

4. How to configure the parameterized JUnit test?

The class Test_GetAuthors implements the JUnit test in the operation testGetAuthor. The test is defined as a ParameterizedTest and can be repeated with given values from a CsvSource.

ParameterizedTest

Here you see the annotation @ParameterizedTest and the configuration of name.
The name contains the count of the parameters, that test has two parameters and can be repeated.

@ParameterizedTest(name = "{index} => name=''{0},{1}''")

The concrete test implementation itself happens in the operation testGetAuthor. The operation contains the names for the parameters which were defined before. These parameters we will use in the test implementation.

public void testGetAuthor( final String authorName, final String expectedResult)

For more details visit the JUnit documentation.

CsvSource

The annotation @CsvSource contains a comma separated list of values for the test execution. The values are in order to fit to parameters nameAuthor (sample value: Thomas) and expectedResult (sample value: Thomas Suedbroecker).

@CsvSource({"Thomas,Thomas Suedbroecker", 
            "Niklas,Niklas Heidloff", 
            "Michael,Michael Heinrich" })

For more details visit the JUnit documentation

5. How to write the concrete parameterized JUnit test?

Step 1: Create a REST Client

To invoke our REST Endpoint getauthor of the Authors microservice we use the RestClientBuilder from MicroProfile to create our REST Client.

We use our defined MicroProfile RestClient Interface AuthorTestClient.class and the RestClientBuilder will create for us an object instance for the AuthorTestClient.

final AuthorTestClient authorClient = RestClientBuilder
 .newBuilder() .baseUri(baseURI) 
 .build(AuthorTestClient.class);

Step 2: Invoke the REST Client

Now we invoke the REST Client and we use our test parameter nameAuthor as input.

final String response = authorClient.getAuthor(nameAuthor);

Step 3: Convert the response to a Author data object

Once more the usage of JSON-B.

final JsonbConfig config = new JsonbConfig().withAdapters(new AuthorJsonbAdapter()); 
final Jsonb jsonb = JsonbBuilder.create(config); 
final Author author_json = jsonb.fromJson(response, Author.class);

Step 4: Compare the actual value of response with the expected value from the test parameter

To compare the actual and expected value we use the assertEquals from JUnit.

assertEquals(expectedResult, author_json.getName());

If a the values aren't equal, assertEquals throws a AssertionFailedError exception and document the error in target/surefire-reports/TEST-authortests.Test_GetAuthors.txt file.

Here is an sample output of the TEST-authortests.Test_GetAuthors.txt file:

org.opentest4j.AssertionFailedError: expected: <Michael Heinrich> but was: <Niklas Heidloff> at authortests.Test_GetAuthors.testGetAuthor(Test_GetAuthors.java:60)

For more details see in the JUnit documentation

6. How to execute the JUnit test?

With the usage of the liberty-maven-plugin we can start the OpenLiberty server in the development mode with following Maven command:

$ mvn liberty:dev

Then we just press return.

$ [INFO] Press the Enter key to run tests on demand.

7. How to find the test results?

The test results are stored in the target/surefire-reports/* folder.

Here are additional useful blog posts, videos or manuals

MicroProfile RestClient

Overview MicroProfile REST Client (Tomitribe)

JUnit

Jsonb

I hope this was useful for you and let’s see what’s next?

Greetings,

Thomas

PS: My GitHub project with the source code is here.
Resource on www.suedbroecker.net

A short introduction of the Node-RED Starter kit on IBM Cloud for Hackathons

Thomas Südbröcker — Tue, 17 Mar 2020 15:06:30 +0000

In that post I want to highlight the new way of the instantiation of a Node-RED application on IBM Cloud with the Node-RED Starter kit. From my perspective Node-RED is a very helpful tool at hackathons. (just take a look in my blog post How to prepare for a Hackathon with IBM Cloud?)

The Node-RED instantiation has changed and with the Node-RED Starter kit we can take the advantage of the capabilities of IBM Cloud to control the build, deployment and execution of the Node-RED application and we setup automatically all development tools to work effectively in a small development team in our hackathon.

I want to provide a short walkthrough from my perspective and created a 13 min YouTube video in addition to that blog post.(just take a look in my blog post How to prepare for a Hackathon with IBM Cloud?)

Let's get started with the description of the Node-RED Starter kit in IBM Cloud.

“This starter kit provides a pre-configured Node-RED application, including a Cloudant service to store the application flow configuration. Add services, generate and download the code, use the IBM Cloud Developer Tools CLI to run and debug locally, then deploy to Cloud Foundry or a DevOps Pipeline.“

With that Node-RED Starter kit we setup all needed development tools to work effectively in a small team at a hackathon.

The Node-RED Starter kit will create six new service instances in IBM Cloud as you see in the dashboard below.

What are the created resources in the image above:

Cloud Foundry apps: A Cloud Foundry Node-RED application instance.
Cloud Foundry services: A alias for the Cloudant service to bind that service to the Cloud Foundry Node-RED application. You can find more about the alias topic in my blog post: How to use VCAP with a IAM enabled service in IBM Cloud?
Services: Two created services:
- Cloudant service
- Continuous Delivery service
Apps: The App, from my point of view the “App” is technically, a single point of entry to all resources we usually need to build, deploy and run an application on IBM Cloud ("here the App was pre-configured with the Node-RED Starter kit“).
Developer tools: The created toolchain.

Let's have a short walkthrough, how to use the Node-RED Starter kit to create the Node-RED application instance on IBM Cloud

Here are five major steps, I see, when we use the Node-RED Starter kit.

Step 1: First we will create an App. That App is the single point of entry to all relevant resources for build, deploy and run the Node-RED application instance. In the following image we see the fully deployed App, with links to the related resources as the services, the toolchain, the application URL and the source code repository.

Step 2: Then automatically a Cloudant service instance will be instantiated. That instance does contain a Cloudant database which saves configurations and flows and more of the Node-RED application. The image displays the noderedmyhackathon database in the Cloudant service instance.

Step 3: The toolchain will be automatically configured, based on configuration information we will provide. The image shows a example list of toolchains in Dallas.

Step 4: The toolchain will create a Continues Delivery service, that service will clone the source code for the Node-RED application from origin Node-RED IBM Cloud GitHub project to our GitLab project in IBM Cloud. We can use the source code to modify the Node-RED application instance later. The GitLab project is the input for the delivery pipeline which does the concrete deployment of the application. The picture displays the configured toolchain.

Step 5: The deployment pipeline now creates the Cloud Foundry Node-RED application and the needed Cloud Foundry alias to bind the Cloudant service to the Cloud Foundry Node-RED application. The image shows the deployment pipeline.

The gif below shows how to access the running Node-RED application from the App.

I hope this was useful for you and let’s see what’s next?

Greetings,

Thomas

Resource on www.suedbroecker.net

DEV Community: Thomas Südbröcker

SIMPLE NODE.JS SERVER EXAMPLE USING THE WATSON ASSISTANT API V2

CONNECT TO AN ASSISTANT IN THE RIGHT ENVIRONMENT

IMPLEMENT A SIMPLE ENDPOINT

USE THE EXAMPLE

STEP 1: CLONE THE PROJECT TO YOUR LOCAL COMPUTER

STEP 2: CONFIGURE THE ENVIRONMENT VARIABLES

STEP 3: RUN LOCAL ON PORT 3010

STEP 4: OPEN BROWSER AND INVOKE THE GET SESSION ENDPOINT

STEP 5: IMPORT THE POSTMAN COLLECTION INTO POSTMAN

STEP 6: AFTER THE IMPORT YOU HAVE A NEW COLLECTION INSIDE YOUR POSTMAN

STEP 7: SEND A MESSAGE

STEP 7.1: GET A SESSIONID

STEP 7.2: INSERT THE SESSION ID INTO THE BODY OF THE POST REQUEST USING THE SENDMESSAGE ENDPOINT

SUMMARY

watsonassistant, #nodejs, #ibmcloud, #rest, #javascript

RUN WATSON NLP FOR EMBED ON IBM CLOUD CODE ENGINE

1. SOME SIMPLIFIED TECHNICAL BASICS ABOUT IBM WATSON LIBRARIES FOR EMBED

2. SOME SCREEN SHOTS FROM A RUNNING EXAMPLE ON CODE ENGINE

3. EXECUTE THE EXAMPLE ON IBM CLOUD CODE ENGINE

STEP 1: CLONE THE EXAMPLE PROJECT TO YOUR LOCAL COMPUTER

STEP 2: SET NEEDED ENVIRONMENT VARIABLES IN THE .ENV FILE

STEP 3: EXECUTE THE RUN-WATSON-NLP-WITH-CODE-ENGINE.SH BASH SCRIPT

EXAMPLE OUTPUT AND THE LINKS TO THE RELATED FUNCTIONS IN THE BASH SCRIPT CODE

LINKS TO THE RELATED FUNCTIONS IN THE BASH SCRIPT CODE

EXAMPLE OUTPUT

4. SUMMARY

CUSTOM DOMAIN AND TLS CERTIFICATE FOR YOUR APPLICATION ON IBM CLOUD CODE ENGINE

1. USED SERVICES OR SOFTWARE

2. SIMPLIFIED ARCHITECTURE OVERVIEW

3. EXAMPLE SETUP

3.1 GET THE SOURCE CODE

STEP 1: CLONE THE PROJECT

STEP 2: SET THE NEEDED ENVIRONMENT VARIABLES IN THE .ENV FILE¶

3.2 REGISTER A DOMAIN

STEP 1: REGISTER DOMAIN AT IBM CLOUD

STEP 2: VERIFY YOUR DOMAIN IS READY TO USE

3.3 REQUEST/CREATE A TLS CERTIFICATE WITH LET’S ENCRYPT AND CERTBOT

STEP 1: INSTALL CERTBOT ON MACOS¶

STEP 2: EXECUTE TO CREATE-CERT.SH TO CREATE THE TLS CERTIFICATE

STEP 3: CONFIGURE THE DOMAIN MAPPING IN CODE ENGINE

STEP 4: WE NEED TO INSERT THE LINK FROM CODE ENGINE INTO THE CNAME

STEP 5: OPEN THE DOMAIN URL

4. SUMMARY

CONNECT TO A POSTGRESQL DATABASE, CREATE TABLES, INSERT DATA, AND USE A FILE FROM A GITHUB PROJECT WITH GO

OBJECTIVE

BASIC FLOW

BASIC PROGRAMMING STEPS

UNDERSTAND THE GITHUB URL FORMAT

SOME USEFUL RESOURCES:

RUN THE EXAMPLE APPLICATION

STEP 1: GIT CLONE

STEP 2: VERIFY THAT THE MOD FILE “GO.MOD”

STEP 3: SET THE ENVIRONMENT VARIABLE

STEP 5: EXECUTE THE GO PROGRAM

SUMMARY

go, #postgressql, #github, #buildlabs4saas

DEBUG A KUBERNETES OPERATOR WRITTEN IN GO

1. PREREQUISITES

2. BASICS

3. CONFIGURE VISUAL STUDIO CODE TO DEBUG THE OPERATOR

STEP 1: CREATE A .VSCODE/LAUNCH.JSON FILE IN VISUAL STUDIO CODE WITH A GO CONFIGURATION¶

STEP 2: NOW YOU CAN OPTIONAL CONFIGURE THE LOCATION WHERE YOU WANT TO START THE MAIN.GO FILE.

4. DEBUG YOUR GO OPERATOR

STEP 1: EXECUTE THE[ MAKE INSTALL RUN] FOR YOUR OPERATOR

STEP 2: STOP THE OPERATOR EXECUTION

STEP 3: ADD A BREAK POINT TO THE CONTROLLER AND START THE DEBUGGING FOR THE GO OPERATOR[¶]

SUMMARY

operator, #go, #operatorsdk, #kubernetes, #delve, #buildlabs4saas

How to simply examine a JSON response from a Cloudant search in Java

JSON format of the Cloudant search response

Java code and imports

Imports

Code

Console output for the example

Additional useful links :

Other useful links:

Summary

Run a Docker image as a Cloud Foundry App on IBM Cloud

LET’S START WITH A SHORT GUIDE: HOW TO SETUP A CLOUD FOUNDRY APPLICATION USING A DOCKER IMAGE

STEP 7.2: INSERT THE `SESSION ID` INTO THE BODY OF THE `POST` REQUEST USING THE `SENDMESSAGE ENDPOINT`

STEP 2: SET NEEDED ENVIRONMENT VARIABLES IN THE `.ENV` FILE

STEP 3: EXECUTE THE `RUN-WATSON-NLP-WITH-CODE-ENGINE.SH` BASH SCRIPT

STEP 2: SET THE NEEDED ENVIRONMENT VARIABLES IN THE `.ENV` FILE¶

3.3 REQUEST/CREATE A TLS CERTIFICATE WITH `LET’S ENCRYPT AND CERTBOT`

STEP 1: INSTALL CERTBOT ON MACOS ¶

STEP 4: WE NEED TO INSERT THE LINK FROM CODE ENGINE INTO THE `CNAME`

STEP 1: CREATE A `.VSCODE/LAUNCH.JSON` FILE IN VISUAL STUDIO CODE WITH A GO CONFIGURATION¶

STEP 2: NOW YOU CAN OPTIONAL CONFIGURE THE LOCATION WHERE YOU WANT TO START THE `MAIN.GO` FILE.

STEP 1: EXECUTE THE[ `MAKE INSTALL RUN`] FOR YOUR OPERATOR

STEP 6: GET THE GUID FROM THE CLOUD FOUNDRY APP INSTANCE. `A_GUID=APPLICATION GUID`

STEP 10: NOW WE EXTRACT THE GUID FOR THE NEWLY CREATED ROUTE. (`R_GUID=ROUTE GUID`)

STEP 11: WE NEED TO UPDATE THE ROUTE MAPPINGS WITH OUR GUID’S. (`R_GUID` AND `A_GUID`)

b. Twitter API documentation `get-followers-list`

Content of the `com.ibm.authors` package for the Authors microservice

Content of the `authortests` package for the JUnit test

1.2 The `pom.xml` configuration for the JUnit and OpenLiberty