DEV Community: swenhelge

Securely connecting to Solace Cloud from Boomi

swenhelge — Tue, 09 Feb 2021 17:25:28 +0000

So you followed the "Getting started with Boomi and Solace" tutorial or played with the "Solace PubSub+ Connector" in your Boomi account.
Everything is great - you can connect, produce and consume events using the PubSub+ cloud service.

(Note - for the rest of this article we assume you have a Connection to Solace defined in your Boomi workspace, e.g. because you followed the tutorial above)

But hang on - in the tutorial you used a plain TCP socket to connect to PubSub+. Wouldn't it be better to use TLS encryption?

That's when you look at your PubSub+ Service in Solace Cloud and discover there's a TLS encrypted endpoint - labelled "Secured SMF Host". This should be easy - just use the secure endpoint and you're done? Right?

Well it isn't otherwise I wouldn't have written this post.

Next you replaced the plain TCP connection URL with the secured URL (as I did in the screenshot below). Then you tested the connection again using the handy "Test Connection" button in the Boomi Connection Setup dialog.

Looks good, right?

Oh no, not if you are using a cloud hosted Atom!

As you can see the Connector tries to use the default Java trust store so it can verify the certificate presented by the PubSub+ service. And for good reason this access isn't allowed in the cloud.

How do we fix this?

we will need to obtain or create a suitable trust store
we will need to make the trust store available on the Atom
then tell the connector to use it.

Luckily we can do this. Let's look at these steps.

Obtain a suitable Trust Store

PubSub+ Cloud services use a certificate signed by Digicert. The standard Java trust store includes the Root CA certificate.
You have two options here:

the easy way
the hard way

Let's do easy first - as you can see the Connector attempts to use cacerts. This is included in all Java installations - chances are high you have a few copies on your hard disk already. So just do a search and grab one of these. You can also obtain a copy from the OpenJDK repository - e.g. here.

If you wanted to make the effort to create a trust store with only the Root CA required by the Connector to connect to Solace Cloud there's always the hard way.

First you need to download the root certificate from the Solace Cloud console in PEM format:

Java requires the trust store in a Java Keystore format. To create the trust store in the required format you can follow steps 1 to 6 in Converting PEM-format keys to JKS format.
In the following screenshots I used the easy way, just replace cacerts where ever you see it with the name of your trust store if you did it the hard way.

Uploading the Trust Store to Boomi Atom(s)

Now you have a trust store, how do you deploy it to your Boomi environment?
There are multiple folders that a Boomi process has access to.
From that list the work and the userlib location stood out to me. Any process can write into the work directory so that may be an option, but although it is marked as "Permanent" storage it reads like it's intended for temporary storage. The other folder - userlib - is used to store "Custom Libraries". These are a way to add additional Java libraries to a Boomi environment. It goes through the same packaging, versioning and deployment cycle as Boomi processes themselves.
Sounds like a good way to manage the rollout of our trust store.

The steps to do this are:

Upload the trust store to the Account Library in your Boomi account
Create and deploy a Custom Library -Verify the trust store was uploaded to your Atom(s)

I'll add some key screen shots below, detailed instructions can be found in the links.

Here's how you add the "Account Library". Note that you are only allowed to upload JAR files - you need to add the .jar extension to your trust store's file name. I have renamed mine to cacerts.jar

Then create the "Custom Library", here's a screen shot how you do that. Also the library I have created is in the background, note how I added the cacerts.jar in there:

If you prefer - when creating the custom library you can choose to associate it with the Connector by setting the library type to "Connector" and connector type to "PubSub+ Connector". This will put the trust store in a dedicated sub-directory of userlib

You then create a "Packaged Component" and deploy it to your environment, I'll skip this here and fast forward to the result, the library was applied to the Atom:

And here is what it looks like if you set the Custom Library type to "Connector", note the sub directory that was created in userlib.

Set Trust Store used by the Connector

The last step is to let the Connector know to use our trust store.
This is where the "Custom Properties" come into play. There is a property SSL_TRUST_STORE we can set and we point that to the file that was uploaded - remember it's in the userlib directory

If you selected "Connector" as Custom Library Type the trust store will be located in a sub directory of userlib - you can look up the location in the Atom Management - see the screen shot in the preceding section.

There's some information on these "Custom Properties" in the Connector's documentation and a guide on the properties you can use in the Solace API documentation.

Now try "Test Connection" again, select an Atom in the Environment that you applied the "Custom Library" to and ...

Integration/Functional Testing with JUnit and Solace PubSub+

swenhelge — Thu, 19 Nov 2020 10:36:19 +0000

When developing event driven Java services for Solace PubSub+, at some stage you want to move from unit testing Java code to testing services against a broker.

This repository demonstrates a base class for JUnit tests that launches a new PubSub+ instance in docker for use in tests.
It utilises testcontainers, manages the container life cycle manually as there is no testcontainers wrapper (yet) for PubSub+ and follows the singleton container pattern.

At the same time I put my original version of this together spierepf (Peter-Frank Spierenburg) created an example using JUnit 5, check it out here:
https://github.com/spierepf/solace-junit5-test

Port Mapping and Ports Exposed by Container

When the container is created the mapped ports are provided. This tells testcontainers which ports it shall expose for the container. testcontainers exposes random ports and maps these to the internal ports of the container.
For example - the base class maps port 55555 (SMF plain). This may be exposed as any port. The base class logs the SMF port used and output will look similar to this:

INFO: Started Solace PubSub+ Docker Container, available on host [localhost], SMF port [32872]

So an SMF client needs to connect on port 32872.

Base class - AbstractPubSubPlusTestCase

This class starts the docker container. It logs a message when the container before and after start up.
It exposes many of the plain services such as SMF, REST/HTTP, MQTT/TCP. The ports used by the container can be obtained from the class with getXxxxx methods.
The class also provides getAdminUser and getAdminPassword methods so SEMP v2 calls can be sent to configure the broker.

Example test - SolaceIntegrationTest

This is an example unit test, extends the AbstractPubSubPlusTestCase class.

It includes some test cases to demo how the base class can be used:

Publish message using SMF
Publish and subscribe to message using SMF
Publish using REST/HTTP and subscribe using SMF
Obtain a. list of message vpns form the broker using SEMPv2/HTTP

Event Mess to Event Mesh - Automating the Configuration of a Hybrid IoT Event Mesh with Ansible

swenhelge — Tue, 08 Sep 2020 13:40:54 +0000

In this part of the Solace and Ansible blog series, we will describe how to configure and reconfigure a complex Solace event mesh in an automated fashion with a repeatable, consistent approach. The use case discussed here is a globally distributed, hybrid IoT event mesh to set up for an international shipping company. The company wants to enable a digital shipping eco-system with the following examples:

Analyse tracking and quality data such as geo-location, cooling, and shock of all their customer’s shipments in real time to create an overall better customer experience
Include data streams from their own containers, 3rd party ships, ports, etc.
Provide selectively and securely tracking and quality data to regional partners – the freight forwarders – to better service their end-customers

Let’s dive into the details of the project. In our approach, we have separated the deployment concerns into 1) infrastructure, and 2) application-specific configuration so they can be managed and applied independently:

Infrastructure: the Event Mesh topology which consists of ungoverned DMR (dynamic message routing) links and governed links – static bridges – to public cloud providers
Application: The use case specific resources such as client usernames, queues as well as attaching subscriptions to bridges – governed links – where events are allowed to flow to brokers in the public cloud

The Ansible tasks described in the previous article give us the capability to safely configure both of these aspects.

Infrastructure – Event Mesh Topology

The topology we intend to set up is a fully connected, internal event mesh with governed connections to services hosted in Solace PubSub+ Event Broker: Cloud. This diagram illustrates our original test deployment of three on-premise nodes connected to two Solace Cloud Services.

As we want to grow the event mesh over time by adding additional nodes to the internal Mesh and bridges to cloud services, we are looking to create an Ansible playbook that can accommodate a growing inventory of brokers. Thus, the playbook can serve as a good starting point for your own deployment, possibly with only small adjustments.

The full topology playbook can be found on GitHub. It has two main sections: creating the static bridges (light green in the diagram above) and the DMR cluster and internal links (dark green in the diagram above).

Playbook to set up Event Mesh Topology: Static Bridgesby solace-iot-teamOpen on GitHub

When setting up the topology, we only set up the bridges but do not attach subscriptions, i.e. we create the link, but no events will initially flow. Static bridges are backed by queues for guaranteed message propagation between brokers. Below is the sequence of the tasks:

Remove and re-create queues required by bridges
Remove and re-create the bridges
Add connection to the remote Message VPN
Add trusted common names for TLS
Enable the bridges

This is applied to all brokers in the mesh, regardless of broker location as it links up the internal mesh brokers to the Solace Cloud services. There are two groups of hosts defined:

internal_dmr: on-premise brokers that form the full mesh
public_cloud: the Solace Cloud brokers

The code snippet below shows how the bridges are removed and re-created.

# remove existing bridges and re-create
- name: Remove Bridge
  loop: "{{ bridges|default([]) }}"
  solace_bridge:
    name: "{{ item.name }}"
    msg_vpn: "{{ msg_vpn }}"
    virtual_router: "{{ item.virtual_router }}"
    state: absent

- name: Create or update bridge
  loop: "{{ bridges|default([]) }}"
  solace_bridge:
    name: "{{ item.name }}"
    msg_vpn: "{{ msg_vpn }}"
    virtual_router: "{{ item.virtual_router }}"
    settings:
      enabled: false
      remoteAuthenticationBasicClientUsername: "{{item.remote_auth_basic_client_username}}"
      remoteAuthenticationBasicPassword: "{{ item.remote_auth_basic_password }}"
      remoteAuthenticationScheme: basic

Note that many parameters are set from variables defined in the inventory. We will look at the inventory further below.

The second section takes care of creating the Solace DMR cluster. Messages flow freely between the nodes as all brokers are connected via internal DMR links:

Remove and re-upload the trusted CA’s certificate (this is required to link brokers via TLS)
Remove the DMR cluster from all brokers and re-create
Add DMR links
Add connection to a remote broker to the links
Add trusted common names for TLS to the links
Enable the DMR links now that all configuration is done

This code snippet shows how the DMR cluster is added to all brokers in the internal mesh. Again, note that most information comes from the inventory. This playbook is only applied to the “internal_dmr” group of brokers:

- name: Add DMR Cluster
    solace_dmr:
      name: "{{dmr_internal_cluster}}"
      state: present
      settings:
        tlsServerCertEnforceTrustedCommonNameEnabled: "{{enforce_trusted_common_name}}"
        tlsServerCertValidateDateEnabled: false
        tlsServerCertMaxChainDepth: 6
        authenticationBasicPassword: "{{dmr_cluster_password}}"

  - name: Add DMR Link
    solace_link:
      name: "{{item.remote_node}}"
      dmr: "{{dmr_internal_cluster}}"
      state: present
      settings:
        enabled: false
        authenticationBasicPassword: "{{dmr_cluster_password}}"
        span: "{{dmr_span}}"
        initiator: "{{item.initiator}}"
        transportTlsEnabled: false
    loop: "{{ dmr_links|default([]) }}"

As mentioned above, the inventory defines two groups of brokers: internal_dmr and public_cloud. For each host/broker we define the queues, bridges, DMR links and so on that should be created.

Here’s an example of the configuration for an “internal_dmr” broker. See GitHub for the full inventory.

Internal DMRby solace-iot-teamOpen on GitHub

A couple of things to note:

A host contains two lists: “dmr_links” and “bridges”.
While in the Solace broker the queues associated to a queue are separate objects, we list the required queue(s) for a bridge underneath it.
For each DMR link we need to set the initiator of the underlying connection. We will discuss the implications further below.
The vars within this host group hold the common information for the DMR cluster such as the cluster name and password.
Reference back to the “Add DMR Cluster” and “Add DMR links” tasks in the snippet above
- The former uses the “vars” in the snippet below, for example, name: "{{dmr_internal_cluster}}"
- The latter loops through the DMR links defined for a host and uses information such as the connection initiator: loop: "{{ dmr_links|default([]) }}"

internal_dmr:
  hosts:
    macbook:
      ansible_connection: local
      msg_vpn: default
      host: localhost
      dmr_links:
      - name: internal-lenovo
        remote_node: v195858
        dmr_remote_address: 192.168.0.34:55555
        initiator: remote
        trusted_common_names:
        - name: "CA-CERT"
      bridges:
      - name: mac-partner-edge-01
        remote_auth_basic_client_username: solace-cloud-client
        remote_auth_basic_password: secret
        virtual_router: auto
        queues: 
        - name: mac-partner-edge-01_Queue
          owner: default
        remote_vpns: 
        - name: partner-edge-01
          location: mr2.messaging.solace.cloud:55555
          queue_binding: mac-partner-edge-01_Queue
          tls_enabled: false
        trusted_common_names:
          - name: "CA-CERT"
  vars:
    # use common connection parameters as we use the cloud API SEMP v2 proxy
    secure_connection: false
    port: 8080
    password: admin
    username: admin 
    dmr_internal_cluster: internal-cluster
    dmr_cluster_password: secret
    dmr_span: internal
    cert_content: |
        -----BEGIN CERTIFICATE-----
        -----END CERTIFICATE-----
    enforce_trusted_common_name: false

Why did we explicitly define the DMR links in the inventory? Arguably, we could write a playbook that simply links every broker to every other broker automatically. The reason is that we needed to plan out which node initiates the DMR connection – in our case some nodes cannot be reached from the outside due to security policies preventing incoming connections. For links between these nodes and others, the initiator of connections to other nodes in the cluster needs to be the “isolated” node. Configuring the event mesh may require knowledge of the network infrastructure and means you may need to plan and configure DMR links explicitly. Especially since there could be multiple nodes that have restricted inbound connectivity so you will need to figure out how to connect a pair of these brokers.

Now that we have the topology set up, let’s see how we can add the resources required by an application interacting with this topology.

Use Case-Specific Resources

An example similar to our actual use case is a global shipment company that tracks containers as they move across regions and continents. For internal processes the company needs to be able to track any shipment from anywhere in the world. The company offers partners to propagate all shipping and tracking events to a Solace Cloud service so they can process and analyse events in real time – but only the data that concerns shipments into or out of their region.

Tracking events are sent on a topic structure like shipment/from/US/to/UK/{shipment_id}/scan/{location}. Mapping this use case onto our topology will make it clearer what we need to configure:

We have multiple clients that require to connect to different parts of the mesh:

Global Ops – they can connect to any node on the internal mesh and receive all shipment tracking events.
US-Based Partner – it can only connect to the public cloud broker in the US. It should only be able to receive events concerning their region. We could enforce this via their Access Control List (ACL). As the service is connected to the internal mesh via a bridge, we can enforce this rule on the bridge itself and only relevant events are actually propagated to the broker servicing the partner.
EU-based partner – it can only connect to the EU broker. As above, we can enforce only France and UK relevant events ever are propagated to the partner service.
Tracking points within the shipment networks that emit the network events — they need to be able to connect to a specific node in the mesh and allowed to publish on any topic within the “shipment” namespace.

To enable this use-case we need to create

ACL profiles
Client Usernames
Bridge subscriptions

An example playbook to do exactly these steps can also be found in GitHub.

Shipment Setup Exampleby solace-iot-teamOpen on GitHub

Here are some examples from the playbook – as for the topology you can see we use a lot of variables form the inventory file so we can easily add additional client usernames, subscriptions, and so on:

- name: Remove Client Usernames before removing the ACL Profile
  solace_client:
    name: "{{ item.name }}"
    msg_vpn: "{{ msg_vpn }}"
    state: absent
  loop: "{{ users | default([]) }} "

- name: Add ACL Profile
  solace_acl_profile:
    name: "{{ item.name }}"
    msg_vpn: "{{ msg_vpn }}"
    settings:
      clientConnectDefaultAction: "{{ item.client_connect_default }}"
  loop: "{{ acls | default([]) }} "

- name: Add ACL Publish Exception
  solace_acl_publish_exception:
    name: "{{ item.1.topic }}"
    acl_profile_name: "{{ item.0.name }}"
    msg_vpn: "{{ msg_vpn }}"
    topic_syntax: "{{ item.1.syntax }}"
  with_subelements: 
    - "{{ acls | default([]) }} "
    - publish_topic_exceptions
    - flags:
      skip_missing: true

- name: Add Client
  solace_client:
    name: "{{ item.name }}"
    msg_vpn: "{{ msg_vpn }}"
    settings:
      clientProfileName: "{{ item.client_profile }}"
      aclProfileName: "{{ item.acl_profile }}"
      password: "{{ item.password }}"
      enabled: "{{ item.enabled }}"
  loop: "{{ users | default([]) }} "

- name: add subscriptions
  solace_bridge_remote_subscription:
    name: "{{ item.1.name }}"
    msg_vpn: "{{ msg_vpn }}"
    bridge_name: "{{ item.0.name }}"
    virtual_router: "{{ item.0.virtual_router }}"
    deliver_always: true
  with_subelements: 
    - "{{ bridges | default([]) }}"
    - remote_subscriptions
    - flags:
      skip_missing: true

Note that before removing an ACL, all client usernames using the ACL must be removed or updated to use a different ACL.

The full playbook contains some more tasks, however, it is a straightforward playbook whose results can be easily changed by adjusting the inventory. The inventory defines only one host group – “shipment_demo”. For each broker in the mesh (cloud and on premise), it specifies the configurations to make. The snippet below shows the configuration for one internal broker and one cloud service. You can find the full inventory in GitHub:

Local Shipment Setupby solace-iot-teamOpen on GitHub

# internal broker
macbook:
  ansible_connection: local
  secure_connection: false
  port: 8080
  host: 192.168.0.8
  password: admin
  username: admin 
  msg_vpn: default
  bridges:
  - name: mac-partner-edge-01
    virtual_router: auto
    guaranteed_remote_subscriptions: 
    - name: "shipment/from/UK/to/>"
      queue: mac-partner-edge-01_Queue
    - name: "shipment/from/FR/to/>"
      queue: mac-partner-edge-01_Queue
    - name: "shipment/from/*/to/UK/>"
      queue: mac-partner-edge-01_Queue
    - name: "shipment/from/*/to/FR/>"
      queue: mac-partner-edge-01_Queue
  acls: 
  - name: shipping_publisher
    client_connect_default: allow
    publish_topic_exceptions:
    - topic: shipment/>
      syntax: smf
  - name: global_shipping_ops
    client_connect_default: allow
    subscribe_topic_exceptions:
    - topic: shipment/>
      syntax: smf
  users:
  - name: publisher
    password: HLP2
    acl_profile: shipping_publisher
    client_profile: default
    enabled: true
  - name: global_ops_europe
    password: wzC3B
    acl_profile: global_shipping_ops
    client_profile: default
    enabled: true
# partner broker in Europe
partner_edge_01_na:
  ansible_connection: local
  secure_connection: true
  port: 943
  host: mr2.messaging.solace.cloud
  password: nrkijp
  username: partner-edge-01-admin
  msg_vpn: partner-edge-01
  acls: 
  - name: partner-subscriber
    client_connect_default: allow
    subscribe_topic_exceptions:
    - topic: "shipment/>"
      syntax: smf
  users:
  - name: demo
    password: HLP2
    acl_profile: partner-subscriber
    client_profile: default
    enabled: true

The first thing to note is that we attach guaranteed message subscriptions to the bridge configuration on the internal broker so messages for the EU partner can flow to the cloud service. We also declare ACLs for the internal broker so publishers of shipment events can connect and send events as well as consumers from the global Ops applications can connect and receive events. The allowed topic permissions are very wide allowing these clients to publish or respectively consume messages on a wide range of topics.

Having ensured relevant messages are propagated to the EU partner broker, we simply declare an ACL and a client username on this broker – we can even grant very permissive subscription privileges to the ACL as the bridge configuration gives us tight control over the events that are propagated to the partner broker.

Summary

In this article we have seen how we can use the solace-ansible tasks to completely automate the setup of a use case utilizing a hybrid event mesh. We have discussed why it makes sense to separate event mesh topology configuration from the specific configuration required to support a use case with all its connecting apps. Creating generic playbooks that are driven by the inventory allows us to easily amend the event mesh topology, de-provision and re-provision use case configurations and support different use cases by simply creating new inventory files.

Responsible for Solace PubSub+ DevOps? Ansible modules for PubSub+ may make your life easier …

swenhelge — Fri, 04 Sep 2020 09:15:21 +0000

**
Use Ansible to Automate the Configuration of Solace PubSub+ Event Broker**

Managing and configuring a number of Solace PubSub+ event brokers – or even a single one – in a consistent and repeatable manner is a crucial requirement for production deployments.

Though the PubSub+ Event Broker itself supports a number of configuration options such as a REST API, you still need to implement logic to manage a configuration, apply configurations consistently, and re-apply configurations in a safe manner. In addition, removing a configuration is just as important during the development lifecycle of your project. This is where tools such as Ansible come in useful.

For a few recent engagements we were looking into how we can address this and came across a number of open source projects set up by Solace colleagues and end-user, such as this one: Automating Solace configuration management using SEMP and Ansible. The most promising project we found was Mark Street’s ansible-solace. As he describes in his blog, the project was developed for in-house production use, so it was sufficiently battle hardened. The project defines a framework for creating Ansible tasks for the Solace PubSub+ SEMP API and in its initial version supported all the most important resources on the broker. We required additional resources for our projects such as Bridges, DMR Cluster configuration, and REST Delivery Points (RDP). Fortunately, adding tasks was pretty simple due to the foundation that Mark Street et al had built.

What’s the current state of ansible-solace?

It supports a majority of Message VPN and broker-level resources that are manageable via REST/SEMPv2.

Message VPN Level:

Message VPN
ACL Profile
Client Username
Client Profile
Queue and Queue Subscriptions
Topic Endpoint
Bridge
Rest Delivery Point

Broker Level:

Certificate Authority
DMR Bridge and Link

Note: Some of these resources are not available in Solace PubSub+ Event Broker: Cloud – such as Client Profile and the broker-level resources, so these respective tasks only work for self-managed deployments.

Getting Started with Ansible-Solace

The quickest way to get started is to head over to the ansible-solace-samples GitHub repository.

Sample projects using ansible-solaceby solace-iot-team
Follow the instructions in the README, clone it, and run the sample projects. Open on GitHub

If you want to look at the code, the development repository lives here: https://github.com/solace-iot-team/ansible-solace-modules.

Using the Ansible Tasks

All tasks have common parameters describing which broker to connect to via SEMPv2 Administrator API. The defaults are set to work against a local broker with the standard admin username and password – e.g. deployed in docker following one of Solace’s Docker Install Guides.

These are the connection parameters:

username: "{{ username }}"

password: "{{ password }}"

host: "{{ host }}"

port: "{{ port }}"

secure_connection: "{{ secure_connection }}"

timeout: 30

Username and password are of the management user, e.g. admin on a locally installed broker. Port is the management port – 8080 by default for plain connections or 943 for HTTPS in Solace PubSub+ Event Broker: Cloud. The secure connection parameter can be set to “True” to switch to the HTTPS protocol.

As usual, in Ansible you can use variables or facts to set these dynamically as required.

Here’s a basic example to create a VPN:

- name: Playbook to add a vpn named 'foo'
  hosts: localhost
  tasks:
  - name: Add 'foo' VPN
    solace_vpn:
      name: foo
      state: present # default state
      settings:
        enabled: true
        dmrEnabled: false
        eventTransactionCountThreshold:
          clearPercent: 66
          setPercent: 98

It is pretty straightforward, but there are a few things to note:

You can request the state of the resource – present or absent. The ansible tasks will choose the right SEMPv2 method (POST, PATCH, DELETE) accordingly.
We have omitted the connection parameters for better legibility.
The settings dictionary can pass in additional arguments to the POST/PATCH call.

The last point is important as all tasks have the settings argument and you can pass in any parameters that the underlying REST resource supports. It is helpful to have the SEMPv2 developer documentation ready when writing your playbooks.

Looking at the create Message VPN resource operation, we discover the settings in the example above and we can also find the other parameters we could set such as those enabling MQTT Connectivity for example.

If we wanted to update the message VPN created by the playbook above to enable plain text MQTT, the playbook looks like this:

- 
name: Playbook to configure plain text MQTT on a vpn named 'foo'
hosts: localhost
tasks:
  - name: Add 'foo' VPN
    solace_vpn:
      name: foo
      state: present 
      settings:
        serviceMqttPlainTextEnabled: true
        serviceMqttPlainTextListen

This playbook will preserve the state of the message VPN in all other aspects.

What if we wanted to delete the message VPN? The playbook looks like this:

- name: Playbook to delete a vpn named 'foo'
hosts: localhost
tasks:
  - name: Add 'foo' VPN
    solace_vpn:
      name: foo
      state: absent

Finally, let’s see how we can connect to a remote broker via HTTPS with a non-default password to create a Message VPN:

- 
  name: Playbook to add a vpn named 'foo'
  hosts: localhost
  tasks:
  - name: Add 'foo' VPN
    solace_vpn:
      password: "secret!"
      host: remote-broker.com
      port: 943
      secure_connection: true
      name: foo
      state: present # default state
      settings:
        enabled: true
        dmrEnabled: false
        eventTransactionCountThreshold:
          clearPercent: 66
          setPercent: 98

How to contribute?

If you find anything that’s missing or could be improved, we value all contributions. You can start by forking our repository and we will be happy to incorporate your pull requests. As stated above, there’s a base class that makes it really easy to add additional modules. The easiest way is to clone an existing module and adjust it as needed. Here’s a detailed guide on module development.

PubSub+ - Configuring DMR using SEMPv2

swenhelge — Tue, 21 Apr 2020 15:44:17 +0000

Dynamic Message Routing (DMR) is a mechanism in Solace PubSub+ brokers to create a network or mesh of brokers within which events can flow freely between brokers. It is used in scenarios where events need to be distributed across sites such as in hybrid or multi-cloud or for horizontal scalability when single broker limits are exceeded.

You can find more information on DMR on the Solace documentation site.

In this article I'll describe how to set up or join DMR clusters and establish links between brokers via the PubSub+ broker's REST based SEMPv2 API.
Two postman collections illustrate how to create and tear down a DMR connection.

Full documentation of the SEMPv2 API is available online. We will need to use the Config API - whose Swagger API doc you can find here.

We will look at the scenario of connecting brokers across sites to facilitate data movement e.g. between on-premise and cloud. As a prerequisite you should have access to two PubSub+ brokers. You could use Solace Cloud services (with the exception of the "Free" plan), Standard Edition brokers deployed using Docker (configured for the 1000 connection scaling tier) or a combination of both.

Setting up DMR requires configuration on two resources:

Message VPN
- Ensure the VPN is enabled for DMR. This should be the case with new deployments, it may be disabled on upgrades from previous versions of the broker.
- Add bridges between the local VPN and remote VPN that you want to connect.
DMR Cluster
- Create a DMR cluster or join an existing cluster
- Create external or internal links to remote nodes
- Attach the address of the cloud broker to the link on the local broker as the local broker will establish the connection.

DMR links establish the physical connection between nodes, DMR bridges connect the data channel and enable the flow of events between two Message VPNs.

Gather information about the brokers

First let's gather the information we need to connect to the SEMPv2 API and other data we need:

Broker hostnames - in Solace Cloud you can find a service's host name on the Status tab of your service - look for "Host Name".
Broker SEMPv2 base path - host name, HTTP or HTTPS admin port. For a locally deployed broker it may look like http://localhost:8080/SEMP/v2/. In cloud you can find the SEMPv2 base path on the Manage tab in the cloud console, look at the SEMP - REST API section.
Names of the VPNs that you would like to connect
Management username and password. Refer to your docker container setup or to your services' Management tab in the Cloud Console.
- I'll admin/secret and admin/cloud
Primary router name / virtual router name of both brokers. In Solace Cloud you can find this on the Status tab of your service - look for "Primary Router Name". For any broker you can use the following cUrl command, replacing http://localhost:8080 with your broker URL and changing the --user parameter to your admin username and password. I have added this call to the postman collection as well.

curl --location --request POST 'http://localhost:8080/SEMP' \
--header 'Content-Type: application/xml' \
--user admin:secret  \                                  
--data-raw '<rpc>
        <show>
        <router-name>      

        </router-name>
    </show>
</rpc>'

If using Solace cloud you need the following information about the cluster that is pre-configured on the service. You can find this on the Status tab of your service
- Cluster name
- Cluster password

In the Postman collections I'll use the following parameters for the brokers. I set these up in an environment so you can easily import the collection, adjust the environment and test against your brokers.

Parameter	Local broker (docker)	Solace Cloud Service
Host Name	localhost	service.messaging.solace.cloud
Base URL	http://localhost:8080/SEMP/v2/	https://service.messaging.solace.cloud:943/SEMP/v2/
Message VPN	default	cloud-vpn
Management username	admin	admin
Management password	secret	cloud
Primary Router Name	local-router	cloud-router
DMR Cluster Name	zone-1	cloud-cluster
DMR Cluster Password	cluster-1	cloud-secret

Setting up the DMR cluster and connection

This Postman collection illustrates how to setup the DMR cluster and connection between local and cloud broker.
Executing all the request in the order they appear in the collection - manually or by using Postman's "Run" dialog - results in a functional external link between the brokers.

Removing the connection and the DMR cluster

This collection removes the connection created by the previous collection.
Executing all the request in the order they appear in the collection - manually or by using Postman's "Run" dialog - undos all configuration.

Installing Photon OS on a Dell Edge Gateway 3001/3002

swenhelge — Tue, 17 Sep 2019 13:18:48 +0000

Installing Photon OS - actually any OS - on the Dell Edge Gateway 300x series is generally quite tricky as neither the 3001 nor 3002 offer a display port.
Any OS installation therefore must be headless and automated.
During a recent attempt to install Photon OS I found the instructions provided for the Edge Gateway 300x on the Photon OS site missed a bit of detail.
Fortunately I got some very good instructions from a helpful person at VMware that got me nearly there with a few tweaks that I had to figure out.

In this post I’ll provide a step by step guide to create a bootable USB stick for an automated Photon OS installation and how to run this on the Dell Gateway.
The Photon OS ISO includes configuration for grub and MBR boot options. The instructions on the Photon OS site (https://vmware.github.io/photon/assets/files/html/3.0/photon_installation/Installing-Photon-OS-on-Dell-300X.html) reference the grub based approach.
I’ll use the MBR approach instead - the basic steps are:

Create a bootable USB media from the Photon OS ISO image
Configure the boot menu for automated boot
Configure kickstarter for headless installation
Add BIOS script to instruct Gateway to boot from USB
Install Photon OS onto the Gateway using the USB media

It's best to use a USB stick with an indicator LED for disk activity as it will make it easier to monitor the installation process.

Create bootable USB media from Photon OS ISO

Download the ISO file for your processor architecture (x86/64 for the Dell Edge GW) from the Photon OS site: https://github.com/vmware/photon/wiki/Downloading-Photon-OS

Copy the contents of the ISO onto a USB stick. As a result the stick must be in FAT format and bootable via MBR.

On Windows this can easily be achieved using Rufus, select “ISO file” in the "boot selection” and locate the Photon ISO file on your disk.
Make sure the options are similar to below. If you don’t see the "FAT32(Default)" option for “File System” go with any other FAT32 option there may be and use its default cluster size.

Configure the boot menu for automated start

Open the syslinux.cfg file on the USB stick that you have prepared and verify it looks like this:

DEFAULT loadconfig

LABEL loadconfig
  CONFIG /isolinux/isolinux.cfg
  APPEND /isolinux/

This file references isolinux.cfg in the isolinux folder. Open it in a text editor.
The line default vesamenu.c32 in this file forces the install into graphic mode even though the prompt and timeout values should ensure the boot runs automatically.
Change the line to default install, your file should look similar to this:

# D-I config version 2.0
include menu.cfg
default install
prompt 0
timeout 0

The boot menu is now configured to auto-run the install installation option.

The next steps will configure the install option for automated installation using kickstarter.
Open /isolinux/menu.cfg and change the append line as below so kickstarter picks up a configuration file (accessible as a CD ROM):

menu hshift 7
menu width 61

menu title Photon installer boot menu
include stdmenu.cfg

default install
label install
    menu label ^Install
    menu default
    kernel vmlinuz
    append initrd=initrd.img root=/dev/ram0 loglevel=3 photon.media=cdrom ks=cdrom:/isolinux/sample_ks.cfg console=ttyS0,115200n8

Configure kickstarter for headless installation

Now let's review the /isolinux/sample_ks.cfg which configures the automated installation. The changes required in this file are also documented on the Photon OS site - it's the same file the grub boot option uses.
The important part in this file is to adjust the postinstall section to enable remote SSH for the root account so you can actually connect to the Gateway once the installation is finished.
The postinstall step simply patches the /etc/ssh/sshd_config.

Also adjust the hostname and password.text as needed and take note of the values as you will need these to connect to the Gateway after installation.
Also note for the 300X series the disk device name must be specified as /dev/mmcblk0

{
    "hostname": "photon-machine",
    "password": 
        {
            "crypted": false,
            "text": "Secret!"
        },
    "disk": "/dev/mmcblk0",
    "partitions": [
                        {"mountpoint": "/", "size": 0, "filesystem": "ext4"},
                        {"mountpoint": "/boot", "size": 128, "filesystem": "ext4"},
                        {"mountpoint": "/root", "size": 128, "filesystem": "ext4"},
                        {"size": 512, "filesystem": "swap"}
                    ],
    "packagelist_file": "packages_minimal.json",
    "additional_packages": ["vim"],
    "postinstall": [
                        "#!/bin/sh",
                        "sed -i 's/PermitRootLogin no/PermitRootLogin yes/g' /etc/ssh/sshd_config"
                   ],
    "public_key": "<ssh-key-here>",
    "install_linux_esx": false
}

Make sure there is a packages_minimal.json in the isolinux folder and that it looks like this:

{
    "packages": [
                 "minimal",
                 "linux",
                 "initramfs"
                 ]
}

Add BIOS script to instruct Gateway to boot from USB

In order to force the Gateway to boot from the USB stick we will need to add a file that instructs the BIOS to do so.

Create a UsbInvocationScript.txt file in the root of the USB drive with the content below:

usb_disable_secure_boot noreset;
usb_one_time_boot usb;

The Gateway Manual provides more information on Using USB invocation scripts

Install onto Gateway using the USB media

Now you are ready to install the OS onto the Gateway.

There are two visual indicators for the installation process:

The USB stick's activity LED
The Gateway's status LEDs - only two will be active, the power and network LED.

Power-off the Gateway - unplug network/POE or power cable. Insert your USB stick and power up the Gateway again.

I'll describe what to generally expect to happen but will skip details.
If you want to understand in detail what Gateway LEDs you should expect to flash and in which sequence have a look at "Edge Gateway USB script utility User's Guide" accessible from Using USB invocation scripts.

Once you have powered up the Gateway I'd recommend to closely monitor the installation.
The following will happen:

Power and Network LED turn green
The gateway scans for UsbInvocationScript.txtfiles on the USB interfaces.
The USB stick's LED flashes once the Gateway found our invocation script.
The gateway LEDs flash in different patterns as the BIOS script is executed. The flash patterns correspond to the commands in the UsbInvocationScript.txt
Eventually the power LED turns green and the network LED turns amber and both will be on continuously.
The activity LED of the USB stick flashes indicating that the Gateway boots from the stick and has started copying data during the installation.
Eventually - after 5 to 10 minutes both Gateway LEDs will turn green again. Unplug your USB stick as soon as that happens, otherwise the Gateway may reboot from the USB stick and kick off the installation again.

Once the installation is complete find the IP address of the Gateway in your router. Depending on your router you can identify the IP address by the MAC address of the Gateway or the host name that is set in the /isolinux/sample_ks.cfg.

This is how it looks like on my router, showing the IP, host name and MAC address:

You can now connect to the Gateway via SSH using the IP address above and the root password that you set in the /isolinux/sample_ks.cfg:

ssh root@192.168.0.112

Designing, Documenting and Testing Event APIs for IoT Platforms

swenhelge — Thu, 15 Aug 2019 18:42:21 +0000

In this article we will dive deeper into how we designed and documented the Event APIs of our IoT reference platform that we outlined in our previous blog post about prototyping an IoT platform. We will:

Describe the event flow between platform components
Identify the core APIs for the use case
Explain how we designed and documented Event APIs and provide examples and links to the full design and documentation.
Briefly explain how we on-boarded partner applications and tested apps

Throughout the project we started to adopt AsyncAPI, so we will also briefly outline how AsyncAPI and Solace’s Event Horizon initiative will eventually provide full, tool-supported Event API life-cycle management. The APIs and tooling described here can be found on GitHub.

Introduction

In our earlier post we introduced a reference platform for IoT that we built with several partners. Solace’s event mesh connects all the components of the platform.

We had to ensure that all our and our partner’s contributions played together, and that development could be carried out independently by:

Designing the contract between the components
Documenting this contract
On-boarding partners onto the event mesh
Coming up with a way to independently test contributions

In other words, we had to design, document, and test event driven APIs.

The diagram below depicts the event flow between our platform components:

Sensor data/telemetry flows from the device.
This data is consumed/processed by visual analytics – Altair Panopticon.
Telemetry is also fed into SL’s RTView-based real-time dashboards.
Commands and configuration events flow from a custom user command and control app implemented in Dell Boomi to the device.
Configuration events signal a status change – this is of interest to analytics systems. For example, setting devices into race mode indicates the start of a race on our slot-car track.
Notifications and alerts are created based on data analytics and are consumed to create service cases in Salesforce.

Looking at this use case, we identified three functional areas or APIs:

Telemetry – everything related to sensor data from the device
Command & Configuration – control and status messages from and to the device
Notification – alert and notification events that can be consumed and used to generate actions

As you can see, we separated the data and control communication with the device into two APIs. This enables us to do more fine-grained routing of events through the IoT Event Mesh.

The actors in the diagram above fall into two different categories:

Distributed devices – the Bosch XDK110
Platform services including analytics, device configuration, dashboards, and monitoring

Compared to REST/HTTP, the roles of the actors in an Event API are harder to classify. In REST a client initiates a request, a server provides the response and this communication is always point to point. Event APIs are much more varied. They are inherently bi-directional, and they support publish/subscribe and request/reply message exchanges. This makes it harder to assign client and server roles.

However, we still decided to identify the client side of an interaction and the service provider and describe the interaction from the client’s point of view similar to REST APIs. We considered topics as resource identifier and assigned the provider role to the actor that is managing the data.

In case of device interaction, this means we identify the device as the client that pushes data to a telemetry resource from which the provider consumes the events and processes them. For configuration data we consider the service provider to be the device or configuration management application as it is responsible for maintaining configuration and notifying the client – the device – when changes occur.

How to design event-driven APIs?

Expanding on the previous sections we identified the following aspects of an Event API that we would need to address:

Actors – client and service provider
Channels these actors communicate on – topics or queues
Exchange patterns – publish, subscribe, request-response
Intent or verbs – Is the event intended as a create, update, or delete? We considered it important to include the verb similarly to the HTTP request method.
Payloads – the event data
Attributes such as protocols, Quality of Service, security.

As mentioned previously, we decided to describe the interaction from the client’s point of view with a provider implementation mirroring this interaction.

Solace PubSub+ is a polyglot event broker and we leveraged a number of protocols for the showcase such as MQTT, REST, SMF (Solace API), and WebSockets based on the client’s or provider’s preferred technology/APIs. This is quite a powerful feature when it comes to rapid integration of many different technologies, but it has its drawbacks as well. The implication is that we can only rely on the event characteristics that all these protocols share in order to maintain interoperability.

The typical approach to solving this would be to create a convention for a message envelope that encloses message headers as well as the payload. However, this has some fundamental drawbacks as well. Each application needs to parse the payload to extract the headers and only then apply processing rules. This is both expensive in development and maintenance but also at runtime because it typically results in each message being parsed multiple times in the system.

An alternative approach is to make use of fine-grained topic structures to convey all required information at run time about the event. In practice, this requires that the event broker be able to support potentially many millions of topics – which is exactly one of the characteristics the Solace PubSub+ broker has. This is the approach we have chosen here.

Essentially, the topic becomes the way to communicate meta information such as the verb (or operation), the resource identifier and resource hierarchy or the expected payload type.

Therefore, we need to define conventions for the topic namespace and to design this namespace carefully. The way Solace PubSub+ implements topics helps us to create a flexible and extendable namespace. This blog post contrasts Solace’s topic implementation to a more traditional approach and explains it in more detail:

“With Solace, topics aren’t actually configured on the broker: they’re instead defined by the publishing application at publish time. They can be thought of as a property of a message, or metadata, and not like “writing to a file” or “sending to a destination.” Each and every message could be published with a unique topic.”

This is also why it’s easy to commission additional IoT devices into our solution that can communicate on dedicated topics immediately without additional broker configuration.

We defined the following convention for topic names (you can find more details and examples in this github project):

{method}/{representation}/{base-topic}/{version}/{resource-categorization}/{resource}/{id}/{aspect}

This convention is loosely based on typical REST resource identifiers and also incorporates some information that in the REST world is typically conveyed via HTTP headers:

Method – indicates clearly the action that the recipient of the event shall take such as “CREATE”, “UPDATE”, and “REPLACE”. Equivalent to HTTP verbs.
Representation – the data format used such as JSON or XML
Base topic – similar to the base URL
Version – the API version
Resource categorization – this can be a hierarchical location, functional categorisation or similar
Resource – such as device or IoT gateway
Id – the unique resource Id
Aspect – such as metrics, status, configuration

As we standardised on JSON as the payload format and didn’t intend to introduce versioning, we omitted these items from the topics we defined.

For a device emitting telemetry data the destination topic looks something like this:

CREATE/iot-control/bcw/solacebooth/handheld-solace/device/7bbb15d6-ca36-498a-9d52-7ddcef2a1d75/metrics

Armed with all of this we started describing our API design. We used Async API for this, an initiative similar to Open API in the REST world.

Let’s have a quick look at the channels and messages defined in the Telemetry API.

It defines one channel that the client – the device – is supposed to publish events to (see the “publish” definition in the excerpt below). The intent of the client is to create a new telemetry event, hence the verb used is “CREATE”. You will recognize the topic design outlined above. The elements of the resource categorization and the device id are defined as parameters in the channel:

channels: CREATE/iot-event/{region}/{location}/{production-line}/device/{deviceId}/metrics: description: Topic to subscribe to connected events parameters: - $ref: '#/components/parameters/region' - $ref: '#/components/parameters/location' - $ref: '#/components/parameters/production-line' - $ref: '#/components/parameters/deviceId' publish: summary: device publishes sensor readings. each event represents a new (create) reading for the timestamp. operationId: publishTelemetry message: $ref: '#/components/messages/telemetryMessage'

The telemetryMessage transferred via this channel is referenced above and you can see how it is defined in the components section.

components: messages: telemetryMessage: name: telemetryMessage title: Telemetry Message summary: Telemetry Message description: "Depending on the mode of the device, the telemetry message can contain all of the sensor readings or only a partial set. For units and ranges please see the datasheet for the XDK. You can find it on this site: https://xdk.bosch-connectivity.com" contentType: application/json payload: $ref: "#/components/schemas/telemetryEvent" schemas: telemetryEvent: type: array items: type: object title: Telemetry Reading/Sample

The components section also defines schemas and parameters using JSON Schema.

The second API we created was for the device to receive command and configuration events.

It was important to use a topic hierarchy that allowed us to target either a specific device or a group of devices on a production line, a location, or region. To achieve this, we designed a set of topics (or channels) that follows our design convention and creates a unique topic for each of the devices and device groups.

Here’s an excerpt from the channel hierarchy which strips out elements of the categorisation one by one to address a higher-level group. You can see that a device subscribes to a number of topics and once again these topics do not need to be defined on the broker upfront and the broker supports use of millions of topics.

channels: UPDATE/iot-control/{region}/{location}/{production-line}/device/{deviceId}/configuration: description: device subscribes to configuration events. each event represents an update to the existing configuration. parameters: - $ref: '#/components/parameters/region' - $ref: '#/components/parameters/location' - $ref: '#/components/parameters/production-line' - $ref: '#/components/parameters/deviceId' subscribe: summary: Subscribe to device specific configuration events operationId: subscribeDeviceConfiguration message: $ref: '#/components/messages/configurationMessage' UPDATE/iot-control/{region}/{location}/{production-line}/device/configuration: description: device subscribes to configuration events. each event represents an update to the existing configuration. parameters: - $ref: '#/components/parameters/region' - $ref: '#/components/parameters/location' - $ref: '#/components/parameters/production-line' subscribe: summary: Subscribe to production line specific configuration events operationId: subscribeProductionLineConfiguration message: $ref: '#/components/messages/configurationMessage'

You can find the full definitions for all three APIs on GitHub:

These are not very easy on the eye, so let’s see how we can create HTML or Markdown documentation.

Documentation

We first started out handcrafting our documentation on our internal wiki. However, as AsyncAPI progressed over time, it became the best option to maintain the API design and generate documentation as required. We have published the full documentation on GitHub in markdown format so it renders really nicely.

The AsyncAPI tooling is still in its early days, however, we found their template-based generator tool quite useful. It comes with HTML and Markdown templates.

You can explore how to design and create different documentation formats on the AsyncAPI playground for their upcoming 2.0.0 release of the specification.

It loads an example API:

Let’s load an API definition directly from our GitHub:

copy this URL https://raw.githubusercontent.com/solace-iot-team/iot-platform-api-docs/master/api-definitions/iot-event.yml
paste this URL into the textbox that reads “Type the URL of the AsyncAPI …” and then click “Import document”:

You can toggle the different views between HTML and markdown. The AsyncAPI Generator allows you to automate document generation as part of a build process in a similar manner to this web tool.

To explore our three APIs in detail, start with the readme on GitHub.

Onboarding

We manually onboarded applications onto the PubSub+ platform. Onboarding of an API client or provider required the following steps:

Create an Access Control List (ACL) for each application: the ACL provides access to the topics an application utilising one or more APIs needs to publish and subscribe as per the API definitions. A future enhancement could be to generate these ACL profiles directly from the Async API definition.
Create a client user name for the application to be registered. At this time Solace supports username/password and client certificate authentication. OAuth support utilising an external Authorization Server will be released shortly.

Here at Solace we have things in the pipeline that will automate this process:

Event Portal – will provide application lifecycle capabilities that provision the information above into a Solace Event Mesh. Read more in the “What’s next?” section below.
A set of IoT APIs that simplify integration with Device Management platforms and provide capabilities to provision device types and devices into the Event Mesh. This will be released as part of our IoT Labs repository.

API Testing

As described in the previous blog post in this series, we mainly used MQTT Box to test APIs. In the future we may be able to use the AsyncAPI Generator to generate stub applications and build automated tests based on these – at the time of writing the code generation templates are at a lower level of maturity compared to the document generation, so pursuing this avenue was not deemed feasible for our project.

What’s next?

Solace recently has announced its Event Horizon initiative aiming to create a developer experience and ecosystem around Event APIs similar to the synchronous API/REST experience.

The first step will be an Event Portal that enables the creation and discovery of events and the definition of event-driven application interfaces. Solace is also contributing to frameworks such as Spring CloudStreams and AsyncAPI adding code generators for CloudStreams which will accelerate Event API implementation and aid in testing.

In the IoT space we will add more open source capabilities for Device Management and IoT Platform integration. We also evolve the IoT APIs described in this blog post as we add more prototypes and examples on other platforms such as Azure IoT Hub and Azure Sphere.

Conclusion

A crucial factor in the development of our prototyping showcase was to design our event-driven IoT APIs upfront similar to contract first development with REST APIs. By making sure the topic structure/namespace we used was flexible and extendable, we avoided payload introspection and enabled fine-grained and flexible filtering for every subscriber.

We followed a typical API management lifecycle to design and document our IoT APIs through to on-boarding of applications and testing. AsyncAPI proved very useful in the design and document phases. As the Solace Event Portal, AsyncAPI and code generators evolve the full API life-cycle will eventually be tool-supported.

The next article (coming soon) will focus on the implementation of the custom RTOS C application we have developed for the Bosch XDK110 which implements the device APIs described here.

Stay tuned!

The post Designing, Documenting and Testing Event APIs for IoT Platforms appeared first on Solace.