Let us consider that you have implemented an integrated solution and are using Serverless360 Business Activity Monitoring (BAM) to help provide your support users and business users with visibility of what is happening in the business transaction. BAM provides you with distributed tracing to attain maximum visibility on the integration solution that the functional operations team needs. You might have a scenario where you have an unstructured message format that you want to archive in BAM so that the users can see the data which was processed.
Some examples of these might be:
- Processing CSV files
- Processing plain text files
- Processing EDI messages
In this article, we will look at how you can implement BAM with a Logic App solution that will allow you to archive unstructured messages but use data from them within BAM.
Before continuing, if you are unfamiliar with BAM or how it can be used in Logic Apps, please check out the resources below.
https://docs.serverless360.com/docs/checkpointing-in-bam
Sending flat file data to a system
To start, we will look at the scenario where we have an interface that will run on a schedule and look up data from one of our systems and then produce a flat-file CSV message which will be written to the partner’s SFTP site.
The solution will look like the below diagram.
A Logic App implements the solution with a recurrence trigger that will call API Management to query some data from our SaaS system. It will then use the Integration Account to convert the resulting data to a CSV format and deliver the data with the SFTP connector.
We want to share the raw messages with BAM to support the solution effectively. The level 1 support operator can troubleshoot any data issues, and the user can do some self-service actions by checking transactions executed and the data passed between systems.
To implement this, we will start by adding the BAM Start transaction shape, as shown below, to the Logic App.
We will also have a CheckPoint shape after retrieving the API data. In both shapes, we are passing information to BAM to advise where we currently are in the business transaction execution. In these shapes, we can send message body data to BAM, which can be used to extract properties from the data, or we can choose to archive the message so the support user can retrieve it.
When we called the API in the Logic App, we were returned JSON data. We can use the Transform XML shape as shown below to transform the JSON to XML, representing a schema for the CSV file. You would use the Enterprise Integration Pack to create an XSD file for the schema and load it to the integration account.
I can add a BAM checkpoint shape if I want to archive the XML data in BAM.
The critical bit about sending the XML or JSON data to BAM is these structured representations of the data make it easy to promote properties to your BAM transaction so users can search for specific transactions.
Once we have the XML representation of the data, we can use the Flat File Encoding shape, as shown above. It will let us specify the schema and convert the XML to CSV format.
We can then deliver the file to the SFTP site and do a BAM Checkpoint to indicate the interface was successfully processed. The diagram below shows that the CSV data is sent as the message body to BAM.
When the support or business user uses the BAM distributed tracing module, they will not need to worry about the shapes your Logic App may need to process your complex interface. They will see those critical milestones in the process. In a successful BAM transaction, you can see below that each stage lights up green indicating success.
Users can see the JSON data you archived to BAM if they click on the Get CRM Data shape.
If the user clicks on the Complete stage in the BAM transaction, they can see the CSV message body sent to the destination system.
Receiving flat file data from a system
The opposite scenario is when we receive data from a system and then need to decode and parse it before processing it.
In this case, we also want to share the raw data with the support and business super user to help them see the data we are processing for troubleshooting purposes.
In this case, we can implement BAM in our Logic App, and we will use the message archiving feature on the Start Transaction and CheckPoint shapes and pass the data at different stages in the process.
You can see below in this case, and we are using the StartTransaction shape. I have passed the raw CSV file as the message body.
We would then use the Integration Account flat file decoding shape to reference an XSD schema in the integration account, and this would convert the CSV data to XML format.
We can then use a CheckPoint shape after the decoding if we want to send the XML message to BAM.
In the BAM distributed tracing module, as earlier in the article, you will see the key milestones in your interface showing green if they were successful.
You can click in the first stage where I archived the CSV data, and the BAM user can see the message in BAM.
Summary
In the real world, business users, level 1 support operators, and support people outside the integration team are often quite familiar with some data formats and business transactions. Using BAM distributed tracing to give them visibility into the execution and data being processed within your interface can let them participate in your interface’s day-to-day operations, leading to a lower operating cost for your solution.
With Logic Apps, we have some excellent features for decoding, parsing, and transforming message formats like CSV, EDI, and other formats. If we send the raw messages to BAM, we can let the users see what they look like, and if we send the decoded message formats, such as the XML or JSON data, then it’s easy for us to use BAM. It helps to promote metadata from the message to allow users to search for transactions that might have processed specific data.
Oldest comments (0)