DEV Community: Henrique Gonçalves Dias

fhir-integratedml-example overview

Henrique Gonçalves Dias — Wed, 23 Feb 2022 15:45:44 +0000

Talking with a friend of mine, Machine Learning specialist @renatobanzai , he brought one of the biggest challenges faced by companies nowadays: deploying ML/AI in live environments.
InterSystems IRIS offers IntegratedML. IntegratedML it's a great feature to train, test, and make deploys of ML/AI models.

The hardest part in creating ML/AI is to make the data treatment, clean up, and make them trusted.

This is where we can take advantage of FHIR powerful standard!

The project idea shows how we can create/train/validate ML/AI models with FHIR and utilize them with data from different sources.

We believe that this project has great potential and a few ideas that can be explored:

Reuse/extend DTL transformations in other FHIR databases for custom ML models
Use DTL transformations to normalize FHIR messages and publish ML models as services
Create a kind of models + transformations rules repository for use within any FHIR dataset
Exploring new possibilities with this project, we can imagine data from different sources.

In the image above, the FHIR Resource, consuming the REST API, can be used with an FHIRaaS.

And not only using FHIRaaS on AWS, but we can also make use of the new service HealthShare Message Transformation Services, that automates the conversion of HL7v2 to FHIR® to populate Amazon HealthLake, where you can extract more value from your data.

With these small demonstrations, I see these resources being used very well in larger scenarios, enabling and delivering more easily deploys in production in truly innovative environments, like the AWS Healthlake. Why not?! 😃

An overview of npm-iris

Henrique Gonçalves Dias — Mon, 05 Jul 2021 18:15:19 +0000

What is npm-iris?

N.P.M stands for "No Project Mess."

N.P.M. is a Project & Task Management app that uses InterSystems IRIS and Bootstrap 4.

No Project Mess is created to help developers and small business companies to reduce complexity in their daily problems, with a simple and intuitive projects and tasks management software.

It offers different views for your tasks, from a spreadsheet, kanban, calendar, or even Gantt!

Why?

Working within different teams, you’ll see different people that prefer different tools.

So, many times you’ll be using a Gantt chart for a project, a Kanban for another, a list on a paper in other…

N.P.M. goes straight to the task. No matters how you and your team prefer to see them. Just click and change your view.

Features

Initial Setup
- Projects
- Users
Tasks - Create and manage tasks
Scheduler – A calendar view for your tasks
Kanban – Manage your tasks in Kanban Style
Gantt – See your deadlines, milestones, and your progress using a Gantt chart

Roadmap for New features / Improvements

OAuth2 authentication
Security by Project/Team/User
Time Tracking
Custom Calendar (Holidays)
Support for attached files
Make use of AppS.REST framework
Vue.js Edition
Home dashboard to see the general status for the activities

Download

You can try the application for free ! =]
npm-iris on OpenExchange

Screenshots

Tasks

The task view it's easy to use as filling cells in an Excel spreadsheet.

Scheduler
Scheduler view gives you the option to see your tasks as appointments, in the week, month, and Agenda viewer, acting as a Google Calendar clone.

Kanban
For those who prefers deal with the tasks in a Kanban Style, this view it's perfect to do it.

Gantt

Finally, for the Project Managers out there, looking forward to follow the project in real time, using a MS Project style, this view is for you!

Building an Alternative IRIS Message Viewer

Henrique Gonçalves Dias — Wed, 28 Oct 2020 14:41:13 +0000

If you had the opportunity to change something in the IRIS Interoperability Message Viewer, what would you do?

After publishing the article Dashboard IRIS History Monitor, I received some great feedback and some requests. One request was for an enhanced Message Viewer.

If you haven’t yet done so, check out the project—it’s definitely worth your time, and it won the Bronze award as one of The Best InterSystems Open Exchange Developers and Applications in 2019.

I started drafting some ideas about the features I’d want to include in the “new” Message Viewer, but how could I show these resources in the fastest and easiest way?

Well, first things first. You generally start by setting up an interoperability production, then exporting and deploying it on the target system, as indicated in the documentation. This is a process I really don’t like. Not that there’s anything wrong with it, really. I’ve just idealized doing everything using code.

I expect that every time someone runs this sort of project, they start like this:

$ docker-compose build
$ docker-compose up -d

And voilá!!!

With those simple steps in mind, I started to look in the InterSystems community and found a few tips. One of the posts brought up the question I was asking myself: How to create productions via routine?

In that post, Eduard Lebedyuk answered, showing how to create a production using code.

To create production class automatically you need to:

Create %Dictionary.ClassDefinition object for your test production

Create Ens.Config.Production object

Create %Dictionary.XDataDefinition

Serialize (2) into (3)

Insert XData (3) into (1)

Save and compile (1)

I also found a comment from Jenny Ames:

One best practice we often recommend is to build backward. Build business operations first, then business processes, then business services…

So, let’s do it!

Requests, Business Operations, and Business Services

The class diashenrique.messageviewer.util.InstallerProduction.cls is, as the name suggests, the class responsible for installing our production. The installer manifest invokes the ClassMethod Install from that class:

/// Helper to install a production to display capabilities of the enhanced viewer

ClassMethod Install() As %Status
{

    Set sc = $$$OK

    Try {

        Set sc = $$$ADDSC(sc,..InstallProduction()) quit:$$$ISERR(sc)

        Set sc = $$$ADDSC(sc,..GenerateMessages()) quit:$$$ISERR(sc)

        Set sc = $$$ADDSC(sc,..GenerateUsingEnsDirector()) quit:$$$ISERR(sc)

    }

    Catch (err) {

        Set sc = $$$ADDSC(sc,err.AsStatus())

    }

    Return sc

}

The classmethod InstallProduction brings together the main structure for creating a production by creating:

a request
a business operation
a business service
an interoperability production

Since the idea is to create an interoperability production using code, let’s go into full coding mode to create all classes for the request, the business operation, and the business services. In doing so, we’ll make extensive use of some InterSystems library packages:

%Dictionary.ClassDefinition
%Dictionary.PropertyDefinition
%Dictionary.XDataDefinition
%Dictionary.MethodDefinition
%Dictionary.ParameterDefinition

The classmethod InstallProduction creates two classes that extend from Ens.Request, using the following lines:

Set sc = $$$ADDSC(sc,..CreateRequest("diashenrique.messageviewer.Message.SimpleRequest","Message")) quit:$$$ISERR(sc)

Set sc = $$$ADDSC(sc,..CreateRequest("diashenrique.messageviewer.Message.AnotherRequest","Something")) quit:$$$ISERR(sc)

ClassMethod CreateRequest(classname As %String, prop As %String) As %Status [ Private ]

{

    New $Namespace

    Set $Namespace = ..#NAMESPACE

    Set sc = $$$OK

    Try {

        Set class = ##class(%Dictionary.ClassDefinition).%New(classname)

        Set class.GeneratedBy = $ClassName()

        Set class.Super = "Ens.Request"

        Set class.ProcedureBlock = 1

        Set class.Inheritance = "left"

        Set sc = $$$ADDSC(sc,class.%Save())

        #; create adapter

        Set property = ##class(%Dictionary.PropertyDefinition).%New(classname)

        Set property.Name = prop

        Set property.Type = "%String"

        Set sc = $$$ADDSC(sc,property.%Save())

        Set sc = $$$ADDSC(sc,$System.OBJ.Compile(classname,"fck-dv"))

    }

    Catch (err) {

        Set sc = $$$ADDSC(sc,err.AsStatus())

    }

    Return sc

}

Now let’s create the class for a business operation that extends from Ens.BusinessOperation:

Set sc = $$$ADDSC(sc,..CreateOperation()) quit:$$$ISERR(sc)

Besides creating the class, we create the MessageMap and the method Consume:

ClassMethod CreateOperation() As %Status [ Private ]
{
    New $Namespace
    Set $Namespace = ..#NAMESPACE
    Set sc = $$$OK
    Try {
        Set classname = "diashenrique.messageviewer.Operation.Consumer"
        Set class = ##class(%Dictionary.ClassDefinition).%New(classname)
        Set class.GeneratedBy = $ClassName()
        Set class.Super = "Ens.BusinessOperation"
        Set class.ProcedureBlock = 1
        Set class.Inheritance = "left"

        Set xdata = ##class(%Dictionary.XDataDefinition).%New()
        Set xdata.Name = "MessageMap"
        Set xdata.XMLNamespace = "http://www.intersystems.com/urlmap"
        Do xdata.Data.WriteLine("<MapItems>")
        Do xdata.Data.WriteLine("<MapItem MessageType=""diashenrique.messageviewer.Message.SimpleRequest"">")
        Do xdata.Data.WriteLine("<Method>Consume</Method>")
        Do xdata.Data.WriteLine("</MapItem>")
        Do xdata.Data.WriteLine("<MapItem MessageType=""diashenrique.messageviewer.Message.AnotherRequest"">")
        Do xdata.Data.WriteLine("<Method>Consume</Method>")
        Do xdata.Data.WriteLine("</MapItem>")
        Do xdata.Data.WriteLine("</MapItems>")      
        Do class.XDatas.Insert(xdata)
        Set sc = $$$ADDSC(sc,class.%Save())

        Set method = ##class(%Dictionary.MethodDefinition).%New(classname)
        Set method.Name = "Consume"
        Set method.ClassMethod = 0
        Set method.ReturnType = "%Status"
        Set method.FormalSpec = "input:diashenrique.messageviewer.Message.SimpleRequest,&output:Ens.Response"
        Set stream = ##class(%Stream.TmpCharacter).%New()
        Do stream.WriteLine("   set sc = $$$OK")
        Do stream.WriteLine("   $$$TRACE(input.Message)")
        Do stream.WriteLine("   return sc")
        Set method.Implementation = stream
        Set sc = $$$ADDSC(sc,method.%Save())

        Set sc = $$$ADDSC(sc,$System.OBJ.Compile(classname,"fck-dv"))
    }
    Catch (err) {
        Set sc = $$$ADDSC(sc,err.AsStatus())
    }
    Return sc
}

In the last step before actually creating the interoperability production, let’s create the class responsible for the business service:

Set sc = $$$ADDSC(sc,..CreateRESTService()) quit:$$$ISERR(sc)

This class has UrlMap and Routes to receive Http requests.

ClassMethod CreateRESTService() As %Status [ Private ]
{
    New $Namespace
    Set $Namespace = ..#NAMESPACE
    Set sc = $$$OK
    Try {
        Set classname = "diashenrique.messageviewer.Service.REST"
        Set class = ##class(%Dictionary.ClassDefinition).%New(classname)
        Set class.GeneratedBy = $ClassName()
        Set class.Super = "EnsLib.REST.Service, Ens.BusinessService"
        Set class.ProcedureBlock = 1
        Set class.Inheritance = "left"

        Set xdata = ##class(%Dictionary.XDataDefinition).%New()
        Set xdata.Name = "UrlMap"
        Set xdata.XMLNamespace = "http://www.intersystems.com/urlmap"
        Do xdata.Data.WriteLine("<Routes>")
        Do xdata.Data.WriteLine("<Route Url=""/send/message"" Method=""POST"" Call=""SendMessage""/>")
        Do xdata.Data.WriteLine("<Route Url=""/send/something"" Method=""POST"" Call=""SendSomething""/>")
        Do xdata.Data.WriteLine("</Routes>")
        Do class.XDatas.Insert(xdata)
        Set sc = $$$ADDSC(sc,class.%Save())

        #; create adapter
        Set adapter = ##class(%Dictionary.ParameterDefinition).%New(classname)
        Set class.GeneratedBy = $ClassName()
        Set adapter.Name = "ADAPTER"
        Set adapter.SequenceNumber = 1
        Set adapter.Default = "EnsLib.HTTP.InboundAdapter"
        Set sc = $$$ADDSC(sc,adapter.%Save())

        #; add prefix
        Set prefix = ##class(%Dictionary.ParameterDefinition).%New(classname)
        Set prefix.Name = "EnsServicePrefix"
        Set prefix.SequenceNumber = 2
        Set prefix.Default = "|demoiris"
        Set sc = $$$ADDSC(sc,prefix.%Save())

        Set method = ##class(%Dictionary.MethodDefinition).%New(classname)
        Set method.Name = "SendMessage"
        Set method.ClassMethod = 0
        Set method.ReturnType = "%Status"
        Set method.FormalSpec = "input:%Library.AbstractStream,&output:%Stream.Object"
        Set stream = ##class(%Stream.TmpCharacter).%New()
        Do stream.WriteLine("   set sc = $$$OK")
        Do stream.WriteLine("   set request = ##class(diashenrique.messageviewer.Message.SimpleRequest).%New()")
        Do stream.WriteLine("   set data = {}.%FromJSON(input)")
        Do stream.WriteLine("   set request.Message = data.Message")
        Do stream.WriteLine("   set sc = $$$ADDSC(sc,..SendRequestSync(""diashenrique.messageviewer.Operation.Consumer"",request,.response))")
        Do stream.WriteLine("   return sc")
        Set method.Implementation = stream
        Set sc = $$$ADDSC(sc,method.%Save())

        Set method = ##class(%Dictionary.MethodDefinition).%New(classname)
        Set method.Name = "SendSomething"
        Set method.ClassMethod = 0
        Set method.ReturnType = "%Status"
        Set method.FormalSpec = "input:%Library.AbstractStream,&output:%Stream.Object"
        Set stream = ##class(%Stream.TmpCharacter).%New()
        Do stream.WriteLine("   set sc = $$$OK")
        Do stream.WriteLine("   set request = ##class(diashenrique.messageviewer.Message.AnotherRequest).%New()")
        Do stream.WriteLine("   set data = {}.%FromJSON(input)")
        Do stream.WriteLine("   set request.Something = data.Something")
        Do stream.WriteLine("   set sc = $$$ADDSC(sc,..SendRequestSync(""diashenrique.messageviewer.Operation.Consumer"",request,.response))")
        Do stream.WriteLine("   return sc")
        Set method.Implementation = stream
        Set sc = $$$ADDSC(sc,method.%Save())

        Set sc = $$$ADDSC(sc,$System.OBJ.Compile(classname,"fck-dv"))
    }
    Catch (err) {
        Set sc = $$$ADDSC(sc,err.AsStatus())
    }
    Return sc
}

Using Visual Studio Code

Creating the classes using the %Dictionary package can be difficult, and difficult to read as well, but it’s handy. To demonstrate a slightly more straightforward approach with better code readability, I’ll also create new request, business service, and business operations classes using Visual Studio Code:

diashenrique.messageviewer.Message.SimpleMessage.cls
diashenrique.messageviewer.Operation.ConsumeMessageClass.cls
diashenrique.messageviewer.Service.SendMessage.cls

Class diashenrique.messageviewer.Message.SimpleMessage Extends Ens.Request [ Inheritance = left, ProcedureBlock ]
{
Property ClassMessage As %String;
}

Class diashenrique.messageviewer.Operation.ConsumeMessageClass Extends Ens.BusinessOperation [ Inheritance = left, ProcedureBlock ]
{
Method Consume(input As diashenrique.messageviewer.Message.SimpleMessage, ByRef output As Ens.Response) As %Status
{
    Set sc = $$$OK
    $$$TRACE(pRequest.ClassMessage)
    Return sc
}
XData MessageMap [ XMLNamespace = "http://www.intersystems.com/urlmap" ]
{
  <MapItems>
    <MapItem MessageType="diashenrique.messageviewer.Message.SimpleMessage">
      <Method>Consume</Method>
    </MapItem>
  </MapItems>
} 
}

Class diashenrique.messageviewer.Service.SendMessage Extends Ens.BusinessService [ ProcedureBlock ]
{ 
Method OnProcessInput(input As %Library.AbstractStream, ByRef output As %Stream.Object) As %Status
{
    Set tSC = $$$OK
    // Create the request message
    Set request = ##class(diashenrique.messageviewer.Message.SimpleMessage).%New()
    // Place a value in the request message property
    Set request.ClassMessage = input
    // Make a synchronous call to the business process and use the response message as our response 
    Set tSC = ..SendRequestSync("diashenrique.messageviewer.Operation.ConsumeMessageClass",request,.output)
    Quit tSC
}
}

From a code readability perspective, it’s a huge difference!

Creating the Interoperability Production

Let’s finish up the creation of our interoperability production. To do so, we’ll create a production class, then associate it with the business Operation and Service classes.

Set sc = $$$ADDSC(sc,..CreateProduction()) quit:$$$ISERR(sc)

ClassMethod CreateProduction(purge As %Boolean = 0) As %Status [ Private ]
{
    New $Namespace
    Set $Namespace = ..#NAMESPACE
    Set sc = $$$OK
    Try {
         #; create new production
        Set class = ##class(%Dictionary.ClassDefinition).%New(..#PRODUCTION)
        Set class.ProcedureBlock = 1
        Set class.Super = "Ens.Production"
        Set class.GeneratedBy = $ClassName()
        Set xdata = ##class(%Dictionary.XDataDefinition).%New()
        Set xdata.Name = "ProductionDefinition"
        Do xdata.Data.Write("<Production Name="""_..#PRODUCTION_""" LogGeneralTraceEvents=""true""></Production>")  
        Do class.XDatas.Insert(xdata)
        Set sc = $$$ADDSC(sc,class.%Save())
        Set sc = $$$ADDSC(sc,$System.OBJ.Compile(..#PRODUCTION,"fck-dv"))
        Set production = ##class(Ens.Config.Production).%OpenId(..#PRODUCTION)
        Set item = ##class(Ens.Config.Item).%New()
        Set item.ClassName = "diashenrique.messageviewer.Service.REST"
        Do production.Items.Insert(item)
        Set sc = $$$ADDSC(sc,production.%Save())
        Set item = ##class(Ens.Config.Item).%New()
        Set item.ClassName = "diashenrique.messageviewer.Operation.Consumer"
        Do production.Items.Insert(item)
        Set sc = $$$ADDSC(sc,production.%Save())    
        Set item = ##class(Ens.Config.Item).%New()
        Set item.ClassName = "diashenrique.messageviewer.Service.SendMessage"
        Do production.Items.Insert(item)
        Set sc = $$$ADDSC(sc,production.%Save())    
        Set item = ##class(Ens.Config.Item).%New()
        Set item.ClassName = "diashenrique.messageviewer.Operation.ConsumeMessageClass"
        Do production.Items.Insert(item)
        Set sc = $$$ADDSC(sc,production.%Save())    
    }
    Catch (err) {
        Set sc = $$$ADDSC(sc,err.AsStatus())
    }
    Return sc
}

We use the class Ens.Config.Item to associate the production class with the business Operation and Service classes. You can do this whether you created your class using the %Dictionary package or with VS Code, Studio, or Atelier.

In any case, we did it! We created an interoperability production using code.

But remember the original purpose of all this code: to create a production and messages to show the capabilities of the enhanced Message Viewer. Using the classmethods that follow, we’ll execute both of our business services and generate the messages.

Generating Messages using %Net.HttpRequest

ClassMethod GenerateMessages() As %Status [ Private ]
{
    New $Namespace
    Set $Namespace = ..#NAMESPACE
    Set sc = $$$OK
    Try {
        Set action(0) = "/demoiris/send/message"
        Set action(1) = "/demoiris/send/something"
        For i=1:1:..#LIMIT {
            Set content = { }
            Set content.Message = "Hi, I'm just a random message named "_$Random(30000)
            Set content.Something = "Hi, I'm just a random something named "_$Random(30000)
            Set httprequest = ##class(%Net.HttpRequest).%New()
            Set httprequest.SSLCheckServerIdentity = 0
            Set httprequest.SSLConfiguration = ""
            Set httprequest.Https = 0
            Set httprequest.Server = "localhost"
            Set httprequest.Port = 9980
            Set serverUrl = action($Random(2))
            Do httprequest.EntityBody.Write(content.%ToJSON())
            Set sc = httprequest.Post(serverUrl) 
            Quit:$$$ISERR(sc)
        }
    }
    Catch (err) {
        Set sc = $$$ADDSC(sc,err.AsStatus())
    }
    Return sc
}

Generating Messages using EnsDirector

ClassMethod GenerateUsingEnsDirector() As %Status [ Private ]
{
    New $Namespace
    Set $Namespace = ..#NAMESPACE
    Set sc = $$$OK
    Try {
        For i=1:1:..#LIMIT {
            Set tSC = ##class(Ens.Director).CreateBusinessService("diashenrique.messageviewer.Service.SendMessage",.tService)
            Set message = "Message Generated By CreateBusinessService "_$Random(1000)
            Set tSC = tService.ProcessInput(message,.output)
            Quit:$$$ISERR(sc)
        }
    }
    Catch (err) {
        Set sc = $$$ADDSC(sc,err.AsStatus())
    }
    Return sc
}

}

That’s it for the code. You’ll find the complete project at https://github.com/diashenrique/iris-message-viewer

Running the Project

Now let’s see the project in action. First, git clone or git pull the repo into any local directory:

git clone https://github.com/diashenrique/iris-message-viewer.git

Next, open the terminal in this directory and run:

docker-compose build

Finally, run the IRIS container with your project:

docker-compose up -d

Now we’ll access the Management Portal using http://localhost:52773/csp/sys/UtilHome.csp. You should see our interoperability namespace MSGVIEWER, as in the image below:

And here’s our baby Production, with two business services and two business operations:

We have so many messages:

With everything up and running in our custom Message Viewer, let’s take a look at some of its features.

The Enhanced Message Viewer

Keep in mind that only namespaces that are enabled for interoperability productions will be displayed.

http://localhost:52773/csp/msgviewer/messageviewer.csp

The enhanced Message Viewer brings features and flexibility that allow you to create different filters, group the columns into n-levels, export to Excel, and much more.

You can use different filters to achieve the results you need. You can also use multiple sorts by pressing Shift and clicking on the column header. You even export the data grid to Excel!

In addition, you can create complex filters with the filter builder option.

You can group data against any column available, grouping the information using the n-levels you want. By default, the group is constructed using the Date Created field.

And there’s a feature that allows you to select columns. The following page has all the columns from Ens.MessageHeader, showing only the default columns in the initial view. But you can choose the other columns using the "Column Chooser" button.

You can collapse or expand all groups with a single click.

Collapse or Expand all groups in a single click.

The information in the SessionId field has a link to the Visual Trace feature.

You can resend messages if you need to. Simply select the messages you need and click to resend. This feature uses the following classMethod:

##class(Ens.MessageHeader).ResendDuplicatedMessage(id)

Finally, as mentioned, you can export your data grid to Excel:

The result in Excel will show the same format, content, and group defined in the cache server pages (CSP).

PS: I want to give special thanks to Renan Lourenco who helped me a lot on this journey.