DEV Community: InterSystems Developer

AI-Powered Clinical Matching: Introducing iris-medmatch

InterSystems Developer — Sat, 30 May 2026 18:50:37 +0000

In the modern healthcare landscape, finding clinically similar patients often feels like looking for a needle in a haystack. Traditional keyword searches often fail because medical language is highly nuanced; a search for "Heart Failure" might miss a record containing "Congestive Cardiac Failure."

I am excited to share iris-medmatch, an AI-powered patient matching engine built on InterSystems IRIS for Health. By leveraging Vector Search, this tool understands clinical intent rather than just matching literal strings.
## The Core Innovation: Semantic Clinical Search

`iris-medmatch` bridges the gap between raw FHIR data and actionable AI insights. By utilizing the `all-MiniLM-L6-v2` model, the engine transforms clinical conditions into mathematical vectors.

While standard searches look for exact words, this engine understands **clinical context**. For example, it can match a patient with "Hypertension" to a search for "High Blood Pressure" using mathematical vector similarity.

✨ Key Technical Features

Core: InterSystems IRIS , Embedded Python, InterSystems FHIR Server, Vector search
AI: Python, ONNX Runtime, HuggingFace Transformers
Frontend: Angular 18+

Technical Architecture

The strength of this solution lies in its architectural efficiency. By running Transformers via Embedded Python, we eliminate "data gravity" issues. The data stays in IRIS, and the AI processing happens where the data lives.

🚀 Application Walkthrough

1. Semantic Similarity Search (The "Wow" Factor)

This module uses Vector Search to understand medical synonyms. A search for "Cardiac Issues" will mathematically find "Myocardial Infarction" by comparing their vector positions within IRIS. This is achieved using Native IRIS SQL to calculate similarity scores in sub-seconds.

2. Patient Directory & Condition Enrichment

This module manages existing FHIR resources. Users can add new diagnoses through a high-performance modal, demonstrating real-time synchronization between standard FHIR data and AI-ready vector data.

3. New Patient Registration

A streamlined entry point for creating new `Patient` resources within the InterSystems ecosystem. This features direct interaction with the FHIR R4 Repository via standard RESTful POST requests, ensuring data is indexed and searchable immediately.

Conclusion

iris-medmatch demonstrates how InterSystems IRIS is evolving into a comprehensive AI-Native database. By combining the reliability of FHIR with the power of Vector Search, we can create healthcare applications that truly "understand" the clinical data they store.

An Introduction to AI Hub, Part 2: Custom MCP Servers

InterSystems Developer — Sat, 30 May 2026 18:39:47 +0000

Welcome back to a series of introductory articles on AI Hub, the new product feature currently in an early access program! (links: EAP Site for download, documentation)

In the last article, we covered how to create agents and agent tools directly in ObjectScript using the new %AI classes. However, sometimes, instead of creating a new agent, you just want to add some custom tools to an existing agent so you can ask your local claude code, codex, copilot or other agent of choice to query your data directly. This is where MCP Servers might come in.

In this guide, we will walk through how you can create your own MCP Servers to access your data.

Disclaimer: AI Hub is an early access preview, with features likely to change before production releases, any issues identified can be raised as issues on the documentation GitHub repo linked above. The EAP preview is not to be used in production settings.

A very brief intro to MCP

I'm going to keep this brief because there are loads of other good articles on MCP Servers Model context protocol (I recommend starting with this article from @pietro.DiLeo or this brilliant introductory video from InterSystems President Don Woodlock).

Model Context Protocol is a transport protocol allowing external tools to be added to an agent. There is a discovery 'handshake' where the MCP server sends a list of tools to the MCP Client. After the tools are discovered, the agent can send requests for tool executions, including parameters, to the MCP server, which executes the tool call and returns the result.

MCP servers can be remote servers, i.e. running on a different machine to a client, this usually uses a streamable http/https connection or Server-Side Events. Or MCP servers can be local servers, i.e. running on the same machine, usually using a stdio connection.

An important distinction

AI hub allows you to create custom MCP servers within your IRIS environment, allowing agents to access or monitor your IRIS databases, productions and statuses. It is not a pre-configured MCP server.

If you are looking for a developer tool which gives your agent free access to an IRIS environment to speed up development, you may be looking for a pre-configured MCP server. If you are looking to create production MCP servers which are secure, auditable and fit within IRIS's governed security environment, AI Hub is what you are looking for.

There are many pre-configured MCP servers which provides tools for developing with IRIS, including iris-agentic-dev, an MCP tool and skills library created by @tomd. This is a separate project from AI Hub, so look out for an article about this!

MCP in AI Hub

In the previous article, we covered creating agent tools and toolsets, here we will go through how to serve these tools as an MCP server using both HTTP and STDIO. The code covered in this article is available in the ai-hub-dev-template which is a nice place to start if you want to play around with IRIS AI hub.

Before getting to this though lets, take a look at the architecture of an AI Hub MCP server:

MCP Clients (blue) communicates with iris-mcp-server to bridge the gap between the MCP calls (discovery and execution) and IRIS. This binary then communicates with an MCP Server web application, defined using %AI.MCP.Service as a dispatch class. This dispatch class then routes the tool calls to ObjectScript tool classes which can then operate on IRIS databases. This diagram skips the reverse routing (returning the tool responses) as well as the initial handshake between the MCP Client and the iris-mcp-server.

There are more details on this architecture in the documentation, but this simplified view covers the key elements that we need to define. These are:
1. Tools / Toolsets
2. %AI.MCP.Service dispatch class
3. MCP Application
4. iris-mcp-server configuration
5. MCP Client connection

Lets go through these one by one.

1. Tools

We define tools or toolsets by extending %AI.Tool or %AI.ToolSet, this was covered in detail in Part 1, so I'm going to skip over this.

2. Defining the dispatch class

To define an MCP Service dispatch class, we just need to extend %AI.MCP.Service and point it at the tools/toolsets we want to include in the SPECIFICATION parameter:

Class Sample.MCPService Extends %AI.MCP.Service
{
    Parameter SPECIFICATION = "Sample.ToolSet";
}

We could include multiple tool/toolset classes by adding the classes as a comma-separated list, but here we've kept it simple with just one.

3. Creating the MCP Application

Next up, we create an MCP server application. Like other web applications this can be managed from the management portal, or programmatically with the Security.Applications class. In this case, there is a new MCP server management portal in the Management Portal:

But everything else will feel familiar to developers creating Web Applications in IRIS.

The key points are to give an endpoint (e.g /mcp/sample) and the MCP Service Class we created earlier. I won't show the programmatic version, but this can be done with Security.Applications, just set the Type value to 18 to register it in the MCP Server menu.

At this point, you can see the JSON description of tools being served at http://localhost:52773/mcp/sample/v1/services. This means it is discoverable by the iris-mcp-server binary, it is not discoverable directly by an mcp client.

4. iris-mcp-server configuration

The iris-mcp-server binary takes a configuration file when it is run, this is run with the following command:

iris-mcp-server -c config.toml run

We'll get to exactly how this command is actually used in the next section (MCP Client), but for now we are going to focus on the config file.

The first thing to do when writing your config file is to set your connection to IRIS - this requires the credentials for a gateway-privileged user e.g. CSPSystem, the superserver port used for web-gateway (default 1972) and your MCP endpoints:

[[iris]]
name = "local"
server = { host = "localhost", port = 1972, username = "SuperUser", password = "SYS" }
endpoints = [
    {path = "/mcp/sample" }
]

Then you have to define your transport. This is done in the [mcp] block.

For stdio, you just need set the type of transport:

[mcp]
transport="stdio"

For http/https, you also have to give the host and port:

[mcp]
transport = "http"
host      = "0.0.0.0"
port      = 8080

This port is a different port to the management portal and will be used only for this MCP server. This is a common mistake because, although you can find the tool catalog on the management portal port at http://localhost:52773/mcp/sample/v1/services, to actually connect to the MCP server you have to set a different port to communicate to the iris-mcp-server bridge.

Authentication

We set the web application to unauthenticated above, but this should be avoided for production use. To add authentication, first set the web application to password authenticated. You can add a username and password or a bearer token to the endpoint:

[[iris]]
name = "local"
server = { host = "localhost", port = 1972, username = "SuperUser", password = "SYS" }
endpoints = [
    {path = "/mcp/sample", username="SuperUser" password="SYS" }
]

If you are using an HTTP/HTTPS connection, you can also choose to not authenticate here, and instead handle authentication in the requests from the MCP client, this will be shown in the connecting from a client setting below.

Other settings

There are loads more settings to configure here, like the setting up OAuth, using environment secrets rather than hard-coding settings, configuring logging and telemetry and enabling smart tool discovery. To get more details on this, there is a full guide on the iris-mcp-server usage, but for basic usage you just need to define [[iris]] and [mcp].

5. Connecting from an MCP Client

The method for adding an MCP server will differ depending on which client you are using, but in general there will be an option somewhere in your agent customization settings to add an MCP server. For example, to set up an MCP server on GitHub Copilot, type >MCP: Add Server... into the VS Code Search bar, or for Claude Code, you can run claude mcp add....

The first option will likely be a choice between stdio or http(s) transport. These have quite different connection methods so lets tackle them individually.

Stdio

To use a stdio mcp server, you add /path/to/iris-mcp-server as the executable. The default location for this in a docker container is /usr/irissys/bin.

You also need to add arguments for the stdio config file config_stdio.toml and the run command:

/usr/irissys/bin/iris-mcp-server -c config_stdio.toml run

As the authentication is is included in the config file, this is all that is required.

Note, the iris-mcp-server binary has to be on the same machine (or container) as your MCP client!

HTTP

To connect to a remote HTTP MCP server from an MCP client, you first need to start the iris-mcp-server transport by opening a shell and running:

iris-mcp-server -c config_http.toml run

Unlike the STDIO connection, this needs to be continuously running for the HTTP connection to be usable.

With this running, we can then add this server to our MCP client by selecting HTTP as the transport or type, and giving the server URL: http://localhost:8080/mcp/sample.

If we wanted to set authentication at the MCP connection level (rather than the configuration level detailed above), we use standard HTTP authentication headers, like Basic base64(Username:Password) or Bearer <token>. The following Python snippet shows an example of connecting to an MCP server using Langchain's MultiServerMCPClient:

import base64
from langchain_mcp_adapters.client import MultiServerMCPClient

AUTH_HEADER = base64.b64encode(b"SuperUser:SYS").decode("utf-8")
async def get_tools():
    client = MultiServerMCPClient(
        {
            "minimal": {
                "transport": "http",
                "url": "http://localhost:8080/mcp/sample",
                "headers": {"Authorization": f"Basic {AUTH_HEADER}"},
            }
        }
    )

    tools = await client.get_tools()
    return tools

Conclusions

We've reached the end of Part 2 of this series! It has been a long part, so well done if you have got this far. Hopefully this guide will give you the confidence to use the AI Hub preview to start building your own MCP servers inside IRIS, giving agents secure and governed access to your IRIS instance.

If you want to see example code from this article, it is all included in the ai-hub-dev-template on open exchange. This is an example docker project that you can easily clone and use to start working with AI Hub on your local machine. In it, there is sample MCP server and as well as example code to programmatically create the MCP Server Web Application.

New SMART on FHIR v2 Scopes

InterSystems Developer — Sun, 24 May 2026 15:13:59 +0000

In v2026.1 we introduced support for a more robust and real-life secure authorization for your FHIR endpoints.

This is achieved by using SMART on FHIR v2 fine-grained scopes.

Focus - Not SMART in general, rather, the fine-grained scopes; Hands-on easy sample

I have dived into the topic of SMART on FHIR in the past, for example see this article I wrote (with an accompanying Open Exchange app, and related video series).

Also others have discussed this topic, for example @LuisAngel.PérezRamos in his Developing SMART On FHIR Applications with Auth0 and InterSystems IRIS FHIR Server article series, @nicole.Sun in her SMART on FHIR EHR Launch with IRIS for Health article, and @kate.Lau in her two-part Using Postman for testing the OAuth2.0 of the InterSystems FHIR repository.

In addition this Learning Services video - Configuring OAuth for InterSystems FHIR Server - explains this nicely, and even demonstrates part of the latest SMART scope-based result filtering that we'll discuss here.But in the above mentioned articles and samples, we either used InterSystems IRIS itself as the OAuth Server, or a 3rd party cloud OAuth Server (like auth0 by Okta), but in this article and sample I want to do a few things differently -

1. I want to use a 3rd party OAuth Server, but not one you will need to register (and perhaps pay) for. This will be Keycloak.

2. I want to take care of all of the setup for you in a Dockerized sample -

InterSystems IRIS for Health up an running with a FHIR Endpoint defined, including an OAuth client defined, and some Resources in the Repository.
Keycloak up and running with a client corresponding to the IRIS OAuth client.
A Postman Collection to allow for quick testing and demonstration.

3. I want to focus on the relatively newer fine-grained SMART scopes, not the basic ones. This is really the crux of the matter here. The other two above, are enablers for letting us focus just on this item.

SMART Scopes - The Granular Fine-grained Version

Above I generated (thank you NotebookLM) a nice infographic that summarizes the general syntax and usage of SMART scopes.

In particular I want to focus on the filter part, the part in the scopes from the question mark (?).

Here you can use standard FHIR Search syntax, with standard FHIR Search Parameters.

Let's take this example:

Instead of just allowing access (Read & Search in this case) to all categories of Observations, here we are allowing only access to lab results (category=laboratory).

So to illustrate, instead of getting access to a set like this:

We can limit the access to a set like this:

This allows for an ABAC (Attribute-Based Access Control) approach to access FHIR data (see more about this topic in the FHIR docs).

Some local national regulations mandate enforcing this kind of access control, including for example using security tags to limit access to data.

One example is the ONC certification in the US, but other countries have similar demands.

So supporting this is not only important for securing your data, it is also a hard requirement by local law, in a growing number of places.

The Power to Filter (or Not to)

You can control whether, if the FHIR request does not adhere exactly to the scopes, to filter out the unauthorized data and return just what is allowed, or to fail the request and return a 403 error HTTP status.

This setting is in the FHIR endpoint Authorization settings:

Note this is relevant not only to fine-grained scopes but also without using the ? filter. For example if you use _include or the $everything operation, this could filter "whole" Resource Types from the Result Set.

Here's an example to illustrate -

Observation Search Example

Say we issue a Search for Observations, using Basic Authentication, so no SMART Scope are applied.

You can see here we are getting back 793 Resources.

Looking a little closer we can see for example the first one has a category of vital-signs:

And in comparison if I use OAuth 2 authentication and have a scope of user/Observation.rs?category=laboratory, we get only 385 (vs. 793 above) Resources:

And the first one (instead of vital-signs) is usurpingly of a category of laboratory:

A similar comparison can been seen with $everything -

$everyting Example

With Basic Authentication (no Scopes):

We get of course the Patient Resource itself, but also related Resources (per the example above): Encounter, Practitioner, Organization, Condition, Claim, ExplanationOfBenefit, Observation (of various types), MedicationRequest, Immunization, DiagnosticReport

With OAuth (and scopes that include only: user/Patient.rs and user/Observation.rs?category=laboratory):

Here apart from the Patient itself, we only get Observations (laboratory ones), and no other related Resources.

So, 171 Resources vs. 35 after the filtering.

Some Technical Notes

As mentioned you need to be at least on v2026.1 to support this.
Most FHIR Interactions are supported for these kind of Scopes (Create, Read, Update, Delete, Search), some not yet (History, VRead)
As mentioned in the filter search string you can use standard FHIR Search syntax, but some parameters simply won't make sense in this context (like _include), so some might fail the request and others might simply be ignored, see referenced Docs for details.
There are some notes re the $everything and $lastn, again see Docs for details.

Debugging

While using OAuth in general, and with SMART scopes in particular, not everything will work always as expected at first.

Good resources to debug your situation will be the FHIR Server Log (aka FSLOG) and the HTTP Request Log (aka ISCLOG), see more details in the Docs here.

To illustrate here's an example -

Say this time we turned off the filter results settings, and we're trying to Search for all Observation while our scope allows only laboratory.

We will get a 403 Forbidden HTTP status:

And if we turned on the FHIR Server Log, we can see something like this:

Sample Demo

Tackling a topic hands-on always helps and deepens the understanding, so I encourage you to take the related Open Exchange app for a ride. It is a very simple click & go sample, where a docker compose will build and start up everything you need, and includes a sample Postman Collection for you test drive with.

Here's a recording of the demo from a READY 2026 session:

SMARTer Scopes in Action - Live Demo of SMART v2 with InterSystems FHIR Server

Continuous integration in IRIS with Git and Jenkins

InterSystems Developer — Sun, 24 May 2026 14:36:04 +0000

Introduction

In healthcare interoperability environments, InterSystems Health Connect typically contains critical components such as productions, business processes, operations, services, utility classes, routines, and other ObjectScript artifacts. Traditionally, many deployments of these components have been done manually, by copying classes, importing XML, or using administrative tools from the management portal.

While this approach may work in the initial stages, it becomes difficult to maintain as the project grows, when multiple developers are working in parallel, or when repeatable deployments are needed across environments such as development, integration, pre-production, and production.

A more robust alternative is to integrate Health Connect within a continuous integration flow , using Git as the source code repository and Jenkins as the deployment orchestrator.

The aim of this article is to show a practical approach to:

Versioning Health Connect code on GitHub.
Detect only the files modified since the last deployment.
Copy those files to a staging folder.
Load and compile the changes to a Health Connect namespace.
Run the entire process remotely from Jenkins using SSH.

Architecture

For our example, we have configured the following elements:

IRIS for Health Instance

I have deployed InterSystems IRIS for Health on an AWS machine with RHEL10 with its own Apache Server and enabled connectivity via HTTP and SSH.

For development, I have configured Visual Studio Code to work on a local instance of IRIS, on which I will make the code changes that I will then upload to GitHub.

GitHub repository

We have chosen GitHub as our version control system, taking advantage of the extension available in Visual Studio Code. This will allow us to work with branches if necessary.

This element will be key to the CI/CD process since it is where we can obtain the latest code developed for deployment.

Jenkins

For those of you who don't know Jenkins, it's an open-source automation server widely used for continuous integration processes because it has a multitude of plugins that will make the task easier.

Jenkins has a Groovy scripting tool that allows us to implement the necessary steps for the integration process. For this example, we won't get too complicated.

Integration procedure

For this example, we've assumed we're working on an interoperability project with a DEVELOPMENT instance (deployed on the AWS server) where we want to deploy the changes developers make to their local instances for testing. The steps would be roughly as follows:

The developer implements the functionalities in their local instance.
The developer uploads changes to the corresponding branch of the GitHub repository.
The person responsible for the deployment accesses Jenkins and launches a pipeline.
Jenkins connects via SSH to the DEVELOPMENT server.
A Linux script is running on the server.
The script downloads the latest changes from the repository using a git pull.
This script identifies new or modified files that are copied to a server directory.
With the files identified, the script invokes a second script in ObjectScript.
The second script loads and compiles the files into the IRIS for Health instance.
If the upload was successful, the script restarts production.

As you can see, we have chosen a very basic operation, but one that can be quite helpful.

Let's now take a look at the scripts we will run using Jenkins on our DEVELOPMENT server:

#!/usr/bin/env bash 

set -euo pipefail 

  
  
  =========================


  
  
  Configuration


  
  
  =========================


REPO_URL="https://github.com/intersystems-ib/workshop-cicd-demo" 

BRANCH="main" 


  
  
  Local clone used to compare commits


CACHE_REPO="/opt/git-cache/project_repo" 


  
  
  Folder to copy the files to be uploaded into Health Connect


EXPORT_DIR="/projectGit" 


  
  
  File with the latest processed commit


STATE_FILE="${CACHE_REPO}/.last_sync_commit" 


  
  
  CLean up EXPORT_DIR before to copy the new updates


CLEAN_EXPORT_DIR="true" 


  
  
  =========================



  
  
  Validations



  
  
  =========================


if ! command -v git >/dev/null 2>&1; then 

  echo "Error: git is not installed." 

  exit 1 

fi 

mkdir -p "${EXPORT_DIR}" 

mkdir -p "$(dirname "${CACHE_REPO}")" 


  
  
  =========================



  
  
  Clone or update cache folder



  
  
  =========================


if [ ! -d "${CACHE_REPO}/.git" ]; then 

  echo "Cloning repository into cache..." 

  git clone --branch "${BRANCH}" "${REPO_URL}" "${CACHE_REPO}" 

else 

  echo "Updating local cache..." 

  git -C "${CACHE_REPO}" fetch origin 

  git -C "${CACHE_REPO}" checkout "${BRANCH}" 

  git -C "${CACHE_REPO}" reset --hard "origin/${BRANCH}" 

fi 

REMOTE_COMMIT="$(git -C "${CACHE_REPO}" rev-parse HEAD)" 


  
  
  =========================



  
  
  First execution



  
  
  =========================


if [ ! -f "${STATE_FILE}" ]; then 

  echo "First execution." 

  echo "Copying all the contains from branch into ${EXPORT_DIR}..." 

if [ "${CLEAN_EXPORT_DIR}" = "true" ]; then 

    find "${EXPORT_DIR}" -mindepth 1 -maxdepth 1 -exec rm -rf {} + 

  fi 

rsync -av --delete --exclude ".git" "${CACHE_REPO}/" "${EXPORT_DIR}/" 

echo "${REMOTE_COMMIT}" > "${STATE_FILE}" 

  echo "First export finished." 

  exit 0 

fi 

LAST_COMMIT="$(cat "${STATE_FILE}")" 

if [ "${LAST_COMMIT}" = "${REMOTE_COMMIT}" ]; then 

  echo "No updates." 

  exit 0 

fi 

echo "Comparing commits:" 

echo "  anterior: ${LAST_COMMIT}" 

echo "  actual:   ${REMOTE_COMMIT}" 

if [ "${CLEAN_EXPORT_DIR}" = "true" ]; then 

  echo "Cleaning up export folder..." 

  find "${EXPORT_DIR}" -mindepth 1 -maxdepth 1 -exec rm -rf {} + 

fi 


  
  
  =========================



  
  
  Export just added or modified files



  
  
  =========================


while IFS= read -r -d '' status && IFS= read -r -d '' path1; do 

  case "${status}" in 

    M|A) 

      echo "Exporting ${status}: ${path1}" 

      mkdir -p "${EXPORT_DIR}/$(dirname "${path1}")" 

      cp -f "${CACHE_REPO}/${path1}" "${EXPORT_DIR}/${path1}" 

      ;; 

D) 
  # Ignoring deletes 
  echo "Ignoring deleted: ${path1}" 
  ;; 

R*) 
  IFS= read -r -d '' path2 
  echo "Exporting renamed: ${path1} -&gt; ${path2}" 
  mkdir -p "${EXPORT_DIR}/$(dirname "${path2}")" 
  cp -f "${CACHE_REPO}/${path2}" "${EXPORT_DIR}/${path2}" 
  ;; 

*) 
  echo "Change not automatically managed: ${status} ${path1}" 
  ;; 


esac 

done < <(git -C "${CACHE_REPO}" diff --name-status -z "${LAST_COMMIT}" "${REMOTE_COMMIT}") 

echo "${REMOTE_COMMIT}" > "${STATE_FILE}" 

echo "Incremental export concluded in ${EXPORT_DIR}"

echo "Starting file upload and compile in Health Connect" 

(echo '_system'; echo 'SYS'; cat iris.script) | iris session IRISHEALTH 

echo "Compilation successfully finished"

As you can see, this script executes the git pull on our GitHub repository, updates the source code in a directory on the DEVELOPMENT server, detects the changes compared to the last downloaded version, extracts them to a second directory ( /projectGit ) and finally invokes the IRIS script.

(echo '_system'; echo 'SYS'; cat iris.script) | iris session IRISHEALTH

Those first two echo commands will allow us to pass the username and password to the terminal session we need to open to run our ObjectScript script:

zn "DEMO" 

set sc = $SYSTEM.OBJ.LoadDir("/projectGit/src/Demo", "ck", , 1) 

if '$SYSTEM.Status.IsOK(sc) do $SYSTEM.Status.DisplayError(sc) quit 

set production = "Demo.Order.Production" 

set ^Ens.Configuration("csp","LastProduction") = production 

do ##class(Ens.Director).SetAutoStart(production) 

do ##class(Ens.Director).StartProduction(production) 

write !,"Produccion iniciada correctamente: ",production,!

This script is where we import the classes we've identified as modified or created and compile them. If the compilation is successful, we restart the corresponding production environment of our DEMO namespace so that the changes are implemented.

Perfect, we have our scripts, our DEVELOPMENT server and our GitHub, let's configure our Jenkins.

Configuring Jenkins

Before we start creating our pipeline, we must install a plugin that allows us to connect via SSH to our DEVELOPMENT server with our primary username and password.

From the Jenkins configuration, we created an access credential to our DEVELOPMENT server:

And finally we proceed to create the Pipeline.

Within the pipeline configuration, we define the following script that will allow us to deploy:

pipeline {

    agent any
parameters {
    string(name: 'GIT_BRANCH', defaultValue: 'main', description: 'Repository branch')
    string(name: 'REMOTE_HOST', defaultValue: 'ec2-**-**-***-**.**-*****.compute.amazonaws.com', description: 'Remote Host')
    string(name: 'REMOTE_USER', defaultValue: 'ec2-user', description: 'Remote SSH user')
    string(name: 'REMOTE_SCRIPT_NAME', defaultValue: 'shell_script.sh', description: 'Remote script name')
}

environment {
    REPO_URL = 'https://github.com/intersystems-ib/workshop-cicd-demo'
    SSH_CREDENTIALS_ID = 'ssh-healthconnect-remote'
}

stages {
    stage('Checkout') {
        steps {
            git branch: "${params.GIT_BRANCH}", url: "${env.REPO_URL}"
        }
    }

    stage('Validate script') {
        steps {
            sh '''
                set -eu
                test -f shell_script.sh
                chmod +x shell_script.sh
            '''
        }
    }

    stage('Launch remote script') {
        steps {
            sshagent(credentials: ["${env.SSH_CREDENTIALS_ID}"]) {
                sh '''
                    set -eu

                    ssh -o StrictHostKeyChecking=no "${REMOTE_USER}@${REMOTE_HOST}" \
                      "sudo sh '/${REMOTE_SCRIPT_NAME}'" | tee remote_execution.log
                '''
            }
        }
    }
}

post {
    always {
        archiveArtifacts artifacts: 'remote_execution.log', allowEmptyArchive: true
    }
    success {
        echo 'Remote deployment successfully finished.'
    }
    failure {
        echo 'Remote deployment failed. Check remote_execution.log.'
    }
}

}

What does our script do? Very simple, it checks that our GitHub repository exists with its associated branch and then, via SSH, sends the instruction to execute the Linux script that will be in charge of downloading and updating our instance.

Let's see it in action with a small example.

Running the process

Our production is running normally and we want to make a change to one of our components so that the default value shown in one of the parameters is different:

Now we want our TenantId parameter to have the value ZZZ-999, great, let's correct the code we have in our local instance from Visual Studio Code and upload the change to our GitHub.

With our change now pushed to our repository, we can run the pipeline from our Jenkins instance. Let's see the pipeline's output:

Everything is correct; it has detected our change and executed the script successfully.

Let's verify that the parameter has changed and production has restarted successfully.

There we have our new TenantId! A complete and resounding success!

Conclusions and next steps.

As you may have noticed, there are no technological limitations from IRIS for participating in a continuous integration process. You simply need the appropriate scripts that best suit your daily operations.

In this article we have seen a small example of continuous integration with IRIS for Health, but this could be expanded to certain configurations that could be deployed using features such as Configuration Merge.

Give it a try!

Introducing iris-synthetic-data-gen

InterSystems Developer — Sun, 17 May 2026 15:46:56 +0000

Today I have published a new Open Exchange package for generation of Synthetic Data directly into IRIS.

It can be a frustrating process to find decent datasets when you are looking to make a demo app. Maybe the dataset doesn't matter that much, but you still want it to appear somewhat genuine and with several linked tables that are usable directly within IRIS with the neat implicit joins with ->. Maybe you just want linked tables that are easily installable with IPM to benchmark queries, this dataset generation would be perfect.

I have opted to create datasets using Embedded Python, these datasets are configurable by custom config files. The datasets are generated directly with a single IRIS class method, and can be scaled with a multiplier to create however small or large datasets you want without having to measure configs.

At the moment I have four datasets:
- Financial services (e.g. Bank Cards, accounts, transactions )
- Retail (Stores, Products, Users, Inventory)
- Supply Chain (products, sales orders, inventory movement)
- Theme Park management (parks, zones, rides, incidents)

I am not an expert in any of these domains, so I doubt they are super accurate, and the data generation uses python libraries like faker and statistical weighted generation with numpy, so it all feels a bit synthetic.

I will also be honest that, as a side-of-desk project which I couldn't give a huge amount of time to, this project was only made possible by AI. I used AI extensively for the design of datasets and the generation of the code to create the datasets. I supervised, tested for personal use cases and was very involved with the project design, but the code is all AI generated and I have not carefully reviewed the dataset generation process.

For me, this project is a great use case for full "vibe coding" i.e. letting the agent handle the entire coding process. That is to say, the consequences of bugs is low as these datasets are not designed for any production use. The code can largely be judged on the results outputted, in the knowledge that the details or edge cases don't matter.

Its also a good template to make new datasets - the first of the datasets took me a couple of hours of careful planning, discussion with agents, and iterating as to how best to create the dataset and add it to IRIS. Whereas for the last dataset, I could ask the agent "Create a new dataset with retail tables that is configured and generated like the others here", and it did a pretty good job without any real oversight.

I hope this can be useful for some, and feel free to give feedback, contributions or to use it as a template to make your own synthetic datasets!

An Introduction to AI Hub, Part 1: Agents in ObjectScript

InterSystems Developer — Sun, 17 May 2026 15:44:11 +0000

For those of you that weren't at READY last week, you may have missed the exciting announcement that the Early Access Program for AI Hub is officially open. It was announced during an amazing demo from @benjamin.DeBoe and @Jeffrey.Fried, I recommend catching up with this demo when the recording is released! I had the opportunity to play with AI Hub in advance, and thought I might share an introduction with the community.

Before getting into the details, here is a link for the documentation and here is a link to the EAP portal to download AI Hub, its currently available as standalone install kits or container images.

Please note, this is a preview and there are likely to be significant changes before the official release, it is not designed for production use, and you may run into some issues - if you do, raise an issue on the Github page!

Agents

The most exciting feature, for me at least, has been the new ObjectScript agents SDK. You can now create agents and tools directly in ObjectScript, using an intuitive SDK.

Creating an Agent is simple you can give it a system prompt with the XData INSTRUCTIONS component, then just set the provider, model and tools:

Class Sample.Agent Extends %AI.Agent
{
    /// LLM Model
    Parameter MODEL = "gpt-5-nano";

    /// Toolsets that the agent can use
    Parameter TOOLSETS = "Sample.ToolSet";
    
    /// System Prompt
    XData INSTRUCTIONS [ MimeType = text/markdown ]
    {
    # Sample Assistant

    You are a helpful assistant with access to a set of tools to interact with a database of people.
    }

    Method %OnInit() As %Status
    {
        // Set provider with API key from environment variable
        Set key = $System.Util.GetEnviron("OPENAI_API_KEY")  // or whatever
        Set ..Provider = ##class(%AI.Provider).Create("openai", {"api_key": (key)})
        
        Return $$$OK
    }
}

Tools

Tools are even easier to create - its as simple as extending %AI.Tools, after that, all methods, class methods and queries become tools that agents can use. So we can do something like the following:

Class Sample.Tools Extends %AI.Tool [dependsOn=Sample.Person]
{

/// Tool to add a person to the database
Method AddPerson(name As %String, age As %Integer) As %Status{
   Set person = ##class(Sample.Person).%New()
   Set person.Name = name
   Set person.Age = age
   Set sc =  person.%Save()
   Quit sc
}

/// Tool query database for people younger than a specified age
Query GetPeopleYoungerThan(age As %Integer) As %SQLQuery(ROWSPEC = "Name:%String,Age:%Integer") [ SqlProc ]
{
   SELECT Name, Age From Sample.Person Where Age < :age
}

}

Tools can also be organised into toolsets, these are, as the name suggests, sets of tools that can be used to combine many tools from different classes, filter tools by regex matching, add policies and use MCP servers defined outside of IRIS.

In the example below we combine the tools we defined above, Sample.Tools, with a policy which logs tool calls to the terminal (%AI.Policy.ConsoleAudit) and a custom Python MCP server.

Class Sample.ToolSet Extends %AI.ToolSet [DependsOn=Sample.Tools]
{
    XData Definition
    {
        <ToolSet>
            <Description>Sample Toolset</Description>
            
            <Policies>
           <!--Policy to Log tool calls to Console-->
                <Audit Class="%AI.Policy.ConsoleAudit"/>
            </Policies>
            
            <!--ObjectScript Tools-->
            <Include Class="Sample.Tools"></Include>
            
            <!--Python MCP Server created with FastMCP-->
            <MCP Name="PythonServer"> 
                <Stdio Executable="/usr/irissys/bin/irispython" 
                Args="/home/irisowner/dev/src/Python/multiplication_mcp.py" />
            </MCP>
            
        </ToolSet>
    }    
}

Other ObjectScript features

There are a load more of cool features to create super powerful agents, including support for agent skills (%AI.Agent.Skill), delegation of tasks to subagents (%AI.Agent.SubAgent) and tools for creating knowledge bases with RAG (%AI.RAG). You can also create custom audit or authentication policies, to either log tool calls or decide whether they should be allowed.

One very cool feature is that tools and toolsets can be stateful, meaning they retain the state between tool calls. As such, a tool could be called multiple times, with the actions of the previous tool call being retained. For example, a file could be opened once and the contents 'remembered' the next time the tool is called. To use this, define tools with methods (instead of class methods) and save attributes as properties. There's a nice example of this in the documentation.

I've been working with AI hub for well over a month now, and am still overwhelmed by the amount of features, particularly at the advanced end, that I still need to explore.

Template

If you want to start playing around with AI Hub, I published a dev template to the Open Exchange which includes instructions for downloading and building the AI Hub container, and has a few pre-loaded sample classes (you might recognise them from this article). It even has some agent skills, in case you'd like your AI agent of choice to know what's in the documentation before you do!

It even creates an MCP server and has instructions on how to connect to it.

Next time

In my next article, I'll show how you can package your agent tools into an MCP server to connect directly to your data from any MCP client!

IRIS Dockerization and Embedded Python for Data Science — One-Command Setup for Reproducible ML Workflows

InterSystems Developer — Thu, 30 Apr 2026 15:41:55 +0000

1-command only required for an entire IRIS instance for Data Science projects, and leveraging this to compare query methods' speed (Dynamic SQL, Pandas Query, and Globals).

Before joining InterSystems, I worked in a team of web developers as a data scientist. Most of my day-to-day work involved training and embedding ML models in Python-based backend applications through microservices, mainly built with the Django framework and using Postgres SQL for sourcing the data. During development, testing, and deployment, I realized the importance of repeatability of results, both for the model’s inferences and for the performance inside the application, regardless of the hardware being used to run the code.

This naturally went hand in hand with adopting good coding practices, such as modularization to reduce code repeatability and boilerplate, making maintenance easier and speeding up development. For this reason, Docker in particular became an essential tool in our workflow, not only for scalability and ease of deployment, but also to reduce human error and ensure that code behaves the same way everywhere, regardless of the underlying machine.

When I joined InterSystems, I was immediately impressed by the robustness of IRIS as a data platform. Its resilience to human error when following guidelines to create services through productions, the multi-model nature of how information can be stored, and, in particular, the lightning-fast access to data through globals opened my eyes to a different way of thinking about performance and data access patterns, especially when compared to a traditional relational-only mindset.

I was also lucky to join the company (September 2025) at a time when a rich ecosystem of tools was already in place, significantly flattening the learning curve. The VS Code ObjectScript Extension Pack, Embedded Python, the official IRIS Docker images, and the InterSystems Package Manager (IPM) for easily importing ObjectScript packages (https://github.com/intersystems/ipm) quickly became my everyday toolbelt.

After about three months, I felt confident enough working with this stack that I started standardizing my own development environment. In this article, I’d like to share how I set up a fully containerized IRIS instance for Data Science projects using Docker—ready to use Embedded Python out of the box, with all required dependencies installed from both Python’s pip and IPM.

I’ll also use this setup to share some insights on the incredible speed of using globals to query tables, in a practical scenario where the popular gradient boosting model LightGBM is used to train and make inferences on a mock dataset. This allows us to measure inference speed while comparing the different querying approaches available in IRIS.

Some important highlights that will be addressed in this article are how to:

Link custom Python packages during the Docker build process, so they can be imported naturally (e.g. from mypythonpackage import myclassorfunc) inside any Embedded Python methods living on ObjectScript classes, without repetitive boilerplate.
Automatically execute IRIS terminal commands as soon as the container starts, which in this scenario is used to:
- Import custom ObjectScript packages into IRIS.
- Install IPM and, through it, Shavrov’s csvgenpy utility (https://community.intersystems.com/post/csvgenpy-import-any-csv-intersystems-iris-using-embedded-python), used to create and populate new tables from a single CSV file.
- Check whether an IRIS table already exists and, if it doesn’t, populate it using csvgenpy with a CSV file mounted into the container via Docker volumes.

All of this by only running:

  docker-compose up --build

Finally, the repository accompanying this article uses this setup to create a complete IRIS environment with all the tools and data needed to compare different ways of querying the same IRIS table and converting the results into a Pandas DataFrame (NumPy-based), which is typically what gets passed to Python-based machine learning models.

The comparison includes:

Dynamic SQL queries
Pandas querying the table directly
Direct access through globals

For each approach, execution time is measured to quantitatively compare the performance of the different querying methods. This analysis shows that direct global access provides the lowest-latency data retrieval for machine learning inference workloads by far.

At the same time, consistency across querying methods is validated by asserting equality of the resulting Pandas DataFrames, ensuring that identical dataframes (and therefore identical downstream ML predictions) are produced regardless of the query mechanism used.

Project Structure

.
├── docker-compose.yml             # Docker orchestration configuration
├── dockerfile                     # Multi-stage build with IRIS + Python
├── iris_autoconf.sh               # Auto-configuration script for IRIS terminal commands
├── requirements.txt               # Python libraries
├── MockPackage/                   # Custom package
│   ├── MockDataManager.cls        # Data management utilities
│   ├── MockModelManager.cls       # ML model training
│   └── MockInference.cls          # Data retrieval and inference benchmarks
├── python_utils/                  # Custom Python packages
│   ├── __init__.py
│   ├── utils.py                   # ML preprocessing & inference
|   └── querymethods.py            # Methods for Querying IRIS tables
└── dur/                           # Volume for durable data on host machine and container
    ├── data/                      # CSV datasets
    └── models/                    # Trained LightGBM models

Dockerization of IRIS

This section describes the main building blocks used to dockerize a Python-ready IRIS instance. The goal here is not only to run IRIS inside a container, but to do so in a way that makes it immediately usable for Data Science workflows: Embedded Python enabled, Python dependencies installed, ObjectScript packages available through IPM, and data automatically loaded when the container starts.

The setup relies on three main components:

docker-compose.yml to define how the IRIS container is built and run
a multi-stage Dockerfile to prepare Embedded Python and dependencies
an iris_autoconf.sh script to automate IRIS-side configuration at startup

docker-compose.yml

version: '3.8'

services:
  iris:
    build: # How is the image built
      context: . # Path to the directory containing the Dockerfile
      dockerfile: Dockerfile # Name of the Dockerfile
    container_name: iris-experimentation # Name of the container
    ports:
      - "1972:1972"    # SuperServer port
      - "52773:52773"  # Management Portal/Web Gateway
    volumes:
      - ./dur/.:/dur:rw # map host directory to container directory with read-write permissions
    restart: always # Always restart the container if it stops (unless explicitly stopped)
    healthcheck:
      test: ["CMD", "iris", "session", "iris", "-U", "%SYS", "##class(SYS.Database).GetMountedSize()"] # Health check command
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s
    command: --after "/usr/irissys/iris_autoconf.sh" # Run autoconf script after startup

Docker Compose specifies how the IRIS container is built, which ports are exposed, how storage is handled, and what commands are executed at startup. In particular, I want to highlight the following points:

volumes: ./dur/.:/dur:rw

This creates the /dur directory inside the container and maps it to ./dur (relative to the location of docker-compose.yml) on the host machine, with both read and write permissions.

In practice, this means that both the host machine and the container share the same path. This makes it very easy to load files into IRIS and inspect or modify them from the host without any extra copying steps.

In this project, this is how the /data and /models folders are directly made available inside the container under /dur.

command: --after "/usr/irissys/iris_autoconf.sh"

This command allows the execution of a bash script immediately after the container is up and running. The script contains all the commands needed to open an IRIS terminal session and execute any required IRIS-side configuration.

NOTE: The commands in this script are executed every time the container starts. This means that if the container goes down for any reason and restarts (for example, due to restart: always), all the commands in this script will be executed again. If this behavior is not taken into account when writing the script, it can lead to unintended side effects such as reinstalling packages or resetting tables.

dockerfile

# Stage 1: Build stage for installing dependencies
FROM python:3.12-slim AS builder

# Set the working directory
WORKDIR /app

# Copy the requirements file into the image
COPY requirements.txt requirements.txt

# Install the Python dependencies into a temporary location
RUN pip install --no-cache-dir --target /install -r requirements.txt

# Stage 2: Final image with InterSystems IRIS and the installed Python libraries
FROM containers.intersystems.com/intersystems/iris-community:latest-em

# Switch to the root user to install necessary system packages
USER root

# Install the correct Python 3.12 development library for Ubuntu Noble
RUN apt-get update && apt-get install -y libpython3.12-dev wget && \
    rm -rf /var/lib/apt/lists/*

# Set the environment variables for Embedded Python
ENV PythonRuntimeLibrary=/usr/lib/x86_64-linux-gnu/libpython3.12.so
ENV PythonRuntimeLibraryVersion=3.12

# Update the LD_LIBRARY_PATH
ENV LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu:${LD_LIBRARY_PATH}

# Copy the installed Python packages from the builder stage
COPY --from=builder /install /usr/irissys/mgr/python

# Your own Python package
COPY python_utils /usr/irissys/mgr/python/python_utils
ENV PYTHONPATH=/usr/irissys/mgr/python:${PYTHONPATH}


# Copy ObjectScript classes into the image
COPY MockPackage /usr/irissys/mgr/MockPackage
# Copy and set permissions for the autoconf script while still root
COPY iris_autoconf.sh /usr/irissys/iris_autoconf.sh
RUN chmod +x /usr/irissys/iris_autoconf.sh

# Switch back to the default `irisowner` user
USER irisowner

This is a two-stage Dockerfile.

The first stage is a lightweight build stage used to install all Python dependencies listed in requirements.txt into a temporary directory. This keeps the final image clean and avoids installing build tools directly into the IRIS image.

The second stage is based on the official InterSystems IRIS image. Here, the Python runtime library required for Embedded Python is installed, and IRIS is configured so that Embedded Python can recognize both the runtime library and all installed Python packages, including custom ones.

It is worth highlighting the following configuration:

Embedded Python runtime configuration

  ENV PythonRuntimeLibrary=/usr/lib/x86_64-linux-gnu/libpython3.12.so
  ENV PythonRuntimeLibraryVersion=3.12

These environment variables achieve what would otherwise be configured manually through the Management Portal by navigating to:

System Administration → Configuration → Additional Settings → Advanced Memory

and updating the Embedded Python runtime settings. Defining them in the Dockerfile makes the configuration explicit, reproducible, and version-controlled.

Additionally, the classes inside the package "MockPackage" are copied inside the container through:

COPY MockPackage /usr/irissys/mgr/MockPackage

to be later on, automatically imported to IRIS when the the following bash file is executed after the container is up and running.

iris_autoconf.sh

#!/bin/bash
set -e

iris session IRIS <<'EOF'

/* Install IPM/ZPM client if you still need that first
   (your original snippet did this already) */
s version="latest" s r=##class(%Net.HttpRequest).%New(),r.Server="pm.community.intersystems.com",r.SSLConfiguration="ISC.FeatureTracker.SSL.Config" d r.Get("/packages/zpm/"_version_"/installer"),$system.OBJ.LoadStream(r.HttpResponse.Data,"c")

/* Configure registry */
zpm
repo -r -n registry -url https://pm.community.intersystems.com/ -user "" -pass ""
install csvgenpy
quit

/* Import and Compile the MockPackage */
/* The "ck" flags will Compile and Keep the source */
Do $system.OBJ.Import("/usr/irissys/mgr/MockPackage", "ck")

/* Upload csv data ONCE to Table Automatically using csvgenpy */
SET exists = ##class(%SYSTEM.SQL.Schema).TableExists("MockPackage.NoShowsAppointments")
IF 'exists {   do ##class(shvarov.csvgenpy.csv).Generate("/dur/data/healthcare_noshows_appointments.csv","NoShowsAppointments","MockPackage")   }

halt
EOF

This is a bash script that is executed inside the container immediately after startup. It opens an IRIS terminal session using iris session IRIS and runs IRIS-specific commands to perform additional configuration steps automatically.

These steps include importing custom packages whose classes were copied inside the container's storage, installing IPM (available as zpm inside the IRIS terminal), installing IPM packages such as csvgenpy, and using csvgenpy to load a CSV file mounted into the container at /dur/data/healthcare_noshows_appointments.csv to create and populate a corresponding table in IRIS.

NOTE: This script is executed every time the container starts. If this behavior is not considered, it can lead to unintended side effects such as reloading or resetting data. That is why it is important to make the script safe to run multiple times, for example, by checking whether the target table already exists before creating or populating it. This is especially relevant here because the Docker Compose restart policy is set to restart: always, meaning the container will automatically restart and re-execute these commands whenever it goes down.

Packages for Benchmarking

This section introduces the ObjectScript packages used to benchmark different data access strategies in IRIS for a Machine Learning inference workload. The focus here is not on model quality, but on measuring and comparing the time it takes to retrieve data from IRIS, convert it into a Pandas DataFrame, and run inference using a trained LightGBM model.

Each class plays a specific role in this process, from data preparation, to model training, and finally to inference and performance comparison.

MockDataManager.cls

This class contains methods for taking a given CSV file and duplicating its rows to reach a desired dataset size (AdjustDataSize), as well as updating a given IRIS table with the specified CSV (UpdateTableFromCSV). The main purpose of these utilities is to allow testing query and inference time across multiple table sizes in a controlled way.

Note: Throughout this analysis, we focus exclusively on the inference time of a LightGBM model. We are not concerned with model performance metrics such as F1 score, precision, recall, accuracy, or else at this stage.

MockModelManager.cls

In this class, the only relevant method is TrainNoShowsModel. It leverages the data processing pipeline defined in python_utils.utils to prepare the raw data, passed in as a Pandas DataFrame, fit a LightGBM model, and persist the trained model to disk.

The model is saved to a predefined location, which in this setup corresponds to the persistent storage mounted through Docker volumes in docker-compose.yml. This allows the trained model to be reused across container restarts and inference runs without retraining.

MockInference.cls

The core of the performance comparison lives in this class. The process begins by loading the trained LightGBM model weights from the file path specified in the MODELPATH parameter. While this path is currently hardcoded, it serves as a static reference point shared by all inference tests.

RunInferenceWDynamicSQL represents the first approach. It relies on an ObjectScript method called DynamicSQL, which executes a Dynamic SQL statement to filter records by age. The results are packed into a %DynamicArray of %DynamicObjects. This method is then called by the dynamic_sql_query Python function in python_utils/querymethods.py, where the IRIS objects are converted into a structure that can be easily transformed into a Pandas DataFrame.

The entire workflow, including execution time measurement via a Python decorator defined in python_utils/utils.py, is orchestrated inside RunInferenceWDynamicSQL. The resulting DataFrame is then passed through the inference pipeline to produce predictions and measure end-to-end inference latency.

RunInferenceWIRISSQL follows a simpler path. It uses the iris_sql_query method from python_utils/querymethods.py to execute the SQL query directly from Python. The resulting IRIS SQL iterator is transformed directly into a Pandas DataFrame, after which the same inference and timing logic used in the previous method is applied.

RunInferenceWGLobals is the most direct approach, as it queries the underlying data structures (globals) backing the table. It uses the iris_global_query method to fetch data directly from ^vCVc.Dvei.1. This particular global was identified as the DataLocation in the storage definition of the MockPackage.NoShowsAppointments table.

The global name is a result of the hashed storage automatically generated when the table was built from the CSV file.

Finally, the integrity of all three approaches is verified using the ConsistencyCheck method. This utility asserts that the Pandas DataFrames produced by each query strategy are identical, ensuring that data types, values, and numerical precision remain perfectly consistent regardless of the access method used.

Because this check raises no errors, it confirms that Dynamic SQL, direct SQL access from Python, and high-speed global access are all returning exactly the same dataset.

Performance comparison

To evaluate performance, we measured query and inference times for increasing table sizes and report in the table below the average time over 10 runs for each configuration. Query time corresponds to retrieving the data from the database, while inference time corresponds to running the LightGBM model on the resulting dataset.

Rows	DynamicSQL – Query	DynamicSQL – Infer	IRISSQL – Query	IRISSQL – Infer	Globals – Query	Globals – Infer
100	0.003219271	0.042354488	0.001749706	0.043090796	0.001184559	0.043616056
1,000	0.031865168	0.052698898	0.019246697	0.056159472	0.005061340	0.045210719
10,000	0.237553477	0.082497978	0.099582171	0.068728352	0.036206818	0.061128354
100,000	5.279174852	0.189197206	1.122253346	0.177564192	0.535172153	0.175085044
500,000	68.741133046	0.639807224	7.015313649	0.610818386	2.743980526	0.587647438
1,000,000	196.871173100	1.145034313	22.138613220	1.136569023	5.987578392	1.106307745
2,000,000	711.319680452	3.021180152	60.142974615	2.879153728	11.92040014	2.728573560

To characterise how query and inference times scale with respect to table size, we fitted a power-law regression of the form:

Inference Time

Inference time is very similar across all three query methods, which is expected, as the resulting input DataFrame was verified to be identical in all cases.

From the measurements, the model is able to perform inference on approximately 1 million rows in about 1 second, highlighting the high throughput of LightGBM.

The fitted exponent (k ~ 1.3) indicates slightly superlinear scaling of total inference time with respect to the number of rows. This behaviour is commonly observed in large-scale batch processing and is likely attributable to system-level effects such as cache pressure or memory bandwidth saturation, rather than to the algorithmic complexity of the model itself.

The scaling factor "a" is on the order of tens of nanoseconds, reflecting the efficiency of the per-row computation. While the superlinear exponent implies that the marginal cost per additional row increases with table size, this effect becomes noticeable only at large scales (millions of rows), as illustrated by the increasing slope in the log–log plot.

The marginal inference cost can be estimated from the derivative of the fitted model:

Evaluating this expression shows that the per-row marginal inference time increases from approximately 1.9e-7 seconds at 1000 rows to 1.5 e-6 seconds at 1 million rows, remaining firmly in the microsecond range within the observed data regime.

Finally, the fitted constant offset (c ~ 0.08) seconds likely represents a fixed inference overhead, such as model invocation and runtime initialisation, and should be interpreted as a constant cost independent of table size.

Query Time

Query time exhibits substantially different scaling behavior across the three access methods. In contrast to inference time, which is largely independent of the query mechanism, query performance is dominated by the data access strategy and its interaction with storage and execution layers.

The Globals-based approach shows nearly linear scaling (k ~ 1.03), indicating that the cost of retrieving each additional row remains approximately constant across the measured range. This behavior is consistent with sequential access patterns and minimal query-planning overhead, making Globals the most scalable option for large result sets.

The IRISSQL approach exhibits moderately superlinear scaling (k ~ 1.48). While still efficient for moderate table sizes, the increasing marginal cost suggests growing overhead from SQL execution, query planning, or intermediate result materialization as the number of rows increases.

The DynamicSQL approach displays the most pronounced superlinear scaling (k ~ 1.82), resulting in rapidly increasing query times at larger scales. This behavior explains the steep slope observed in the plot and indicates that DynamicSQL incurs significant additional overhead as result size grows, making it the least scalable method for large batch queries.

Although the fitted scaling factors "a" are numerically small, they must be interpreted jointly with the exponent "k". In practice, the exponent dominates the asymptotic behavior, which is why DynamicSQL, despite a small "a", becomes significantly slower at large table sizes.

The fitted constant term "c" represents the fixed query overhead. For IRISSQL, "c" is close to zero, indicating a small startup cost. This overhead is even smaller for the Globals-based approach, where the fitted value is slightly negative, effectively suggesting a zero fixed cost. This behavior is expected, as data retrieval via a global key proceeds directly without additional query planning or execution overhead.

In contrast, the relatively large constant offset observed for DynamicSQL indicates a substantial fixed overhead, likely associated with query preparation or execution setup. This fixed cost penalizes performance across all table sizes and becomes particularly impactful at both small and large scales.

Overall, these results highlight that query time, unlike inference time, is highly sensitive to the data access method, with Globals offering near-linear scalability, IRISSQL providing a balanced middle ground, and DynamicSQL exhibiting poor scalability for large result sets.

Please refer to the following repository for more details:

https://github.com/JorgeIvanJH/IRIS_dockerization.git

Video demo here:

https://youtu.be/IcShNKQ4jIk

If you have any questions or notice any mistakes, please don’t hesitate to reach out.

Thank you!

Vector Search with Embedded Python in InterSystems IRIS

InterSystems Developer — Thu, 30 Apr 2026 15:35:55 +0000

One objective of vectorization is to render unstructured text more machine-usable. Vector embeddings accomplish this by encoding the semantics of text as high-dimensional numeric vectors, which can be employed by advanced search algorithms (normally an approximate nearest neighbor algorithm like Hierarchical Navigable Small World). This not only improves our ability to interact with unstructured text programmatically but makes it searchable by context and by meaning beyond what is captured literally by keyword.

In this article I will walk through a simple vector search implementation that Kwabena Ayim-Aboagye and I fleshed out using embedded python in InterSystems IRIS for Health. I'll also dive a bit into how to use embedded python and dynamic SQL generally, and how to take advantage of vector search features offered natively through IRIS.

Environment Details:

OS: Windows Server 2025
InterSystems IRIS for Health 2025.1
VS Code / InterSystems Server Manager
Python 3.13.7
Python Libraries: pandas, ollama, iris*
Ollama 0.12.3 and model all-minilm
Dynamic SQL
Sample database of unstructured text (classic poems)

Process:

0. Setup the environment; complete installs

Define an auxiliary table
- The embeddings table User.SamplePoetryVectors has a foreign key on User.SamplePoetry as well as an EMBEDDING property of type %Library.Vector. Ollama all-minilm generates embeddings of 384 dimensions, so we imposed a length constraint accordingly.
  - *Note that because the goal is to ultimately take advantage of IRIS' native HNSWIndex and IRIS' native vector search methods, we must have a column of type %Library.Vector (or %Library.Embedding) of fixed length that is of type decimal or double upon which to index.

Define a `RegisteredObject` class for our vectorization methods, which will be written in embedded python. First let's focus on a `VectorizeTable()` method, which will contain a driver function (of the same name) and a few supporting process functions all written in Python.

The driver function walks through the process as follows:
1. Load from IRIS into a Pandas Dataframe (via supporting function load_table())
2. Generate an embedding column (via supporting class method GetEmbeddingString, which will later be used to generate embeddings for queries as well)
  - Convert the embedding column to a string that's compatible with IRIS vector type
3. Write the dataframe into the auxiliary able
4. Create an HNSW index on the auxiliary table
The VectorizeTable() class method then simply calls the driver function:
Let's examine it step-by-step:

Load the table from IRIS into a Pandas Dataframe

def load_table(sample_size='') -> pd.DataFrame:

    sql = f"SELECT * FROM SQLUser.SamplePoetry{f' LIMIT {sample_size}' if sample_size != '*' else ''}"

    result_set = iris.sql.exec(sql)

    df = result_set.dataframe()
<span class="mention"># Entries without text will not be vectorized nor searchable</span>
<span class="mention">for</span> index, row <span class="mention">in</span> df.iterrows():
    <span class="mention">if</span> row[<span class="mention">'poem'</span>] == <span class="mention">' '</span> <span class="mention">or</span> row[<span class="mention">'poem'</span>] <span class="mention">is</span> <span class="mention">None</span>:
        df = df.drop(index)

<span class="mention">return</span> df</code></pre></li><li data-list-item-id="eefcc1d39f96038e122e461e7526ba90b">This function leverages the <code>dataframe()</code> method of <a href="https://docs.intersystems.com/irislatest/csp/documatic/%25CSP.Documatic.cls?LIBRARY=%25SYS&amp;CLASSNAME=%25SYS.Python.SQLResultSet#METHOD_dataframe" target="_blank">the embedded python SQLResultSet objects</a></li><li data-list-item-id="ea4ede0aa2a09dabf2902a06278aa79df"><code>load_table()</code> accepts an optional <code>sample_size</code> argument for testing purposes. There's also a filter for entries without unstructured text. Though our sample database is curated and complete, some use cases may seek to vectorize datasets for which one cannot assume each row will have data for all columns (for example survey responses with skipped questions). As opposed to implementing a "null" or empty vector, we chose to exclude such rows from vector search by removing them at this step in the process.</li><li data-list-item-id="e821e2611fe70c080e7a022d8f3c21f1b">*Note that <code>iris</code> is the <a href="https://docs.intersystems.com/irisforhealthlatest/csp/docbook/DocBook.UI.Page.cls?KEY=GEPYTHON_reference" target="_blank">InterSystems IRIS Python Module</a>. It functions as an API to access IRIS classes, methods, and to interact with the database, etc.</li><li data-list-item-id="ea69a4d07e45b85e56b943c780424979e">*Note that <code>SQLUser</code> is the <a href="https://docs.intersystems.com/irislatest/csp/docbook/DocBook.UI.Page.cls?KEY=GSQL_tables#GSQL_tables_schemadefault" target="_blank">system-wide default schema</a> which <a href="https://docs.intersystems.com/irislatest/csp/docbook/DocBook.UI.Page.cls?KEY=GOBJ_defpersobj#GOBJ_defpersobj_sqlproj_pkg" target="_blank">corresponds to the default package</a><code>User</code>.</li></ul></li><li class="ck-list-marker-bold" data-list-item-id="e9ca354b9ae137911dc78afd93da5b132"><h4><strong>Generate an embedding column (support method)</strong></h4><ul><li data-list-item-id="ea097b9c25eb08247c89cab9fcfac9e6d"><pre class="codeblock-container" idlang="0" lang="ObjectScript" tabsize="4"><code class="language-plaintext language-cls hljs cos"><span class="mention">ClassMethod</span> GetEmbeddingString(aurg <span class="mention">As</span> <span class="mention">%String</span>) <span class="mention">As</span> <span class="mention">%String</span> [ Language = python ]

{

  import iris

  import ollama

response = ollama.embed(model='all-minilm',input=[ aurg ])

  embedding_str = str(response.embeddings[0])

return embedding_str

}

We installed Ollama on our VM, loaded the all-minilm embedding model, and generated embeddings using Ollama’s Python library. This allowed us to run the model locally and generate embeddings without an API key.
GetEmbeddingString returns the embedding as a string because TO_VECTOR by default expects the data argument to be a string, more on that to follow.
*Note that Embedded Python provides syntax for calling other ObjectScript methods defined within the current class (similar to self in Python). The earlier example uses iris.cls(name) syntax to get a reference to the current ObjectScript class and invoke GetEmbeddingString (ObjectScript method) from VectorizeTable (Embedded Python method inside ObjectScript method).

Write the embeddings from the dataframe into the table in IRIS
- ```
# Write dataframe into new table

print("Loading data into table...")

for index, row in df.iterrows():

    sql = iris.sql.prepare("INSERT INTO SQLUser.SamplePoetryVectors (ID, EMBEDDING) VALUES (?, TO_VECTOR(?, decimal))")

    rs = sql.execute(row['id'], row['embedding'])

print("Data loaded into table.")
```
- Here, we use Dynamic SQL to populate SamplePoetryVectors row-by-row. Because earlier we declared the EMBEDDING property to be of type %Library.Vector we must use TO_VECTOR to convert the embeddings to IRIS' native VECTOR datatype upon insertion. We ensured compatibility with TO_VECTOR by converting the embeddings to strings earlier.
  - The iris python module again allows us to take advantage of Dynamic SQL from within our Embedded Python function.
Create a HNSW Index
- ```
# Create Index

iris.sql.exec("CREATE INDEX HNSWIndex ON TABLE SQLUser.SamplePoetryVectors (EMBEDDING) AS HNSW(Distance='Cosine')")

print("Index created.")
```
- IRIS will natively implement a HNSW graph for use in vector search methods when an HNSW index is created on a compatible column. The vector search methods available through IRIS are VECTOR_DOT_PRODUCT and VECTOR_COSINE. Once the index is created, IRIS will automatically use it to optimize the corresponding vector search method when called in subsequent queries. The parameter defaults for an HNSW index are Distance = Cosine, M = 16, and efConstruction = 200.
- Note that VECTOR_COSINE implicitly normalizes its input vectors, so we did not need to perform normalization before inserting them into the table in order for our vector search queries to be scored correctly!

Implement a `VectorSearch()` class method

Generate an embedding for the query string

# Generate embedding of search parameter

search_vector = iris.cls(name).GetEmbeddingString(aurg)

Reusing the class method GetEmbeddingString

Prepare and execute a query that utilizes `VECTOR_COSINE`

# Prepare and execute SQL statement

stmt = iris.sql.prepare(

        """SELECT top 5 p.poem, p.title, p.author 

        FROM SQLUser.SamplePoetry AS p 

        JOIN SQLUser.SamplePoetryVectors AS v 

        ON p.ID = v.ID 

        ORDER BY VECTOR_COSINE(v.embedding, TO_VECTOR(?)) DESC"""

)

results = stmt.execute(search_vector)

We use a JOIN here to combine the poetry text with its corresponding vector embedding so we can rank results by semantic similarity.

Output the results

results_df = pd.DataFrame(results)

pd.set_option('display.max_colwidth', 25)

results_df.rename(columns={0: 'Poem', 1: 'Title', 2: 'Author'}, inplace=True)


print(results_df)

Utilizes formatting options from pandas to tweak how it appears in the IRIS Terminal:

KMS . Introduction to its use in IRIS and an example of setup on AWS EC2 system

InterSystems Developer — Sun, 26 Apr 2026 16:19:04 +0000

IRIS can use a KMS (Key Managment Service) as of release 2023.3. Intersystems documentation is a good resource on KMS implementation but does not go into details of the KMS set up on the system, nor provide an easily followable example of how one might set this up for basic testing.

The purpose of this article is to supplement the docs with a brief explanation of KMS, an example of its use in IRIS, and notes for setup of a testing system on AWS EC2 RedHat Linux system using the AWS KMS. It is assumed in this document that the reader/implementor already has access/knowledge to set up an AWS EC2 Linux system running IRIS (2023.3 or later), and that they have proper authority to access the AWS KMS and AWS IAM (for creating roles and polices), or that they will be able to get this access either on their own or via their organizations Security contact in charge of their AWS access.

What is KMS and what does it do for IRIS?:

KMS means Key Management Service. Briefly, it provides an external secure method of encrypting and decrypting IRIS encryption keys through a trusted service, the KMS.

In prior implementation, when using unattended startup, IRIS would never store unencrypted encryption keys; IRIS would encrypt a key with an encrypted copy of the key encryption key in that key itself. It would then store a user ID and password in IRIS to unencrypt the encrypted key encryption key. This leaves an unencrypted copy of the user ID and password stored in an IRIS database, which leaves extra burden on IRIS managers of securing that. The key encryption key is encrypted/decrypted by a symmetric key that is based on a key admin’s password using PBKDF2 (Password-Based Key Derivation Function 2). So the key that encrypts the key encryption key is never stored anywhere – it’s derived on the fly when a key admin supplies their password. Since there can be multiple admins for keys in a given key file we store in the key file one encrypted copy of the key encryption key (per admin) and then a single encrypted copy of each database/data element encryption key (encrypted with the key encryption key).

With KMS we do not store the id and password in IRIS. When we create the encryption key with KMS we get an encrypted encryption key, and the KMS keeps the key encryption key for us. We reach out to the kms server with the encrypted encryption key. the kms server decrypts the encryption key. The decrypted key is sent back to us and stored in memory. The communications are secured using TLS.

We don't ever have access to the raw key encryption key. We use it as a service via kms. The key encryption key stays on the kms server. This helps with key management and key security.

Current implementation (as of 1/22/2024) of KMS is Cloud Vendor Specific

In AWS you must specify creation of a symmetric key.

In Azure you must specify creation of an RSA key

Future implementation my include google KMS.

---

Example of workflow setting up new encryption key in IRIS using KMS:

The following assumes you have set up an IRIS system to access an AWS KMS server and your instance has been authorized to access the keys there and you have set up a key for use. (See Setup Notes following this example for an example of setting up KMS on AWS to connect with an AWS EC2 RedHat Linux instance.)

1.%SYS>D ^EncryptionKey

2.Create New Key

3.Name the key

4.Use KMS: yes

Here you specify properties of the key. Choose backup if you want a regular encryption key made to backup this KMS key. This is the only place you can do this. Treat this backup as you would a normal Encryption key.

5. Select AWS for the kms server

6. Get the key ID and the region from your AWS Key Managed Service console

7. Env Key ; you should not need to specify anything here if your system is set up correctly (per this article). See AWS docs for further details if necessary for your needs. Leave blank for the purpose of simplifying this for testing example.

8. You should receive a message like:

Encryption key file created: iriskmstest1
Encryption key created via KMS: 87A85627-9F8C-11EE-8839-0608ECAD1BAF

This key is NOT activated.

Key Activation and use are then usual encryption key setup steps.

If there are issues with the activation at startup it will error and go into interactive mode

For interactive startup if you pass in a kms key it will not prompt for username or password

If you put in the backup key (generated in step 14 above) then it will ask for the username and password you created at key creation time (just like normal key)

If there are issues you will see errors in your startup, or logged in messages.log if silent startup.

In general, your IRIS system does not need to be on AWS or other cloud system, it accesses the KMS for the key over TLS.

IRIS uses credentials of current user when accessing the KMS server, so you need to make sure that user has access to KMS

the AWS key policy defines who can use the key on AWS. See following setup notes for an example.

----

Setup Notes: Getting an AWS EC2 Linux system running IRIS to work with an AWS KMS:

(The following assumes you already have an AWS EC2 RedHat Linux system running an IRIS version that supports KMS)

To set up the AWS EC2 system to use the AWS KMS server:

Follow Setup instructions in following link to install the AWS CLI on your EC2 system: Install or update the latest version of the AWS CLI - AWS Command Line Interface (amazon.com)

There are instructions for different OS types. For the purpose of this instruction set I used an AWS RedHat Linux system. It was fairly strait forward to follow that doc to install the AWS CLI on the system.

I also had to use 'sudo yum install unzip' to install unzip on the system in order to follow the instructions which had me use unzip on the AWS client download zip file.

Here are the steps to create a key that could be used by an IRIS instance for encryption key encryption:

1. In AWS Mgmnt Console go to Key Management Service.

2. Click on Customer Managed Keys

3. Click on Create Key

5. Accept the Defaults

6. Enter an Alias; this is the name for the key

7.Key Admin Options: default policy

8. Click Finish

The IRIS instance will also need to be authorization to use the KMS key. This is done either by running the instance as a user who has authenticated to AWS and is authorized to use the key, specifying a credentials file with the AWS_SHARED_CREDENTIALS_FILE environment variable or by assigning to the EC2 itself an IAM role that either has a policy attached to it that allows key usage or that has an explicit allowance specified in the key policy itself.

For the purpose of this instruction set we are following the 3rd as ISC Development has suggested this would be the most commonly used by customers in AWS. In the following we will create an IAM role that can be assigned to the EC2 instance itself. The role can have a policy attached to it that gives it very targeted privileges to access a given key in the KMS (or even just allow specific operations with the key). We are only exploring the most simple process to give us something to use for testing...

Here are the steps for Authorizing an Instance of IRIS on an AWS EC2 system to use the key on the KMS server:

1.In AWS Managment Console go to Key Management Service

2. Under "Customer managed keys" click on the Key ID of the key you want to use.

3. In the "General configuration" section click the "Copy" icon next to the ARN to copy the ARN to the clipboard. Paste this value somewhere to use later in the policy configuration.

4. In AWS Mgmnt Console go to IAM.
5. Under "Access Management">"Policies" click "Create policy".
6. Under "Select a service" choose KMS from the drop-down list. Click "Next".
7. Under "Actions allowed" click on the "Write" access level expander. Check the "Decrypt" and "Encrypt" checkboxes.
8. Under "Resources" click on the "Add ARNs" link.
9. Paste the entire ARN from Step 3 above into the "Resource ARN" text field. Click "Add ARNs". Click "Next".
10. Under "Policy details" provide a policy name and, if desired, a policy description. Click "Create policy".

11. In IAM under "Access Management">"Roles" click "Create role".
12. Under "Trusted entity type" click "AWS service". Under "Use case" select EC2 from the drop-down list. Click "Next".
13. Under "Permissions policies" start typing the policy name from Step 10 until it appears in the list. Click the checkbox next to it. Click "Next".
14. Under "Role details" provide a role name. Click "Create role".

15. In AWS Mgmnt Console go to EC2. Navigate to "Instances">"Instances".
16. If EC2 instance already exists:
a. Click checkbox next to instance name.
b. Click "Actions">"Security">"Modify IAM role".
c. Choose the role from Step 15 from the drop-down list.
d. Click "Update IAM role".
16. If launching new EC2 instance:
a. Click "Launch instances".
b. Under "Advanced details" choose role from Step 15 in "IAM instance profile" drop-down list.

17.You can now use the kms key in ^EncryptionKey

Notes:
After creating policy/role you might need to refresh the Mgmt Console for these new resources to show up.

---

Supplemental:

Classes methods of interest:

%SYSTEM.Encryption.KMSCreatEncryptionKey()

%SYSTEM.Encryption.ActivateEncryptionKey() ;just supply the kms key, no need for username or password

do ReadFile^EncryptionKey(<key>,.data) zw data ;it will be obvious if the key is kms type from the data returned.

Doc link:

Key Management Tasks | InterSystems IRIS for Health 2023.3

IRIS SIEM System Integration with Crowdstrike Logscale

InterSystems Developer — Sun, 26 Apr 2026 16:17:27 +0000

IRIS makes SIEM systems integration simple with Structured Logging and Pipes!

Adding a SIEM integration to InterSystems IRIS for "Audit Database Events" was dead simple with the Community Edition of CrowdStrike's Falcon LogScale, and here's how I got it done.

CrowdStrike Community LogScale Setup

Getting Started was ridiculously straight forward and I had the account approved in a couple of days with the following disclaimer:

Falcon LogScale Community is a free service providing you with up to 16 GB/day of data ingest, up to 5 users, and 7 day data retention, if you exceed the limitations, you’ll be asked to upgrade to a paid offering. You can use Falcon LogScale under the limitations as long as you want, provided, that we can modify or terminate the Community program at any time without notice or liability of any kind.

Pretty generous and a good fit for this implementation, with the caveat all good things can come to an end I guess, cut your self an ingestion token in the UI and save it to your favorite hiding place for secrets.

Python Interceptor - irislogd2crwd.py

Wont go over this amazing piece of software engineering in detail, but it is as simple as a python implementation that accepts STDIN, breaks up what it sees into events, and ships them off to the SIEM platform to be ingested.

#!/usr/bin/env python
import json
import time
import os
import sys
import requests
import socket
from datetime import datetime
from humiolib.HumioClient import HumioIngestClient


input_list = sys.stdin.read().splitlines() # From ^LOGDMN Pipe!
for irisevent in input_list:
    # Required for CRWD Data Source
    today = datetime.now()
    fqdn = socket.getfqdn()

    payload = [
        {
            "tags": {
                "host": fqdn,
                "source": "irislogd"
            },
                "events": [
                {
                    "timestamp": today.isoformat(sep='T',timespec='auto') + "Z",
                    "attributes": {"irislogd":json.loads(irisevent)} 
                }
            ]
        }
    ]

    client = HumioIngestClient(
        base_url= "https://cloud.community.humio.com",
        ingest_token= os.environ["CRWD_LOGSCALE_APIKEY"]
    )
    ingest_response = client.ingest_json_data(payload)

You will want to chmod +x this script and put it where irisowner can enjoy it.

InterSystems IRIS Structured Logging Setup
Structured Logging in IRIS is documented to the 9's, so this will be a Cliff Note to the end state of configuring ^LOGDMN. The thing that caught my attention in the docs is probably the most unclear part of the implementation, but the most powerful and fun for sure.

After:
ENABLING the Log Daemon, CONFIGURING the Log Daemon and STARTING Logging your configuration should look like this:

%SYS>Do ^LOGDMN
1) Enable logging
2) Disable logging
3) Display configuration
4) Edit configuration
5) Set default configuration
6) Display logging status
7) Start logging
8) Stop logging
9) Restart logging

LOGDMN option? 3
LOGDMN configuration

Minimum level: -1 (DEBUG)
 Pipe command: /tmp/irislogd2crwd.py
       Format: JSON
     Interval: 5

/tmp/irislogd2crwd.py  # Location of our chmod +x Python Interceptor
JSON                   # Important

Now that we are logging somewhere else, lets just pump up the verbosity in the Audit Log and enable all the events since somebody else is paying for it.

Stealing from @sylvain.Guilbaud 's post:

CrowdStrike LogScale Event Processing

It wont take long to get the hang of, but the Search Console is the beginning of all good things with setting up customized observability based on your events. The search pane with filter criteria displays in the left corner, the available attributes on the left sidebar and the matching events in the results pane in the main view.

LogScale uses The LogScale Query Language (LQL) to back the widgets, alerts and actions.

I suck at visualizations, so I am sure you could do better than below with a box of crayons, but here is my 4 widgets of glory to put a clown suit on the SIEM events for this post:

If we look under the hood for the "Event Types" widget, the following LQL is only needed behind a time series graph lql:

timechart(irislogd.event)

So we did the thing!

We've integrated IRIS with the Enterprise SIEM implementation and the Security Team is "😀 "

The bonus here are the things that are also accomplished with the exact same development pattern as above:

Notifications
Actions
Scheduled Searches
Scheduled Daily Reports

From FHIR Events to Explainable Agentic AI: Building a Clinical Follow‑Up Demo with InterSystems IRIS for Health

InterSystems Developer — Wed, 22 Apr 2026 16:50:36 +0000

10:47 AM — Jose Garcia's creatinine test results arrive at the hospital FHIR server.
2.1 mg/dL — a 35% increase from last month.

What happens next?

Most systems: ❌ The result sits in a queue until a clinician reviews it manually — hours or days later.
This system: 👍 An AI agent evaluates the trend, consults clinical guidelines, and generates evidence-based recommendations — in seconds, automatically.

No chatbot. No manual prompts. No black-box reasoning.

This is event-driven clinical decision support with full explainability:

✅ Triggered automatically by FHIR events
✅ Multi-agent reasoning (context, guidelines, recommendations)
✅ Complete audit trail in SQL (every decision, every evidence source)
✅ FHIR-native outputs (DiagnosticReport published to server)

Built with:

InterSystems IRIS for Health — Orchestration, FHIR, persistence, vector search
CrewAI — Multi-agent framework for structured reasoning

You'll learn: 🖋️ How to orchestrate agentic AI workflows within production-grade interoperability systems — and why explainability matters more than accuracy alone.

🎬 What This Demo Produces

When Jose's abnormal creatinine observation arrives, the system automatically generates:

INPUT: FHIR Observation (creatinine 2.1 mg/dL, status: HIGH)

OUTPUT: FHIR DiagnosticReport containing:

Risk Level: Medium-High (confidence: 85%)
Recommendations:
- ⚠️ Repeat creatinine in 7–14 days
- 💊 Review nephrotoxic medications (currently on Ibuprofen)
- 📊 Monitor renal function closely
Evidence Used:
- Patient context: CKD Stage 3 + progressive creatinine rise (>30%)
- Clinical guidelines: KDIGO section on AKI management in CKD
- Lab trend analysis: 1.6 → 1.9 → 2.1 mg/dL over 3 months

AUDIT TRAIL: Every decision, recommendation, and evidence citation persisted in SQL tables for compliance and review.

🎯 What Problem Does This Solve?

Most AI demos in healthcare focus on:

Chat interfaces for asking questions
Unstructured text outputs
Opaque reasoning ("trust the AI")

In real clinical environments, what matters is:

Reacting to clinical events automatically
Understanding complete patient context
Providing explainable recommendations with evidence
Persisting decisions for audit and compliance

This demo answers a simple but realistic question:

What happens when a new abnormal lab result arrives — and how can we automate the initial clinical assessment while maintaining transparency?

🧪 Demo Scenario: CKD + Rising Creatinine

The demo is based on a common healthcare use case:

Patient: Jose Garcia (MRN-1000001)

Conditions: Chronic Kidney Disease (CKD Stage 3), Hypertension
Medications: Ibuprofen (NSAID), Lisinopril
Lab history:
- 3 months ago: 1.6 mg/dL
- 1 month ago: 1.9 mg/dL
- Today: 2.1 mg/dL ← triggers workflow

The >30% progressive increase requires clinical follow-up.

Instead of waiting for manual review, the system automatically:

Detects the event (FHIR Observation POST)
Retrieves patient context (conditions, medications, lab history)
Consults clinical guidelines via RAG (vector search)
Performs agentic reasoning across three specialized agents
Produces explainable recommendations with evidence citations

⏱️ From Event to Evidence: The Complete Journey

Follow a single lab result through the system:

FHIR Observation posted to IRIS server
Interoperability Production triggered
Context Agent queries patient history from FHIR
Guidelines Agent searches vector database (clinical documents)
Reasoning Agent synthesizes 3 recommendations
Results persisted to SQL (Cases, CaseRecommendations, CaseEvidences)
FHIR DiagnosticReport published to server
Complete — Full audit trail available for review

From event to actionable recommendations.

🧠 Architecture Overview

Key Principle

InterSystems IRIS for Health is the orchestrator and system of record.

The AI agents are external capabilities that are governed, triggered, and integrated by the IRIS platform. IRIS owns the data, the workflow, and the audit trail — the agents provide specialized reasoning.

High-Level Flow

Key steps:

FHIR Observation → POSTed to IRIS FHIR server
Interaction Strategy → Detects clinical event
Interoperability Production → Orchestrates workflow
Business Operation → Calls Agentic AI REST service (FastAPI)
Agents Execute → Context retrieval, guideline search, reasoning
Results Return → Structured JSON back to IRIS
Persistence → SQL tables store cases, recommendations, evidence
Publishing → FHIR DiagnosticReport created and stored

Visual Components

The demo includes a Gradio web UI for interactive demonstration:

Post lab values and trigger the workflow
Watch real-time agent progress
View recommendations and evidence citations
Query SQL audit tables
Access IRIS Production message viewer

This makes the complete flow visible and understandable.

🤖 Why CrewAI? Understanding Multi-Agent Architecture

CrewAI is a multi-agent orchestration framework that enables specialized AI agents to collaborate on complex tasks.

In this demo, three agents work sequentially:

1. Context Agent

Role: Gather patient clinical history from FHIR server

Action:

Fetch patient demographics and conditions
Retrieve historical lab results (creatinine trends)
Collect active medications
Identify risk factors (NSAID use + CKD)

Output: Structured patient context for reasoning

2. Guidelines Agent

Role: Search clinical knowledge base using RAG (Retrieval-Augmented Generation)

Action:

Query IRIS vector database with semantic search
Find relevant guideline sections (clinical protocols, etc.)
Retrieve evidence chunks with similarity scores
Provide citations for recommendations

Output: Evidence-based clinical guidance

3. Reasoning Agent

Role: Synthesize recommendations from context + guidelines

Action:

Analyze lab trends (>30% increase = significant)
Identify risk factors (CKD + NSAID + progressive rise)
Apply clinical decision rules
Generate structured recommendations with confidence levels

Output: Risk assessment + actionable follow-up plan

Why Multi-Agent Instead of Single LLM Call?

Agentic workflows provide:

✅ Better structured reasoning — Each agent has a focused responsibility
✅ Tool use — Agents can query FHIR, search vector databases, analyze trends
✅ Explainable decision chains — Each step is traceable
✅ Separation of concerns — Context ≠ Guidelines ≠ Reasoning

Critical: IRIS orchestrates the agents — CrewAI is used as a library, not the platform. IRIS owns persistence, orchestration, FHIR integration, and audit trails.

🔄 Interoperability Production

The workflow is managed by three IRIS components:

Business Service (FHIRObservationIn)
Triggered automatically when FHIR Observation is POSTed
Business Process (FollowUpAI)
Orchestrates three-step workflow:
1. Call agent service
2. Persist results to SQL
3. Publish DiagnosticReport
Business Operations
- ClinicalAgenticOperation → REST call to FastAPI/CrewAI
- ClinicalAiPersistence → SQL table writes
- ClinicalReportPublisher → FHIR DiagnosticReport POST

🔍 Explainability: Proving the AI's Reasoning

One of the most critical aspects of clinical AI is proving why a recommendation was made.

IRIS persists everything in a minimal, queryable SQL model:

Cases — What happened (patient, observation, risk level, confidence)
CaseRecommendations — What to do (action type, description, timeframe)
CaseEvidences — Why (guideline citations, similarity scores, text excerpts)

Example Queries

"What cases were evaluated today?"

SELECT
  CaseId,
  PatientRef,
  RiskLevel,
  Confidence,
  ReasoningSummary
FROM clinicalai_data.Cases
WHERE CreatedAt >= CURRENT_DATE
ORDER BY CreatedAt DESC

Result:

CaseId: CSE-20260108-001
PatientRef: Patient/1 (Jose Garcia)
RiskLevel: medium-high
Confidence: high
ReasoningSummary: The patient with stage 3 chronic kidney disease and hypertension demonstrates a sustained and progressive increase in serum creatinine over 90 days...

"Why did the agent recommend nephrotoxic medication review?"

SELECT
  e.GuidelineId,
  e.Similarity,
  e.Excerpt
FROM clinicalai_data.CaseEvidences e
WHERE e.CaseId = 'b344f121-db68-4cd6-8877-1855c3d547ff'
ORDER BY e.Similarity DESC

Result:

GuidelineId: ckd_creatinine_guideline_demo
Similarity: 0.66
Excerpt: "Recommended actions include repeat serum creatinine testing within 7–14 days, review of current medications for nephrotoxicity, assessment of contributing factors, and close monitoring of renal function."

Every recommendation has:

The clinical context used
The guidelines consulted
The similarity scores showing relevance
The reasoning chain from data to decision

You can answer "Why did the AI recommend this?" with SQL queries and evidence citations.

🩺 Publishing Results as FHIR DiagnosticReport

The final step closes the loop: AI outputs become part of the clinical record.

The system publishes a FHIR DiagnosticReport containing:

Subject: Patient reference (Jose Garcia)
Result: Link to triggering Observation (creatinine 2.1)
Conclusion: Risk level + reasoning summary
PresentedForm: Human-readable recommendations (Base64-encoded)
Extensions: Case ID, confidence score, model metadata

This makes the AI output:

Interoperable — Standard FHIR resource
Consumable — Accessible via FHIR API by EHRs, portals, apps
Auditable — Part of the permanent clinical record
Queryable — GET /DiagnosticReport?result=Observation/14

The DiagnosticReport is not a separate "AI system output" — it's a first-class clinical document that follows the same standards as lab reports and radiology findings.

🚀 Try It Yourself

Quick Start (15 minutes):

Clone the repository

   git clone https://github.com/intersystems-ib/iris-health-fhir-agentic-demo
   cd iris-health-fhir-agentic-demo

Start IRIS container

   docker-compose up -d

Load sample patient data (Jose Garcia with CKD history)
Follow the README setup instructions
Run the Gradio UI

   python run_ui.py

Open browser to http://localhost:7860

POST an abnormal lab value and watch:
- Real-time agent progress
- Evidence retrieval from vector database
- Recommendations generated with confidence scores
- SQL audit trail queries
Query the results using IRIS SQL Explorer or Management Portal

💬 Questions or feedback? Reply to this post — I'd love to hear about your use cases.

🎯 What You've Learned

If you've followed along, you now understand how to:

✅ Trigger AI workflows from FHIR events — No manual initiation required
✅ Orchestrate multi-agent systems with CrewAI — Context, Guidelines, Reasoning agents
✅ Build explainable AI with SQL audit trails — Every decision is traceable
✅ Publish AI outputs as FHIR resources — Interoperable clinical documents
✅ Integrate agentic AI with IRIS Interoperability — Production-grade orchestration

🔮 Beyond Lab Results: What Else Can You Automate?

This pattern applies to many clinical scenarios:

Medication reconciliation alerts — Detect drug-drug interactions or contraindications
Care gap identification — Missing screenings based on age, conditions, guidelines
Risk stratification triggers — Identify high-risk patients for intervention
Clinical trial matching — Find eligible patients based on inclusion criteria

The architecture is the same: event → context → evidence → reasoning → action.

🚀 Conclusion

This demo shows how Agentic AI can be safely and effectively integrated into real clinical workflows using InterSystems IRIS for Health.

By combining:

Event-driven interoperability — React to clinical events automatically
Agentic reasoning — Multi-agent collaboration with tool use
SQL persistence — Full audit trails for compliance
FHIR-native outputs — Standard clinical documents

We move from AI experiments to platform-grade clinical AI.

Next Steps:

⭐ Star the repo: https://github.com/intersystems-ib/iris-health-fhir-agentic-demo

🧪 Try the demo with your own clinical guidelines

💬 Share your use case — What clinical event would you automate first?

Dify Now Supports IRIS as a Vector Store — Setup Guide

InterSystems Developer — Tue, 21 Apr 2026 14:18:22 +0000

Why This Integration Matters

InterSystems continues to push AI capabilities forward natively in IRIS — vector search, MCP support, and Agentic AI capabilities. That roadmap is important, and there is no intention of stepping back from it.

But the AI landscape is also evolving in a way that makes ecosystem integration increasingly essential. Tools like Dify — an open-source, production-grade LLM orchestration platform — have become a serious part of enterprise AI stacks. In Japan in particular, Dify adoption is no longer just for startups or hobbyists; it has reached large enterprises, with employees using it as the backbone of internal AI workflows. Meeting developers and teams where they already are is as valuable as building new capabilities in isolation.

That's the motivation behind this integration: IRIS handles what it does best — reliable, queryable, SQL-accessible data with built-in processing logic — while Dify handles LLM orchestration, RAG pipelines, and agentic workflows. Together, they form a stack where IRIS users don't have to choose between their data infrastructure and the AI tools gaining momentum around them.

This integration was contributed to Dify as an OSS pull request and merged in Dify v1.11.2 (#29480). Several follow-up fixes have been merged since — covered below. This article walks through the setup.

First, I'd like to thank @megumi.Kakechi for encouraging me to post this on the English Developer Community — this would have remained a Japanese-only article without that push. I'd also like to extend my gratitude to @tomohiro.Iwamoto and @Mihoko.Iijima, who took the time to test this integration hands-on and provided invaluable feedback that helped shape the fixes included in later releases.

OSS Background: If you're curious about the contribution journey itself, I wrote about it on Zenn: 「このDB、もっと知られてもいいのでは？」からOSSコントリビュートに至った話 (Japanese)

Prerequisites

Tool	Notes
Docker / Docker Compose	Any recent version
Git	For cloning the Dify repository

Setup

1. Clone the Dify repository

> git clone https://github.com/langgenius/dify.git

> cd dify/docker

2. Prepare the environment file

> cp .env.example .env

3. Enable IRIS as the vector store

Open .env and change one line:

# Before (default)

VECTOR_STORE=weaviate

  
  
  After


VECTOR_STORE=iris

That's the minimum. IRIS connection defaults are pre-configured for local use. To connect to an existing IRIS instance or customize the setup, the relevant parameters are:

Parameter	Default	Description
`IRIS_HOST`	iris	Container service name
`IRIS_SUPER_SERVER_PORT`	1972	SuperServer port
`IRIS_WEB_SERVER_PORT`	52773	Management Portal port
`IRIS_USER`	_SYSTEM	Login username
`IRIS_PASSWORD`	Dify@1234	Set automatically on first launch
`IRIS_DATABASE`	USER	Target namespace
`IRIS_SCHEMA`	dify	SQL schema for Dify tables
`IRIS_TEXT_INDEX`	true	Enable full-text index
`IRIS_TEXT_INDEX_LANGUAGE`	en	Language for text indexing
`IRIS_MIN_CONNECTION`	1	Connection pool minimum
`IRIS_MAX_CONNECTION`	3	Connection pool maximum
`IRIS_TIMEZONE`	UTC	Timezone setting

4. Start the containers

> docker compose up -d

5. Confirm all containers are running

> docker compose ps

Look for the iris container with a STATUS of Up:

> docker % docker compose ps --format "table {{.Name}}\t{{.Service}}\t{{.Status}}"

NAME                     SERVICE         STATUS

docker-api-1             api             Up

docker-db_postgres-1     db_postgres     Up (healthy)

docker-nginx-1           nginx           Up

docker-plugin_daemon-1   plugin_daemon   Up

docker-redis-1           redis           Up (healthy)

docker-sandbox-1         sandbox         Up (healthy)

docker-ssrf_proxy-1      ssrf_proxy      Up

docker-web-1             web             Up

docker-worker-1          worker          Up

docker-worker_beat-1     worker_beat     Up

iris                     iris            Up   <-- this one

Access Dify

Navigate to http://localhost/ in your browser. On first launch you'll be prompted to create an admin account.

Verify IRIS is Storing Your Vectors

Step 1 — Create a Knowledge Base

Log in to Dify
Go to Knowledge → Create Knowledge
Upload a text file or PDF
In Step 2, under Index Method, select "High Quality" (recommended)

If no Embedding Model has been configured yet, the dropdown will show "No model found." Click "Model Provider Settings" at the bottom of the dropdown to proceed.

Step 2 — Set Up a Model Provider (OpenAI)

The Model Provider screen lists available providers. Find OpenAI and click Install.

Note on cost: OpenAI requires an API key separate from a ChatGPT Plus subscription — you'll need to add credits to your OpenAI API account. Embedding costs are extremely low, however; a few dollars will go a long way. If you'd prefer a free alternative, local models via LM Studio or Ollama (OpenAI-API-compatible) are also supported.

After installation, OpenAI appears under "To be configured". Click Setup.

The API Key Authorization Configuration dialog opens. If you don't have an API key yet, click "Get your API Key from OpenAI" to open the OpenAI API keys page directly.

Enter a name (e.g. dify), paste your API key, then click Save.

Step 3 — Select the Embedding Model

Return to the Knowledge creation screen. The Embedding Model dropdown now lists available OpenAI models. Select text-embedding-3-small — it offers an excellent balance of cost and retrieval quality for most use cases.

Click Save & Process. When a green checkmark appears next to your document, embedding is complete — your document chunks are now stored as vectors in IRIS.

Step 4 — Inspect the Data via Management Portal

This is where IRIS users have a distinct advantage. Open the Management Portal:

http://localhost:52773/csp/sys/UtilHome.csp?$NAMESPACE=USER

Field	Value
Username	`_SYSTEM`
Password	`Dify@1234`

Navigate to System Explorer → SQL, set the schema filter to dify, and the tables Dify created will appear. Open one, and you'll see your document chunks — including the raw vector embeddings — stored exactly as you'd expect from any IRIS table.

The IRIS Advantage: With most vector stores, embedding data is opaque — accessible only through a proprietary API. With IRIS, you have full SQL access. Query vector data directly, join it against operational data already in your namespace, inspect what's been indexed, or build custom retrieval logic on top of it. That's a meaningful capability for teams already invested in the IRIS ecosystem.

What's Been Fixed Since the Initial Release

The initial integration shipped in v1.11.2 with some rough edges. All known issues have since been resolved and merged into the official Dify codebase.

PR	Release	Description
#29480	v1.11.2	Initial IRIS vector store support
#31309	post-v1.11.2	Fix full-text search and hybrid search for IRIS backend
#31899	v1.12.1	Fix IRIS data persistence across container recreation using Durable %SYS
#31901	v1.12.1	Further improvements to Durable %SYS data persistence

Note on Durable %SYS: Without the fixes in #31899 and #31901, IRIS data could be lost on container recreation — a common issue with the Community Edition Docker image. IRIS developers will recognize Durable %SYS immediately; these fixes ensure the Docker setup respects it correctly.

Troubleshooting

IRIS container fails to start on Windows

On Windows, the IRIS container may fail to start due to volume directory permission issues. Run the following from the docker directory before starting the containers:

chmod -R 777 ./volumes/iris

Note: This is a known limitation on Windows host environments. A fix to eliminate this step is planned for a future release.

IRIS container fails to start (high core count)

On machines with high core counts, IRIS Community Edition's 20-core limit may be triggered. Add the following to the iris service in docker-compose.yml:

services:


  iris:


    cpuset: "0-19"

Summary

What	How
Enable IRIS in Dify	Set `VECTOR_STORE=iris` in `.env`
Verify stored vectors	Management Portal → SQL → schema: `dify`
Data persistence	Handled by Durable %SYS (fixed in v1.12.1)
Community Edition	Free · 10 GB data · Up to 20 cores

A follow-up post will walk through building a full RAG chatbot on this stack. Questions or issues — drop them in the comments below.