DEV Community: Yuri Marx P. Gomes

Technical View: Crawler and NLP to do website text analytics

Yuri Marx P. Gomes — Thu, 11 Aug 2022 22:06:00 +0000

Web Crawling is a technique used to extract root and related content (HTML, Videos, Images, etc.) from websites to your local disk. This is allows you apply NLP to analyze the content and get important insights. This article detail how to do web crawling and NLP.

To do web crawling you can choose a tool in Java or Python. In my case I'm using Crawler4J. (https://github.com/yasserg/crawler4j).

Crawler4j is an open source web crawler for Java which provides a simple interface for crawling the Web. Using it, you can setup a multi-threaded web crawler in few minutes.

After create your Java project, include maven dependency:


<dependency>
        <groupId>edu.uci.ics</groupId>
        <artifactId>crawler4j</artifactId>
        <version>4.4.0</version>
</dependency>

Now, create a class to do the crawler. See you need extends WebCrawler and implement shouldVisit to indicate the file types that will be extracted and visit to get the content and write in your disk. Very simple!



public class CrawlerUtil extends WebCrawler {

private final static Pattern FILTERS = Pattern.compile(".*(\\.(css|js|gif|jpg|png|mp3|mp4|zip|gz))$");

@Override
public boolean shouldVisit(Page referringPage, WebURL url) {
        System.out.println("shouldVisit: " + url.getURL().toLowerCase());

        String href = url.getURL().toLowerCase();
        boolean result = !FILTERS.matcher(href).matches();

        if (result)
            System.out.println("URL Should Visit");
        else
            System.out.println("URL Should not Visit");

        return result;
    }

    @Override
    public void visit(Page page) {
        String url = page.getWebURL().getURL();
        System.out.println("URL: " + url);

        if (page.getParseData() instanceof HtmlParseData) {
            HtmlParseData htmlParseData = (HtmlParseData) page.getParseData();      
            String text = htmlParseData.getText(); //extract text from page
            String html = htmlParseData.getHtml(); //extract html from page
            Set<WebURL> links = htmlParseData.getOutgoingUrls();

            System.out.println("---------------------------------------------------------");
            System.out.println("Page URL: " + url);
            System.out.println("Text length: " + text.length());
            System.out.println("Html length: " + html.length());
            System.out.println("Number of outgoing links: " + links.size());
            System.out.println("---------------------------------------------------------");

            final String OS = System.getProperty("os.name").toLowerCase();

            FileOutputStream outputStream;

            File file;

            try {

                if(OS.indexOf("win") >= 0) {
                    file = new File("c:\\crawler\\nlp" + UUID.randomUUID().toString() + ".txt");
                } else {
                    file = new File("/var/crawler/nlp/" + UUID.randomUUID().toString() + ".txt");
                }

                outputStream = new FileOutputStream(file);

                byte[] strToBytes = text.getBytes();

                outputStream.write(strToBytes);

                outputStream.close();

            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }
}

After, you need to create a Java PEX Business Operation to allows IRIS call this Crawler implementation into a production. To do this is necessary extends the class BusinessOperation (from intersystems) and write your code, simple! The method OnInit allows you instantiate the Java Gateway (a middleware from the InterSystems to promote ObjectScript and Java conversation). In the method OnMessage, you use gateway instance into iris variable to send the java results to IRIS.

The Crawler Java logic in this class run in the executeCrawler. You need set setMaxDepthOfCrawling to config how many sub pages will be visited and setCrawlStorageFolder with folder path to store Crawler4J processing. So you call start passing the number of pages, to limit your crawling. The robotconfig allows you follow the website policies about crawling and another rules. See all the code:

public class WebsiteAnalyzerOperation extends BusinessOperation {

    // Connection to InterSystems IRIS
    private IRIS iris;

    @Override
    public void OnInit() throws Exception {
        iris = GatewayContext.getIRIS();
    }

    @Override
    public Object OnMessage(Object request) throws Exception {
        IRISObject req = (IRISObject) request;

        String website = req.getString("Website");
        Long depth = req.getLong("Depth");
        Long pages = req.getLong("Pages");

        String result = executeCrawler(website, depth, pages);
        IRISObject response = (IRISObject)(iris.classMethodObject("Ens.StringContainer","%New", result));

        return response;
    }

    public String executeCrawler(String website, Long depth, Long pages) {

        final int MAX_CRAWL_DEPTH = depth.intValue();
        final int NUMBER_OF_CRAWELRS = pages.intValue();

        final String OS = System.getProperty("os.name").toLowerCase();

        String CRAWL_STORAGE = "";

        if(OS.indexOf("win") >= 0) {
            CRAWL_STORAGE = "c:\\crawler\\storage";
        } else {
            CRAWL_STORAGE = "/var/crawler/storage";
        }

        CrawlConfig config = new CrawlConfig();
        config.setCrawlStorageFolder(CRAWL_STORAGE);
        config.setIncludeHttpsPages(true);
        config.setMaxDepthOfCrawling(MAX_CRAWL_DEPTH);

        PageFetcher pageFetcher = new PageFetcher(config);
        RobotstxtConfig robotstxtConfig = new RobotstxtConfig();
        RobotstxtServer robotstxtServer = new RobotstxtServer(robotstxtConfig, pageFetcher);

        CrawlController controller;
        try {
            controller = new CrawlController(config, pageFetcher, robotstxtServer);

            controller.addSeed(website);

            controller.start(CrawlerUtil.class, NUMBER_OF_CRAWELRS);

            return "website extracted with success";

        } catch (Exception e) {
            return e.getMessage();
        }

    }

    @Override
    public void OnTearDown() throws Exception {

    }

}

With the content extracted, you can create a NLP domain pointing to the folder with the extractions, see:

<files listname="File_1" batchMode="false" disabled="false" listerClass="%iKnow.Source.File.Lister" path="/var/crawler/nlp/" recursive="false" extensions="txt">
<parameter value="/var/crawler/nlp/" isList="false" />

All the implementation:

Class dc.WebsiteAnalyzer.WebsiteAnalyzerNLP Extends %iKnow.DomainDefinition [ ProcedureBlock ]
{

XData Domain [ XMLNamespace = "http://www.intersystems.com/iknow" ]
{
<domain name="WebsiteAnalyzerDomain" disabled="false" allowCustomUpdates="true">
<parameter name="DefaultConfig" value="WebsiteAnalyzerDomain.Configuration" isList="false" />
<data dropBeforeBuild="true">
<files listname="File_1" batchMode="false" disabled="false" listerClass="%iKnow.Source.File.Lister" path="/var/crawler/nlp/" recursive="false" extensions="txt">
<parameter value="/var/crawler/nlp/" isList="false" />
<parameter isList="false" />
<parameter value="0" isList="false" />
</files>
</data>
<matching disabled="false" dropBeforeBuild="true" autoExecute="true" ignoreDictionaryErrors="true" />
<metadata />
<configuration name="WebsiteAnalyzerDomain.Configuration" detectLanguage="true" languages="en,pt,nl,fr,de,es,ru,uk" userDictionary="WebsiteAnalyzerDomain.Dictionary#1" summarize="true" maxConceptLength="0" />
<userDictionary name="WebsiteAnalyzerDomain.Dictionary#1">
</domain>
}

}

Now, InterSystems NLP can present the results to you into Text Analytics - Domain Explorer.

See all implementation code in the github, download and run!
See all the source code, build and run it here: https://openexchange.intersystems.com/package/website-analyzer

Mastering the %SYSTEM.Encryption class

Yuri Marx P. Gomes — Thu, 11 Aug 2022 22:00:21 +0000

The InterSystems IRIS has excellent support for encryption, decryption and hashing operations. Inside the class %SYSTEM.Encryption (https://docs.intersystems.com/iris20212/csp/documatic/%25CSP.Documatic.cls?LIBRARY=%25SYS&PRIVATE=1&CLASSNAME=%25SYSTEM.Encryption) there are class methods for the main algorithms on the market.

IRIS Algorithms and Encrypt/Decrypt types

As you can see, the operations are based on keys and include 3 options:

Symmetric Keys: the parts running encrypt and decrypt operations share the same secret key.
Asymmetric Keys: the parts conducting encrypt and decrypt operations share the same secret key for encryption. However, for decryption, each partner has a private key. This key cannot be shared with other people, because it is an identity proof.
Hash: used when you do not need to decrypt, but only encrypt.It is a common approach when it comes to storing user passwords.

Differences Between Symmetric and Asymmetric Encryption

Symmetric encryption uses a single key that needs to be shared among the people who need to receive the message while asymmetric encryption uses a pair of public keys and a private key to encrypt and decrypt messages when communicating.
Symmetric encryption is an old technique while asymmetric encryption is relatively new.
Asymmetric encryption was introduced to complement the inherent problem of the need to share the key in a symmetric encryption model, eliminating the need to share the key by using a pair of public-private keys.
Asymmetric encryption takes relatively more time than symmetric encryption.

Symmetric Encryption:

Smaller cipher text than the original plain text file,
used to transmit big data,
symmetric key encryption works on low usage of resources,
128 or 256-bit key size,
Symmetric Encryption uses a single key for encryption and decryption,
It is an old technique,
A single key for encryption and decryption has chances of the key being compromised,
Symmetric encryption is a fast technique,
RC4, AES, DES, 3DES, and QUAD.

Asymmetric Encryption

Larger cipher text than the original plain text file,
Used to transmit small data,
Asymmetric encryption requires high consumption of resources,
RSA 2048-bit or higher key size,
Much safer as two different keys are involved in encryption and decryption,
Asymmetric Encryption uses two different keys for encryption and decryption,
It is a modern technique,
Two keys are separately made for encryption and decryption which removes the need to share a key,
Asymmetric encryption is slower in terms of speed,
RSA, Diffie-Hellman, ECC algorithms.

Source: https://www.ssl2buy.com/wiki/symmetric-vs-asymmetric-encryption-what-are-differences

Using the %SYSTEM.Encryption class to do Encrypt, Decrypt and Hash

To exercise IRIS support to Encrypt, Decrypt and Hash operations, go to https://github.com/yurimarx/cryptography-samples and follow these steps:

Clone/git pull the repo into any local directory

$ git clone https://github.com/yurimarx/cryptography-samples.git

Open a Docker terminal in this directory and run:

$ docker-compose build

Run the IRIS container:

$ docker-compose up -d

Open IRIS Terminal:

$ docker-compose exec iris iris session iris -U IRISAPP

IRISAPP>

To do RSA Encrypt for asymmetric encryption execute this:

IRISAPP>Set ciphertext = ##class(dc.cryptosamples.Samples).DoRSAEncrypt("InterSystems")
IRISAPP>Write ciphertext
Ms/eR7pPmE39KBJu75EOYIxpFEd7qqoji61EfahJE1r9mGZX1NYuw5i2cPS5YwE3Aw6vPAeiEKXF
rYW++WtzMeRIRdCMbLG9PrCHD3iQHfZobBnuzx/JMXVc6a4TssbY9gk7qJ5BmlqRTU8zNJiiVmd8
pCFpJgwKzKkNrIgaQn48EgnwblmVkxSFnF2jwXpBt/naNudBguFUBthef2wfULl4uY00aZzHHNxA
bi15mzTdlSJu1vRtCQaEahng9ug7BZ6dyWCHOv74O/L5NEHI+jU+kHQeF2DJneE2yWNESzqhSECa
ZbRjjxNxiRn/HVAKyZdAjkGQVKUkyG8vjnc3Jw==

To do RSA Decrypt for asymmetric decryption run this:

IRISAPP>Set plaintext = ##class(dc.cryptosamples.Samples).DoRSADecrypt(ciphertext)
IRISAPP>Write plaintext
InterSystems

To do AES CBC Encrypt for symmetric encryption perform this:

IRISAPP>Do ##class(dc.cryptosamples.Samples).DoAESCBCEncrypt("InterSystems")
8sGVUikDZaJF+Z9UljFVAA==

To do AES CBC Decrypt for symmetric encryption complete this:

IRISAPP>Do ##class(dc.cryptosamples.Samples).DoAESCBCDecrypt("8sGVUikDZaJF+Z9UljFVAA==")
InterSystems

To do MD5 hash for an old hash approach conduct this:

IRISAPP>Do ##class(dc.cryptosamples.Samples).DoHash("InterSystems")
rOs6HXfrnbEY5+JBdUJ8hw==

To do SHA hash for recommended hash approach follow this:

IRISAPP>Do ##class(dc.cryptosamples.Samples).DoSHAHash("InterSystems")
+X0hDlyoViPlWOm/825KvN3rRKB5cTU5EQTDLvPWM+E=

To exit the terminal, do any of the following:

Enter HALT or H (not case-sensitive)

About the source code

About the Symmetric key

# to use with symmetric encrypt/decrypt
ENV SECRETKEY=InterSystemsIRIS

In the Dockerfile, there was created an Environment key to be used as secret key on symmetric operations.

About the Asymmetric key

# to use with asymmetric encrypt/decrypt
RUN openssl req  -new -x509 -sha256 -config example-com.conf -newkey rsa:2048 -nodes -keyout 
example-com.key.pem  -days 365 -out example-com.cert.pem

In the Dockerfile were generated a private key and a public key to be used with asymmetric operations.

Symmetric Encrypt

// Symmetric Keys sample to encrypt
ClassMethod DoAESCBCEncrypt(plaintext As %String) As %Status
{
    // convert to utf-8
    Set text=$ZCONVERT(plaintext,"O","UTF8")

    // set a secret key
    Set secretkey = $system.Util.GetEnviron("SECRETKEY")
    Set IV = $system.Util.GetEnviron("SECRETKEY")

    // encrypt a text
    Set text = $SYSTEM.Encryption.AESCBCEncrypt(text, secretkey, IV)
    Set ciphertext = $SYSTEM.Encryption.Base64Encode(text)

    Write ciphertext
}

The operation AES CBC Encrypt was used to encrypt texts.
Base64 Encode returns the results as a pretty/readable text to the user.

Symmetric Decrypt

// Symmetric Keys sample to decrypt
ClassMethod DoAESCBCDecrypt(ciphertext As %String) As %Status
{
    // set a secret key
    Set secretkey = $system.Util.GetEnviron("SECRETKEY")
    Set IV = $system.Util.GetEnviron("SECRETKEY")

    // decrypt a text
    Set text=$SYSTEM.Encryption.Base64Decode(ciphertext)
    Set text=$SYSTEM.Encryption.AESCBCDecrypt(text,secretkey,IV)

    Set plaintext=$ZCONVERT(text,"I","UTF8")
    Write plaintext
}

The operation AES CBC Decrypt was used to decrypt texts.
Base64 Decode returns the encrypted text to a binary one, so it can be used to decrypt.

Asymmetric Encrypt

// Asymmetric Keys sample to encrypt
ClassMethod DoRSAEncrypt(plaintext As %String) As %Status
{
    // get public certificate
    Set pubKeyFileName = "/opt/irisbuild/example-com.cert.pem"
    Set objCharFile = ##class(%Stream.FileCharacter).%New()
    Set objCharFile.Filename = pubKeyFileName
    Set pubKey = objCharFile.Read()

    // encrypt using RSA
    Set binarytext = $System.Encryption.RSAEncrypt(plaintext, pubKey)
    Set ciphertext = $SYSTEM.Encryption.Base64Encode(binarytext)

    Return ciphertext
}

It is necessary to get the public key file content to encrypt with RSA.
The operation RSA Encrypt was used to encrypt texts.

Asymmetric Decrypt

// Asymmetric Keys sample to decrypt
ClassMethod DoRSADecrypt(ciphertext As %String) As %Status
{
    // get private key
    Set privKeyFileName = "/opt/irisbuild/example-com.key.pem"
    Set privobjCharFile = ##class(%Stream.FileCharacter).%New()
    Set privobjCharFile.Filename = privKeyFileName
    Set privKey = privobjCharFile.Read()

    // get ciphertext in binary format
    Set text=$SYSTEM.Encryption.Base64Decode(ciphertext)

    // decrypt text using RSA
    Set plaintext = $System.Encryption.RSADecrypt(text, privKey)

    Return plaintext
}

It is necessary to get the private key file content to decrypt with RSA.
The operation RSA Decrypt was used to decrypt texts.

Hash text using MD5 (old approach)

// Hash sample 
ClassMethod DoHash(plaintext As %String) As %Status
{
    // convert to utf-8
    Set text=$ZCONVERT(plaintext,"O","UTF8")

    // hash a text
    Set hashtext = $SYSTEM.Encryption.MD5Hash(text)

    Set base64text = $SYSTEM.Encryption.Base64Encode(hashtext)

    // convert to hex text to following best practices
    Set hextext = ..GetHexText(base64text)

    // return using lowercase
    Write $ZCONVERT(hextext,"L")
}

The operation MD5 Hash will encrypt the text, and it will not be possible to decrypt it.
Hash using MD5 is not recommended for new projects because it is considered insecure. That is why it was replaced by SHA. The InterSystems IRIS supports SHA (our next example will demonstrate it).

Hash text using SHA (recommend approach)

We will use the SHA-3 Hash method for this sample. According to InterSystems documentation, this method generates a hash using one of the U.S. Secure Hash Algorithms - 3. (See Federal Information Processing Standards Publication 202 for more information.).


// Hash using SHA 
ClassMethod DoSHAHash(plaintext As %String) As %Status
{
    // convert to utf-8
    Set text=$ZCONVERT(plaintext,"O","UTF8")

    // hash a text
    Set hashtext = $SYSTEM.Encryption.SHA3Hash(256, text)

    Set base64text = $SYSTEM.Encryption.Base64Encode(hashtext)

    // convert to hex text to following best practices
    Set hextext = ..GetHexText(base64text)

    // return using lowercase
    Write $ZCONVERT(hextext,"L")
}

For the SHA method, it is possible to set the bit length used on a hash operation. The greater the number of bits, the more difficult it is to crack the hash. However, the hashing process slows down too. In this sample we used 256 bits. You can choose these options for bit length:

224 (SHA-224)
256 (SHA-256)
384 (SHA-384)
512 (SHA-512)

Using Tesseract OCR and Java Gateway

Yuri Marx P. Gomes — Sun, 19 Jun 2022 11:58:39 +0000

The InterSystems IRIS can be extended using Java or .NET components and its frameworks inside Object Script source code.

I created an application called OCR Service. It built with Docker and installs Google Tesseract inside docker instance configured with english and portuguese dialects, but is possible install more than other 100 dialects. Google Tesseract can receive images and return text extracted from it, using OCR. The results are very good with the trained dialects. But you can train Tesseract to read car plates and any other textual patterns and load it to extract text. Java has a framework called Tess4J to enable Java call Tesseract instances and functions. See running:

OCR in Action

See the Java code to the OCR service:

private String extractTextFromImage(File tempFile) throws TesseractException {

  ITesseract tesseract = new Tesseract();
  tesseract.setDatapath("/usr/share/tessdata/"); //directory to trained models
  tesseract.setLanguage("eng+por"); // choose your language/trained model

  return tesseract.doOCR(tempFile); //call tesseract function doOCR() //passing the file to be processed with OCR technique

}

Tess4J take care all low level communication with Tesseract with ITesseract interface class and we need only set the OCR model to be used and call doOCR() passing the file. Easy!

For our luck, we don't have to create a Tess4ObjectScript framework to interact with Tesseract, it is possible, using C callin/callout, but you need to understand how to talk at low level with Tesseract, I prefer use Java Gateway to save many work days!

To use Java Gateway, you have to install Java into your IRIS OS or docker instance, set JAVA_HOME (variable to know where Java is installed), set the CLASSPATH (variable to point to your Java class, Tess4J java archive and other dependencies) and execute a Java Gateway instance, a proxy to enable ObjectScript and Java to talk. See my dockerfile instructions to install Java and Tesseract to IRIS instance:

ARG IMAGE=store/intersystems/iris-community:2020.1.0.204.0
ARG IMAGE=intersystemsdc/iris-community:2020.1.0.209.0-zpm
ARG IMAGE=intersystemsdc/iris-community:2020.2.0.204.0-zpm
ARG IMAGE=intersystemsdc/irishealth-community:2020.3.0.200.0-zpm
ARG IMAGE=intersystemsdc/iris-community:2020.3.0.200.0-zpm
ARG IMAGE=intersystemsdc/iris-community:2020.3.0.221.0-zpm
ARG IMAGE=intersystemsdc/iris-community:2020.4.0.521.0-zpm
FROM $IMAGE

USER root   

WORKDIR /opt/irisapp
RUN chown ${ISC_PACKAGE_MGRUSER}:${ISC_PACKAGE_IRISGROUP} /opt/irisapp
USER ${ISC_PACKAGE_MGRUSER}

COPY  Installer.cls .
COPY  src src

COPY iris.script /tmp/iris.script

RUN iris start IRIS \
    && iris session IRIS < /tmp/iris.script \
    && iris stop IRIS quietly

USER root   

# Install Java 8 using apt-get from ubuntu repository
RUN apt-get update && \
    apt-get install -y openjdk-8-jdk && \
    apt-get install -y ant && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/* && \
    rm -rf /var/cache/oracle-jdk8-installer;

# Fix certificate issues, found as of 
# https://bugs.launchpad.net/ubuntu/+source/ca-certificates-java/+bug/983302
RUN apt-get install -y ca-certificates-java && \
    apt-get clean && \
    update-ca-certificates -f && \
    rm -rf /var/lib/apt/lists/* && \
    rm -rf /var/cache/oracle-jdk8-installer;

# Setup JAVA_HOME, to enable apps to know where the Java was installed
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/
RUN export JAVA_HOME
ENV JRE_HOME /usr/lib/jvm/java-8-openjdk-amd64/
RUN export JRE_HOME

# Setup classpath, to enable apps to know where java classes and java jar libraries was installed
ENV classpath .:/usr/irissys/dev/java/lib/JDK18/*:/opt/irisapp/*:/usr/irissys/dev/java/lib/gson/*
:/usr/irissys/dev/java/lib/jackson/*:/jgw/*
RUN export classpath
ENV CLASSPATH .:/usr/irissys/dev/java/lib/JDK18/*:/opt/irisapp/*:/usr/irissys/dev/java/lib/gson/*
:/usr/irissys/dev/java/lib/jackson/*:/jgw/*
RUN export CLASSPATH

USER root

ARG APP_HOME=/tmp/app

COPY src $APP_HOME/src

# Tess4J and another java libraries used, are into jgw folder and jgw folder is in the classpath
COPY jgw /jgw

# Copy our Java OCR program, packaged into a jar, to the jgw
COPY target/ocr-pex-1.0.0.jar /jgw/ocr-pex-1.0.0.jar  

COPY jgw/* /usr/irissys/dev/java/lib/JDK18/

# Install tesseract using ubuntu apt-get
RUN apt-get update && apt-get install tesseract-ocr -y

USER root

# Copy trained models eng and por to the models folder
COPY tessdata /usr/share/tessdata

# Install and config default OS locale - it is required to tesseract works fine
RUN apt-get update && apt-get install -y locales && rm -rf /var/lib/apt/lists/* \
 && locale-gen "en_US.UTF-8"
ENV LANG=en_US.UTF-8 \
    LANGUAGE=en_US:en \
    LC_ALL=en_US.UTF-8

When your ObjectScript class is an interoperability business class or operation, set a Java Gateway Business Service (JavaGateway item into my production).

Class dc.ocr.OcrProduction Extends Ens.Production
{

XData ProductionDefinition
{

<Production Name="dc.ocr.OcrProduction" LogGeneralTraceEvents="false">
  <Description></Description>
  <ActorPoolSize>2</ActorPoolSize>
  <Item Name="OcrService" Category="" ClassName="dc.ocr.OcrService" PoolSize="1" Enabled="true" 
Foreground="false" Comment="" LogTraceEvents="false" Schedule="">
  </Item>

  <Item Name="JavaGateway" Category="" ClassName="EnsLib.JavaGateway.Service" PoolSize="1" 
Enabled="true" Foreground="false" Comment="" LogTraceEvents="false" Schedule="">
    <Setting Target="Host" Name="ClassPath">.:/usr/irissys/dev/java/lib/JDK18/*:/opt/irisapp/*
:/usr/irissys/dev/java/lib/gson/*
:/usr/irissys/dev/java/lib/jackson/*:/jgw/ocr-pex-1.0.0.jar
</Setting>
    <Setting Target="Host" Name="JavaHome">/usr/lib/jvm/java-8-openjdk-amd64/</Setting>
  </Item>

</Production>

}

}

Into the production we have our ObjectScript OCRService too. This ObjectScript class receive the file uploaded from a HTTP multipart request and uses JavaGateway to load the Java class, pass the file and get the text from OCR process, so returns the response. See:

Class dc.ocr.OcrService Extends Ens.BusinessService
{

// extends Ens.BusinessService to create a custom Business service using Object Script

// This class receive a file from a multipart http request and save 
// to the folder configured into folder parameter

// choose an adapter to get data from a source of data

// HTTP.InboundAdapter allows you get data from an http request

Parameter ADAPTER = "EnsLib.HTTP.InboundAdapter";

// custom parameter to allows production user set destination folder to multipart file uploaded 

Property Folder As %String(MAXLEN = 100);

// when you set parameter Folder to SETTINGS parameter, the production IRIS interface 
// create a field to the user fills
// so the user will inform host path for the uploaded file 

Parameter SETTINGS = "Folder,Basic";

// This method is mandatory to have a business service. It receives the multipart file into pInput 

// and returns a result to the caller using pOutput

Method OnProcessInput(pInput As %GlobalBinaryStream, pOutput As %RegisteredObject) As %Status
{
    //try to do the actions
    try {
        Set reader = ##class(%Net.MIMEReader).%New() //creates a MIMEReader to extract files 
//from multipart requests 
        Do reader.OpenStream(pInput) //reader open the file

        Set tSC = reader.ReadMIMEMessage(.message) //the reader put the file uploaded into a MIME Message
        //Get Header obtains headers from the request and the multipart file, like content-type or content 
//disposition the content disposition have 3 headers: Content-Disposition: form-data; name="file"; 
//filename="filename.ext". This split content-disposition header into 3 parts
        Set filenameHeader = $PIECE(message.GetHeader("CONTENT-DISPOSITION", .header),";",3) 
        //get filename header value
        Set filename = $EXTRACT(filenameHeader, 12, $LENGTH(filenameHeader)-1)
        //Headers are not more needed. It clean the header to remains only the file content to be saved
        Do message.ClearHeaders()

        //create a file object to save the multipart file
        Set file=##class(%Stream.FileBinary).%New()
        //points the file to folder informed into folder parameter, plus upload filename from header
        Set file.Filename=..Folder_filename 
        //save body message (the file content) to file object
        Do file.CopyFromAndSave(message.Body)

        // Connect a Gateway instance to server JavaGate on the host machine
        set GW = ##class(%Net.Remote.Gateway).%New()
        set st = GW.%Connect("127.0.0.1", "55555", "IRISAPP",,)
        //instantiate java ocr class
        set proxyOcr = ##class(%Net.Remote.Object).%New(GW,"community.intersystems.pex.ocr.OcrOperation")
        //call ocr method to get text from image
        set pResponse = proxyOcr.doOcr(file.Filename)
        //returns to the service
        Set pOutput = pResponse
        Set tSC=$$$OK

    //returns error message to the user
    } catch e {
        Set tSC=e.AsStatus()
        Set pOutput = tSC
    }

    Quit tSC
}

}

Now, IRIS can do OCR to images and PDF, but with JavaGateway function can do many amazing things. See all details into my app code: https://openexchange.intersystems.com/package/OCR-Service.

OCR and NLP together into InterSystems IRIS

Yuri Marx P. Gomes — Sun, 19 Jun 2022 11:51:58 +0000

According to IDC, more than 80% of information it is NoSQL, especially text into documents. When the digital services or applications not process all this information, the business lose. To face this challenge, it is possible use OCR technology. OCR uses machine learning and/or trained image patterns to transform image pixels into text. This is important, because many documents are scanned into images inside PDF, or many documents contains images with text inside. So OCR are an important step to get all possible data from a document.

To do OCR, the main open source solution used is Google Tesseract, the most popular solution into the Python and Java community. Tesseract has support to more than 100 idioms and can be trained with new models to recognize car plates, captchas and so on. Tesseract was created in C++, so Java uses it consuming an intermediate, called Tess4J. My following code shows it to you:

private String extractTextFromImage(File tempFile) throws TesseractException {

  ITesseract tesseract = new Tesseract();
  tesseract.setDatapath("/usr/share/tessdata/"); //directory to trained models
  tesseract.setLanguage("eng+por"); // choose your language/trained model

  return tesseract.doOCR(tempFile); //call tesseract function doOCR() passing the file to be processed with OCR technique

}

To allows IRIS to use this Java Class and get the results from Java, we need to use PEX and Java Gateway solutions.

First it is necessary config Java Proxy into the production and second, config a PEX business operation or service to communicate IRIS and Java into a production.

Class dc.ocr.OcrProduction Extends Ens.Production
{

XData ProductionDefinition
{
<Production Name="dc.ocr.OcrProduction" LogGeneralTraceEvents="false">

  <Description></Description>
  <ActorPoolSize>2</ActorPoolSize>

<Item Name="OcrService" Category="" ClassName="dc.ocr.OcrService" PoolSize="1" Enabled="true" 
Foreground="false" Comment="" LogTraceEvents="false" Schedule=""> </Item>

<Item Name="JavaGateway" Category="" ClassName="EnsLib.JavaGateway.Service" PoolSize="1" 
Enabled="true" Foreground="false" Comment="" LogTraceEvents="false" Schedule="">
  <Setting Target="Host" Name="ClassPath">.:/usr/irissys/dev/java/lib/JDK18/*:/opt/irisapp/*
:/usr/irissys/dev/java/lib/gson/*
:/usr/irissys/dev/java/lib/jackson/*:/jgw/ocr-pex-1.0.0.jar</Setting> 

<Setting Target="Host" Name="JavaHome">/usr/lib/jvm/java-8-openjdk-amd64/</Setting> 
</Item> 

<Item Name="OcrOperation" Category="" ClassName="EnsLib.PEX.BusinessOperation" PoolSize="1" 
Enabled="true" Foreground="false" Comment="" LogTraceEvents="false" Schedule=""> 
    <Setting Target="Host" Name="%gatewayPort">55555</Setting> 
    <Setting Target="Host" Name="%remoteClassname">community.intersystems.pex.ocr.OcrOperation</Setting> 
    <Setting Target="Host" Name="%gatewayExtraClasspaths">.:/usr/irissys/dev/java/lib/JDK18/*
:/opt/irisapp/*:/usr/irissys/dev/java/lib/gson/*
:/usr/irissys/dev/java/lib/jackson/*
:/jgw/ocr-pex-1.0.0.jar
</Setting> 
</Item> 

</Production>
}

}

Now any IRIS production can communicate with Java and Tesseract! See:

  //call ocr method to get text from image, if you want to use pex
  Set pRequest = ##class(dc.ocr.OcrRequest).%New()
  Set pRequest.FileName = file.Filename

  // call java pex operation to do ocr, passing file into pRequest and receive ocr text with pResponse
  Set tSC = ..SendRequestSync("OcrOperation", pRequest, .pResponse, 1200)

  //save the results into database to use text analytics - nlp
  Set ocrTable = ##class(dc.ocr.OcrTable).%New()
  Set ocrTable.FileName = file.Filename
  Set ocrTable.OcrText = pResponse.StringValue

  Set tSC = ocrTable.%Save()

All code details, with comments can be found into my OCR Service repository (https://openexchange.intersystems.com/package/OCR-Service).

Now, with the text extracted, we need to use IRIS NLP engine to analyze textual data and get insights to support decisions. For this, when a text is extracted, it is saved into a table, and this table is used by NLP engine as text source. See the table %Save() above and see the following code with NLP referencing OCRTable (place with texts extracted). See:

Class dc.ocr.OcrNLP Extends %iKnow.DomainDefinition [ ProcedureBlock ]
{

XData Domain [ XMLNamespace = "http://www.intersystems.com/iknow" ]
{
<domain name="OcrNLP" disabled="false" allowCustomUpdates="true">
<parameter name="DefaultConfig" value="OcrNLP.Configuration" isList="false" />

<data dropBeforeBuild="true">
<table listname="OcrNLPTable" batchMode="true" disabled="false" 
listerClass="%iKnow.Source.SQL.Lister" tableName="dc_ocr.OcrTable" idField="ID" 
groupField="ID" dataFields="OcrText" metadataColumns="FileName" metadataFields="filename" />
</data>

<matching disabled="false" dropBeforeBuild="true" autoExecute="true" ignoreDictionaryErrors="true" />

<metadata>
<field name="filename" operators="=" dataType="STRING" storage="0" caseSensitive="false" disabled="false" />
</metadata>

<configuration name="OcrNLP.Configuration" detectLanguage="true" languages="en,pt" 
userDictionary="OcrNLP.Dictionary#1" summarize="true" maxConceptLength="0" />

<userDictionary name="OcrNLP.Dictionary#1" />

</domain>

}

}

See full details and configuration into my OCR Service github repository.

Now we can upload some files and go to the Explorer to see concepts and CRC generated.

See my animation with all steps discussed here:

Happy OCR/NLP hacking!

InterSystems IRIS REST Application Patterns

Yuri Marx P. Gomes — Wed, 23 Feb 2022 14:54:35 +0000

This article suggests to you some patterns to create REST API applications using IRIS.

Note: source code in https://github.com/yurimarx/movie

Class Pattern to the REST Application

To begin, see my suggestion for classes needed to create IRIS API applications:

IRISRESTApplication: CSP.REST class that will be the central controller for all REST requests and responses processed by the business services.
BusinessService: class with a business topic implementation. It can use one or more Persistent Domain Classes to persist and query data required by the business topic requirements.
Persistent Domain: persistent class to manage a SQL table.

Prereqs

VSCode;
Docker Desktop;
InterSystems ObjectScript Extension Pack.

Class Diagram to the Sample Application

I will create a Movie Catalog application to demonstrate the patterns suggested in the article:

Note: thanks to the https://openexchange.intersystems.com/package/iris-rest-api-template application. It was the base to this tutorial.

Setup the Sample Application

Create a folder movie in your file system. Open this folder in a new VSCode window
Create the Dockerfile file inside movie folder to run IRIS Community edition into a Docker container instance. Content:

ARG IMAGE=intersystemsdc/iris-community:2020.3.0.221.0-zpm
ARG IMAGE=intersystemsdc/iris-community:2020.4.0.524.0-zpm
ARG IMAGE=intersystemsdc/iris-community

FROM $IMAGE

USER root   

WORKDIR /opt/irisapp

RUN chown ${ISC_PACKAGE_MGRUSER}:${ISC_PACKAGE_IRISGROUP} /opt/irisapp

USER ${ISC_PACKAGE_MGRUSER}

COPY  src src

COPY module.xml module.xml

COPY iris.script /tmp/iris.script

RUN iris start IRIS \
    && iris session IRIS < /tmp/iris.script \
    && iris stop IRIS quietly

Create the docker-compose.yml file inside movie folder to allows you run your docker instance and other instances together (not in this sample, but it is a good practice run from docker-compose instead dockerfile. Content:

version: '3.6'
services:
  iris:
    build: 
      context: .
      dockerfile: Dockerfile
    restart: always
    ports: 
      - 51773
      - 1972:1972
      - 52773:52773
      - 53773
    volumes:
      - ./:/irisdev/app

Create the iris.script file inside movie folder to do some actions before run IRIS. This file is important to do custom terminal actions necessary for the application, like disable password expiration. Content:

do $System.OBJ.LoadDir("/opt/irisapp/src","ck",,1)

zn "%SYS"

Do ##class(Security.Users).UnExpireUserPasswords("*")

zn "USER"

zpm "load /opt/irisapp/ -v":1:1

halt

Create the module.xml file inside movie folder to install and run your application using ZPM. This file is important to do the application endpoint configuration and install swagger-ui (web app used to run and test your API using swagger file). Content:

<?xml version="1.0" encoding="UTF-8"?>
<Export generator="Cache" version="25">
  <Document name="movie.ZPM">
    <Module>
      <Name>movie</Name>
      <Version>1.0.0</Version>
      <Packaging>module</Packaging>
      <SourcesRoot>src</SourcesRoot>
      <Resource Name="dc.movie.PKG"/>
      <Dependencies>
        <ModuleReference>
          <Name>swagger-ui</Name>
          <Version>1.*.*</Version>
        </ModuleReference>
      </Dependencies>
       <CSPApplication 
        Url="/movie-api"
        DispatchClass="dc.movie.MovieRESTApp"
        MatchRoles=":{$dbrole}"
        PasswordAuthEnabled="1"
        UnauthenticatedEnabled="0"
        Recurse="1"
        UseCookies="2"
        CookiePath="/movie-api"
       />
    </Module>
  </Document>
</Export>

You can see CSPApplication tag, used to run the application API in the /movie-api URI and enable or disable password to consume the API.

Create the LICENSE file inside movie folder to setting the license of your application. Content:

MIT License
Copyright (c) 2019 InterSystems Developer Community Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Create the README.md file inside movie folder to document your application to the users using markdown language. Content:

## movie-rest-application
This is a sample of a REST API application built with ObjectScript in InterSystems IRIS.

Create .vscode folder inside movie folder. Create settings.json file inside .vscode folder to configure server connection between VSCode and your IRIS instance. Content:

{
    "files.associations": {
        "Dockerfile*": "dockerfile",
        "iris.script": "objectscript"
      },

    "objectscript.conn" :{
      "ns": "USER",
      "username": "_SYSTEM",
      "password": "SYS",
      "docker-compose": {
        "service": "iris",
        "internalPort": 52773
      },

      "active": true
    },

    "sqltools.connections": [
      {
        "namespace": "USER",
        "connectionMethod": "Server and Port",
        "showSystem": false,
        "previewLimit": 50,
        "server": "localhost",
        "port": 52773,
        "askForPassword": false,
        "driver": "InterSystems IRIS",
        "name": "objectscript-docker",
        "username": "_SYSTEM",
        "password": "SYS"
      }
    ]
}

Create the folder src inside movie folder to put your source code folders and files.
Create dc folder inside src folder. This is a convention when your build projects to the InterSystems Developer Community, otherwise is not necessary.
Create movie folder inside dc folder. This folder will be the folder to your objectscript classes.
Create our first class, MovieRESTApp.cls file, inside src\dc\movie folder. This file will be the IRISRESTApplication class. Content:


Class dc.movie.MovieRESTApp Extends %CSP.REST
{

Parameter CHARSET = "utf-8";
Parameter CONVERTINPUTSTREAM = 1;
Parameter CONTENTTYPE = "application/json";
Parameter Version = "1.0.0";
Parameter HandleCorsRequest = 1;

XData UrlMap [ XMLNamespace = "http://www.intersystems.com/urlmap"]
{
<Routes>

<!-- Server Info -->
<Route Url="/" Method="GET" Call="GetInfo" Cors="true"/>

<!-- Swagger specs -->
<Route Url="/_spec" Method="GET" Call="SwaggerSpec" />

</Routes>
}


ClassMethod %ProcessResult(pStatus As %Status = {$$$OK}, pResult As %DynamicObject = "") As %Status [ Internal ]
{

  #dim %response As %CSP.Response

  SET tSC = $$$OK

  IF $$$ISERR(pStatus) {

    SET %response.Status = 500

    SET tSC = ..StatusToJSON(pStatus, .tJSON)

    IF $isobject(tJSON) {

      SET pResult = tJSON

    } ELSE {

      SET pResult = { "errors": [ { "error": "Unknown error parsing status code" } ] }

    }

  } 

  ELSEIF pStatus=1 {

    IF '$isobject(pResult){

      SET pResult = {

      }

    }

  }

  ELSE {

    SET %response.Status = pStatus

    SET error = $PIECE(pStatus, " ", 2, *)

    SET pResult = {

      "error": (error)

    }

  }

  IF pResult.%Extends("%Library.DynamicAbstractObject") {

    WRITE pResult.%ToJSON()

  }

  ELSEIF pResult.%Extends("%JSON.Adaptor") {

    DO pResult.%JSONExport()

  }

  ELSEIF pResult.%Extends("%Stream.Object") {

    DO pResult.OutputToDevice()

  }

  QUIT tSC

}


ClassMethod SwaggerSpec() As %Status
{

  Set tSC = ##class(%REST.API).GetWebRESTApplication($NAMESPACE, %request.Application, .swagger)

  Do swagger.info.%Remove("x-ISC_Namespace")

  Set swagger.basePath = "/movie-api" 

  Set swagger.info.title = "Movie API"

  Set swagger.info.version = "1.0"

  Set swagger.host = "localhost:52773"

  Return ..%ProcessResult($$$OK, swagger)

}

}
```

`
Note 1: The class extends CSP.REST to be used as the REST Endpoint.

Note 2: the parameter chaset is used to encode requests and responses with UTF-8.

Note 3: the CONVERTINPUTSTREAM is used to force the request content in the UTF-8, without this you can have problems with special latin chars.

Note 4: CONTENTTYPE is used to declare the content using JSON, not XML.

Note 5: HandleCorsRequest = 1 is necessary to allows you consume the API from other servers different from the IRIS server.

Note 6: Routes are used to declare API URI to each class method.

Note 7: SwaggerSpec from CSP.REST class allows you generate the API swagger (API web documentation) content.

Now you have the following folders and files:
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/aur5um97ewhmsqvjd5d1.png)

13. Open VSCode Terminal (menu Terminal > New Terminal) and type:

```
docker-compose up -d --build
```
This will build the docker instance and run it.

14. Test your API with Swagger-UI. In the browser and type: http://localhost:52773/swagger-ui/index.html. Pay attention to the address bar (fix the url, if necessary to correct address)
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0w0ti5aunovfpl5nuw02.png)

## Connection beetween VSCode and IRIS  
1. Click in the ObjectScript bar (in the VSCode footer)
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3uabr9fvhjgimjlw6bzm.png)

2. Select Toogle Connection in the Top:
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0rruhp6g423v8im0hoh7.png)

3. Check the connection status into ObjectScript Explorer (you will be able to see folders and classes created):
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/stcugup3e5lnvi2yo4wx.png)

## Persistent Classes to the Movie Catalog Application  
In this section we will create the persistent domain classes to store and query the business data. See the DBeaver Diagram:
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pb25ndbkqsta01p4963g.png)

1. Create the folder model inside src\dc\movie folder.

2. Create the Actor.cls file inside model folder. Write the content:  

`

```
Class dc.movie.model.Actor Extends (%Persistent, %JSON.Adaptor)
{
Parameter %JSONREFERENCE = "ID";

Property actorId As %Integer [ Calculated, SqlComputeCode = { set {*}={%%ID}}, SqlComputed ];
Property name As %VarString(MAXLEN = 120);  
Property dob As %Date;
Property genre As %Integer(VALUELIST = ",1,2");

}

```

`
3. Create the Movie.cls file inside model folder. Write the content:

`

```
Class dc.movie.model.Movie Extends (%Persistent, %JSON.Adaptor)
{

Parameter %JSONREFERENCE = "ID";

Property movieId As %Integer [ Calculated, SqlComputeCode = { set {*}={%%ID}}, SqlComputed ]; 
Property name As %VarString(MAXLEN = 120);
Property releaseDate As %Date;
Property duration As %Integer;
Property imdb As %String(MAXLEN = 300);
Property movieCategory As dc.movie.model.MovieCategory;
ForeignKey MovieCategoryFK(movieCategory) References dc.movie.model.MovieCategory();

}
```

`
4. Create the MovieCategory.cls file inside model folder. Write the content:

`

```
Class dc.movie.model.MovieCategory Extends (%Persistent, %JSON.Adaptor)
{

Parameter %JSONREFERENCE = "ID";

Property movieCategoryId As %Integer [ Calculated, SqlComputeCode = { set {*}={%%ID}}, SqlComputed ]; 
Property name As %VarString(MAXLEN = 120);

}
```

`
5. Create the Casting.cls file inside model folder. Write the content:

`

```
Class dc.movie.model.Casting Extends (%Persistent, %JSON.Adaptor)
{


Parameter %JSONREFERENCE = "ID";

Property castingId As %Integer [ Calculated, SqlComputeCode = { set {*}={%%ID}}, SqlComputed ];
Property movie As dc.movie.model.Movie;
ForeignKey MovieFK(movie) References dc.movie.model.Movie();
Property actor As dc.movie.model.Actor;
ForeignKey ActorFK(actor) References dc.movie.model.Actor();
Property characterName As %String(MAXLEN = 100);

Index CastingIndex On (movie, actor) [ Unique ];

}
```

`
See the created files:
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/i1n22lvk121fmwn6a5bx.png)

Note 1: Parameter %JSONREFERENCE = "ID" allows return ID value inside JSON response.

Note 2: Property actorId As %Integer [ Calculated, SqlComputeCode = { set {*}={%%ID}}, SqlComputed ] and the other similar properties are used to return class+id into JSON response.

Note 3: (VALUELIST = "1,2") set possible values to 1 or 2 only.

Note 4: ForeignKey MovieFK(movie) References dc.movie.model.Movie() and similar are used to create a SQL foreign key reference.

Note 5: Index CastingIndex On (movie, actor) [ Unique ] and similar are used to not allows duplicate values combining properties in the On (movie and actor).

Note 6: I'm using Camel Case to property names because a best practice for JSON attribute names.

## Business Service Classes to the Movie Catalog Application 
In this section we will create the classes with business logic (methods to do persistence, query and calculations).

1. Create the service folder inside src\dc\movie.

2. Create CrudUtilService.cls file inside service folder. Write the content:

`

```
Class dc.movie.service.CrudUtilService Extends %CSP.REST
{

Parameter CHARSET = "utf-8";
Parameter CONVERTINPUTSTREAM = 1;
Parameter CONTENTTYPE = "application/json";
Parameter Version = "1.0.0";
Parameter HandleCorsRequest = 1;

/// Return all the records
ClassMethod GetAll(DomainClass As %Persistent) As %Status
{

    #dim tSC As %Status = $$$OK
    Set rset = DomainClass.ExtentFunc()
    Set %response.ContentType = ..#CONTENTTYPEJSON
    Set %response.Headers("Access-Control-Allow-Origin")="*"

    Write "["

    if rset.%Next() {
        Set actor = DomainClass.%OpenId(rset.ID)    
        Do actor.%JSONExport()
    }

    While rset.%Next() {   
        Write "," 
        Set actor = DomainClass.%OpenId(rset.ID)    
        Do actor.%JSONExport()
    }

    Write "]"

    Quit tSC
}

/// Return one record
ClassMethod GetOne(DomainClass As %Persistent, id As %Integer) As %Status
{

    #dim tSC As %Status = $$$OK
    #dim e As %Exception.AbstractException

    #; Set the response header to plain text
    Set %response.ContentType = ..#CONTENTTYPEJSON
    Set %response.Headers("Access-Control-Allow-Origin")="*"

    Set domain = DomainClass.%OpenId(id)

    If '$IsObject(domain) Quit ..Http404()

    Do domain.%JSONExport()

    Quit tSC

}


/// Creates a new record
ClassMethod Create(DomainClass As %Persistent) As %Status
{

    #dim tSC As %Status = $$$OK
    #dim e As %Exception.AbstractException

    Set domain = DomainClass.%New()
    Set data = {}.%FromJSON(%request.Content)

    $$$TOE(tSC, domain.%JSONImport(data))
    $$$TOE(tSC, domain.%Save())

    Write domain.%JSONExport()

    Set %response.Status = 204
    Set %response.ContentType = ..#CONTENTTYPEJSON
    Set %response.Headers("Access-Control-Allow-Origin")="*"

    Quit tSC

}


/// Update a record with id
ClassMethod Update(DomainClass As %Persistent, id As %Integer) As %Status
{

    #dim tSC As %Status = $$$OK 
    #dim e As %Exception.AbstractException

    Set domain = DomainClass.%OpenId(id) 

    If '$IsObject(domain) Return ..Http404()

    Set data = {}.%FromJSON(%request.Content)

    $$$TOE(tSC, domain.%JSONImport(data))

    $$$TOE(tSC, domain.%Save())

    Write domain.%JSONExport()

    Set %response.Status = 200

    Set %response.ContentType = ..#CONTENTTYPEJSON

    Set %response.Headers("Access-Control-Allow-Origin")="*"

    Quit tSC

}


/// Delete a record with id
ClassMethod Delete(DomainClass As %Persistent, id As %Integer) As %Status
{

    #dim tSC As %Status = $$$OK
    #dim e As %Exception.AbstractException

    Set domain = DomainClass.%OpenId(id)

    If '$IsObject(domain) Return ..Http404()

    $$$TOE(tSC, domain.%DeleteId(id))

    Set %response.Status = 200

    Set %response.ContentType = ..#CONTENTTYPEJSON

    Set %response.Headers("Access-Control-Allow-Origin")="*"

    Quit tSC

}

}
```

`
3. Create MovieService.cls file inside service folder. Write the content:

`

```
Class dc.movie.service.MovieService Extends %CSP.REST
{

ClassMethod GetAll() As %Status
{

    Return ##class(dc.movie.service.CrudUtilService).GetAll(##class(dc.movie.model.Movie).%New())

}


ClassMethod GetOne(id As %Integer) As %Status
{

    Return ##class(dc.movie.service.CrudUtilService).GetOne(##class(dc.movie.model.Movie).%New(), id)

}


ClassMethod Create() As %Status
{

    Return ##class(dc.movie.service.CrudUtilService).Create(##class(dc.movie.model.Movie).%New())

}


ClassMethod Update(id As %Integer) As %Status
{

    Return ##class(dc.movie.service.CrudUtilService).Update(##class(dc.movie.model.Movie).%New(), id)

}

ClassMethod Delete(id As %Integer) As %Status
{

    Return ##class(dc.movie.service.CrudUtilService).Delete(##class(dc.movie.model.Movie).%New(), id)

}

/// Return casting from the movie
ClassMethod GetMovieCasting(id As %Integer) As %Status
{

    #dim tSC As %Status = $$$OK

    Set qry = "SELECT actor->name AS actorName, characterName, movie->name AS movieName FROM dc_movie_model.Casting WHERE movie = ?"

    Set tStatement = ##class(%SQL.Statement).%New()  

    Set qStatus = tStatement.%Prepare(qry)

    If tSC'=1 {WRITE "%Prepare failed:" DO $System.Status.DisplayError(qStatus) QUIT}

    Set rset = tStatement.%Execute(id)
    Set %response.ContentType = ..#CONTENTTYPEJSON
    Set %response.Headers("Access-Control-Allow-Origin")="*"

    Set result = []

    While rset.%Next() {

        Set item = {}

        Set item.actorName = rset.actorName

        Set item.movieName = rset.movieName

        Set item.characterName = rset.characterName

        Do result.%Push(item)

    }

    Write result.%ToJSON()

    Quit tSC
}

}
```

`
4. Create MovieCategoryService.cls file inside service folder. Write the content:

`

```
Class dc.movie.service.MovieCategoryService
{

ClassMethod GetAll() As %Status
{

    Return ##class(dc.movie.service.CrudUtilService).GetAll(##class(dc.movie.model.MovieCategory).%New())

}


ClassMethod GetOne(id As %Integer) As %Status
{

    Return ##class(dc.movie.service.CrudUtilService).GetOne(##class(dc.movie.model.MovieCategory).%New(), id)

}

ClassMethod Create() As %Status
{

    Return ##class(dc.movie.service.CrudUtilService).Create(##class(dc.movie.model.MovieCategory).%New())

}


ClassMethod Update(id As %Integer) As %Status
{

    Return ##class(dc.movie.service.CrudUtilService).Update(##class(dc.movie.model.MovieCategory).%New(), id)

}

ClassMethod Delete(id As %Integer) As %Status
{

    Return ##class(dc.movie.service.CrudUtilService).Delete(##class(dc.movie.model.MovieCategory).%New(), id)

}

}
```

`
5. Create ActorService.cls file inside service folder. Write the content:

`

```
Class dc.movie.service.ActorService
{


ClassMethod GetAll() As %Status
{

    Return ##class(dc.movie.service.CrudUtilService).GetAll(##class(dc.movie.model.Actor).%New())

}


ClassMethod GetOne(id As %Integer) As %Status
{

    Return ##class(dc.movie.service.CrudUtilService).GetOne(##class(dc.movie.model.Actor).%New(), id)

}


ClassMethod Create() As %Status
{

    Return ##class(dc.movie.service.CrudUtilService).Create(##class(dc.movie.model.Actor).%New())

}


ClassMethod Update(id As %Integer) As %Status
{

    Return ##class(dc.movie.service.CrudUtilService).Update(##class(dc.movie.model.Actor).%New(), id)

}


ClassMethod Delete(id As %Integer) As %Status
{

    Return ##class(dc.movie.service.CrudUtilService).Delete(##class(dc.movie.model.Actor).%New(), id)

}

}
```

`
6. Create CastingService.cls file inside service folder. Write the content:

`

```
Class dc.movie.service.CastingService
{


ClassMethod GetAll() As %Status
{

    Return ##class(dc.movie.service.CrudUtilService).GetAll(##class(dc.movie.model.Casting).%New())

}


ClassMethod GetOne(id As %Integer) As %Status
{

    Return ##class(dc.movie.service.CrudUtilService).GetOne(##class(dc.movie.model.Casting).%New(), id)

}


ClassMethod Create() As %Status
{

    Return ##class(dc.movie.service.CrudUtilService).Create(##class(dc.movie.model.Casting).%New())

}


ClassMethod Update(id As %Integer) As %Status
{

    Return ##class(dc.movie.service.CrudUtilService).Update(##class(dc.movie.model.Casting).%New(), id)

}


ClassMethod Delete(id As %Integer) As %Status
{

    Return ##class(dc.movie.service.CrudUtilService).Delete(##class(dc.movie.model.Casting).%New(), id)

}

}
```

`
7. Update the file MovieRESTApp.cls to create paths to all new service class methods. Write the content:

`

```
Class dc.movie.MovieRESTApp Extends %CSP.REST
{


Parameter CHARSET = "utf-8";
Parameter CONVERTINPUTSTREAM = 1;
Parameter CONTENTTYPE = "application/json";
Parameter Version = "1.0.0";
Parameter HandleCorsRequest = 1;

XData UrlMap [ XMLNamespace = "http://www.intersystems.com/urlmap" ]
{

<Routes>

<!-- Server Info -->
<Route Url="/" Method="GET" Call="GetInfo" Cors="true"/>

<!-- Swagger specs -->
<Route Url="/_spec" Method="GET" Call="SwaggerSpec" />


<!-- List all movies -->
<Route Url="/movies" Method="GET" Call="GetAllMovies" />

<!-- Get a movie -->
<Route Url="/movies/:id" Method="GET" Call="GetMovie" />

<!-- Get the movie casting -->
<Route Url="/movies/casting/:id" Method="GET" Call="GetMovieCasting" />

<!-- Create new movie -->
<Route Url="/movies" Method="POST" Call="CreateMovie" />

<!-- Update a movie -->
<Route Url="/movies/:id" Method="PUT" Call="UpdateMovie" />

<!-- Delete a movie -->
<Route Url="/movies/:id" Method="DELETE" Call="DeleteMovie" />


<!-- List all movie categories --> 
<Route Url="/categories" Method="GET" Call="GetAllMovieCategories" />

<!-- Get a movie category -->
<Route Url="/categories/:id" Method="GET" Call="GetMovieCategory" />

<!-- Create new movie category -->
<Route Url="/categories" Method="POST" Call="CreateMovieCategory" />

<!-- Update a movie category -->
<Route Url="/categories/:id" Method="PUT" Call="UpdateMovieCategory" />

<!-- Delete a movie category -->
<Route Url="/categories/:id" Method="DELETE" Call="DeleteMovieCategory" />


<!-- List all actors -->
<Route Url="/actors" Method="GET" Call="GetAllActors" />

<!-- Get a actor -->
<Route Url="/actors/:id" Method="GET" Call="GetActor" />

<!-- Create new actor -->
<Route Url="/actors" Method="POST" Call="CreateActor" />

<!-- Update a actor -->
<Route Url="/actors/:id" Method="PUT" Call="UpdateActor" />

<!-- Delete a actor -->
<Route Url="/actors/:id" Method="DELETE" Call="DeleteActor" />


<!-- List all castings -->
<Route Url="/castings" Method="GET" Call="GetAllCastings" />

<!-- Get a actor -->
<Route Url="/castings/:id" Method="GET" Call="GetCasting" />

<!-- Create new actor -->
<Route Url="/castings" Method="POST" Call="CreateCasting" />

<!-- Update a actor -->
<Route Url="/castings/:id" Method="PUT" Call="UpdateCasting" />

<!-- Delete a actor -->
<Route Url="/castings/:id" Method="DELETE" Call="DeleteCasting" />

</Routes>

}


/// List movies
ClassMethod GetAllMovies() As %Status
{

  Return ##class(dc.movie.service.MovieService).GetAll()

}


/// Get movie casting
ClassMethod GetMovieCasting(id As %Integer) As %Status
{

  Return ##class(dc.movie.service.MovieService).GetMovieCasting(id)

}


/// Get a movie
ClassMethod GetMovie(id As %Integer) As %Status
{

  Return ##class(dc.movie.service.MovieService).GetOne(id)

}


// Create a new movie
ClassMethod CreateMovie() As %Status
{

  Return ##class(dc.movie.service.MovieService).Create()

}


// Update a movie
ClassMethod UpdateMovie(id As %Integer) As %Status
{

  Return ##class(dc.movie.service.MovieService).Update(id)

}


// Delete a movie
ClassMethod DeleteMovie(id As %Integer) As %Status
{

  Return ##class(dc.movie.service.MovieService).Delete(id)

}


/// List movies categories
ClassMethod GetAllMovieCategories() As %Status
{

  Return ##class(dc.movie.service.MovieCategoryService).GetAll()

}


/// Get a movie category
ClassMethod GetMovieCategory(id As %Integer) As %Status
{

  Return ##class(dc.movie.service.MovieCategoryService).GetOne(id)

}


// Create a new movie category
ClassMethod CreateMovieCategory() As %Status
{

  Return ##class(dc.movie.service.MovieCategoryService).Create()

}


// Update a movie category
ClassMethod UpdateMovieCategory(id As %Integer) As %Status
{

  Return ##class(dc.movie.service.MovieCategoryService).Update(id)

}

// Delete a movie category
ClassMethod DeleteMovieCategory(id As %Integer) As %Status
{

  Return ##class(dc.movie.service.MovieCategoryService).Delete(id)

}

/// List actors
ClassMethod GetAllActors() As %Status
{

  Return ##class(dc.movie.service.TestActorService).GetAll()

}

/// Get an actor
ClassMethod GetActor(id As %Integer) As %Status
{

  Return ##class(dc.movie.service.ActorService).GetOne(id)

}

// Create a new actor
ClassMethod CreateActor() As %Status
{

  Return ##class(dc.movie.service.ActorService).Create()

}

// Update an actor
ClassMethod UpdateActor(id As %Integer) As %Status
{

  Return ##class(dc.movie.service.ActorService).Update(id)

}

// Delete an actor
ClassMethod DeleteActor(id As %Integer) As %Status
{

  Return ##class(dc.movie.service.ActorService).Delete(id)

}

/// List castings
ClassMethod GetAllCastings() As %Status
{

  Return ##class(dc.movie.service.CastingService).GetAll()

}

/// Get a casting
ClassMethod GetCasting(id As %Integer) As %Status
{

  Return ##class(dc.movie.service.CastingService).GetOne(id)

}

// Create a new casting item
ClassMethod CreateCasting() As %Status
{

  Return ##class(dc.movie.service.CastingService).Create()

}

// Update a casting
ClassMethod UpdateCasting(id As %Integer) As %Status
{

  Return ##class(dc.movie.service.CastingService).Update(id)

}

// Delete a casting
ClassMethod DeleteCasting(id As %Integer) As %Status
{

  Return ##class(dc.movie.service.CastingService).Delete(id)

}


/// General information
ClassMethod GetInfo() As %Status
{

  SET version = ..#Version
  SET fmt=##class(%SYS.NLS.Format).%New("ptbw")

  SET info = {
    "Service": "Movie API",
    "version": (version),
    "Developer": "Yuri Gomes",
    "Status": "Ok",
    "Date": ($ZDATETIME($HOROLOG))
  }

  Set %response.ContentType = ..#CONTENTTYPEJSON
  Set %response.Headers("Access-Control-Allow-Origin")="*"

  Write info.%ToJSON()

  Quit $$$OK

}


ClassMethod %ProcessResult(pStatus As %Status = {$$$OK}, pResult As %DynamicObject = "") As %Status [ Internal ]
{

  #dim %response As %CSP.Response

  SET tSC = $$$OK

  IF $$$ISERR(pStatus) {
    SET %response.Status = 500
    SET tSC = ..StatusToJSON(pStatus, .tJSON)

    IF $isobject(tJSON) {
      SET pResult = tJSON

    } ELSE {

      SET pResult = { "errors": [ { "error": "Unknown error parsing status code" } ] }

    }

  } 

  ELSEIF pStatus=1 {

    IF '$isobject(pResult){
      SET pResult = {

      }

    }

  }

  ELSE {

    SET %response.Status = pStatus

    SET error = $PIECE(pStatus, " ", 2, *)

    SET pResult = {

      "error": (error)

    }

  }


  IF pResult.%Extends("%Library.DynamicAbstractObject") {
    WRITE pResult.%ToJSON()

  }

  ELSEIF pResult.%Extends("%JSON.Adaptor") {
    DO pResult.%JSONExport()

  }

  ELSEIF pResult.%Extends("%Stream.Object") {
    DO pResult.OutputToDevice()

  }

  QUIT tSC

}


ClassMethod SwaggerSpec() As %Status
{

  Set tSC = ##class(%REST.API).GetWebRESTApplication($NAMESPACE, %request.Application, .swagger)

  Do swagger.info.%Remove("x-ISC_Namespace")
  Set swagger.basePath = "/movie-api" 
  Set swagger.info.title = "Movie API"
  Set swagger.info.version = "1.0"
  Set swagger.host = "localhost:52773"
  Return ..%ProcessResult($$$OK, swagger)

}

}
```

`
8. The files and folders to the final project are:
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/j084teyir18qq0aq4ifs.png)

9. Test your new methods acessing http://localhost:52773/swagger-ui/index.html.

Note 1: REST paths are following business topic in plural with /id when we need pass id to the entity and camel case to paths to.

Note 2: We use verb GET to queries, POST to new records, PUT to update record and DELETE to delete a record.

Note 3: In <Route Url="/movies/casting/:id" Method="GET" Call="GetMovieCasting" /> I used /casting indicating a second purpose (get the movie and it casting). This method runs ToJSON(), because is a DynamicArray ([]) with Dynamic items ({}).

Note 4: I created the CrudUtilService class utility to do generic CRUD methods, following the Dont Repeat Yourself principle.


Enjoy this tutorial!

Learning Path for beginners

Yuri Marx P. Gomes — Wed, 23 Feb 2022 12:48:07 +0000

The InterSystems Learning Website has many important iterative courses. So if you want to learn about InterSystems and start to work with InterSystems this is the path:

Overview:

Introduction to InterSystems Products and Technologies: https://learning.intersystems.com/course/view.php?id=1168&ssoPass=1

Fundamentals:

InterSystems ObjectScript Basics: https://learning.intersystems.com/course/view.php?id=959&ssoPass=1
InterSystems SQL Overview: https://learning.intersystems.com/course/view.php?id=960&ssoPass=1
InterSystems IRIS Objects Introduction: https://learning.intersystems.com/course/view.php?id=938&ssoPass=1
Migrate to IRIS (contribution from Kristina Lauer): https://learning.intersystems.com/course/view.php?name=MigratingToIRIS

Backend Development:

Setting Up RESTful Services: https://learning.intersystems.com/course/view.php?id=1298&ssoPass=1
Using JSON in InterSystems IRIS: https://learning.intersystems.com/course/view.php?id=972&ssoPass=1
Hands-On with InterSystems API Manager for Developers: https://learning.intersystems.com/course/view.php?id=1747&ssoPass=1
Building Data-Driven Web Apps: https://learning.intersystems.com/course/view.php?id=1237&ssoPass=1

Integration Development:

Integration Architecture: https://learning.intersystems.com/course/view.php?id=908&ssoPass=1
Building BPL Business Processes: https://learning.intersystems.com/course/view.php?id=1290&ssoPass=1
Building Custom Business Operations: https://learning.intersystems.com/course/view.php?id=1288&ssoPass=1
Building Custom Business Services: https://learning.intersystems.com/course/view.php?id=1291&ssoPass=1
Data Transformations Basics: https://learning.intersystems.com/course/view.php?id=1170&ssoPass=1
Very nice external resource: https://github.com/grongierisc/formation-template
Interoperability Course (contribution from @guillaume Rongier): https://community.intersystems.com/post/ensemble-interoperability-traini...

Analytics Development:

IRIS BI Architect: https://learning.intersystems.com/course/view.php?id=1752&ssoPass=1
IRIS BI Analyzer: https://learning.intersystems.com/course/view.php?id=1793&ssoPass=1
BI Data Models: https://learning.intersystems.com/course/view.php?id=1796&ssoPass=1
InterSystems IRIS Adaptive Analytics Essentials: https://learning.intersystems.com/course/view.php?id=1715&ssoPass=1
Getting Started with InterSystems Reports: https://learning.intersystems.com/course/view.php?id=1538&ssoPass=1
Hands-On with IntegratedML: https://learning.intersystems.com/course/view.php?id=1535&ssoPass=1 Paths from learning.intersystems.com (contribution from @jenny Ames)

If you're looking to get started, you can also personalize which paths you take based on what you're trying to do:

Building a Server-Side Application with InterSystems: https://learning.intersystems.com/course/view.php?name=IRIS%20Server%20Side
Building Business Integrations with InterSystems IRIS: https://learning.intersystems.com/course/view.php?name=InteroperabilityForBusiness
Analyzing Data with InterSystems IRIS BI: https://learning.intersystems.com/course/view.php?name=BuildingAnalyzer
Build Data Models Using Adaptive Analytics: https://learning.intersystems.com/course/view.php?name=AALP
Delivering Data Visually with InterSystems Reports: https://learning.intersystems.com/course/view.php?name=InterSystemsReports
Predicting Outcomes with IntegratedML in InterSystems IRIS: https://learning.intersystems.com/course/view.php?name=Learn%20IntegratedML

Alternately, there are a several larger paths for implementation partners (or others who will be doing everything, paths are open to all) here: https://learning.intersystems.com/course/view.php?name=IPLPs

Mapping Intersystems IRIS documentation to a Corporate Data Architecture

Yuri Marx P. Gomes — Thu, 17 Sep 2020 15:53:53 +0000

If you need write your organization Data Architecture and map to the InterSystems IRIS, consider this Data Architecture Diagram and references to the Intersystems IRIS documentation above:

Architecture mapping:

Integrate SAP and InterSystems IRIS using OData

Yuri Marx P. Gomes — Wed, 12 Aug 2020 14:48:58 +0000

SAP offers a broad support to OData in all your products. So, OData can be an excellent option to exchange data between SAP and InterSystems IRIS.

Follow the instructions in the article https://community.intersystems.com/post/odata-and-intersystems-iris to expose your IRIS data as REST OData services.
To consume InterSystems IRIS data from SAP using OData, follow these steps (credits to the next steps of this tutorial: https://sapyard.com/sapui5-for-abapers-consuming-odata-service-from-sapui5-application-crud-operations/) :
Create a new SAPUI5 application by the name crud_demo.
Create a XML view ‘crud_demo.view’. Write below code in it.

<core:View xmlns:core="sap.ui.core" xmlns:mvc="sap.ui.core.mvc" xmlns="sap.m"
controllerName="crud_demo.crud_demo" xmlns:html="http://www.w3.org/1999/xhtml" xmlns:l="sap.ui.commons.layout">
<Page title="CRUD Operations">
<content>
<l:AbsoluteLayout width="10rem" height="10rem"></l:AbsoluteLayout>
<VBox xmlns="sap.m" id="vboxid"> 
<items> 
<HBox xmlns="sap.m"> 
<items> 
<l:AbsoluteLayout width="20px" height="20px"></l:AbsoluteLayout> 
<Button xmlns="sap.m" id="cbtn" press="oDataCall" text="Create"></Button>
<l:AbsoluteLayout width="20px" height="20px"></l:AbsoluteLayout>
<Button xmlns="sap.m" id="rbtn" press="oDataCall" text="Read"></Button>
<l:AbsoluteLayout width="20px" height="20px"></l:AbsoluteLayout>
<Button xmlns="sap.m" id="ubtn" press="oDataCall" text="Update"></Button>
<l:AbsoluteLayout width="20px" height="20px"></l:AbsoluteLayout>
<Button xmlns="sap.m" id="dbtn" press="oDataCall" text="Delete"></Button>
</items>
</HBox>
<HBox xmlns="sap.m"> 
<items> 
<l:AbsoluteLayout width="20px" height="20px"></l:AbsoluteLayout>
<Input xmlns="sap.m" id="uniqueid" placeholder="ID" value="1"></Input>
<l:AbsoluteLayout width="20px" height="20px"></l:AbsoluteLayout>
<Input xmlns="sap.m" id="nameid" placeholder="Name" value="test"></Input>
<l:AbsoluteLayout width="20px" height="20px"></l:AbsoluteLayout>

<l:AbsoluteLayout width="20px" height="20px"></l:AbsoluteLayout>
<Input xmlns="sap.m" id="mobid" placeholder="Mobile" value="8888888888"></Input>
</items>
</HBox>
<HBox xmlns="sap.m"> 
<items> 
<l:AbsoluteLayout width="20px" height="20px"></l:AbsoluteLayout>
<Table xmlns="sap.m"
id="userdatatable" headerText="User Data"> 
<items>
<ListItemBase xmlns="sap.m" id="id1"></ListItemBase>
</items> 
<columns> <!-- sap.m.Column -->
<Column xmlns="sap.m"> <header> <Text xmlns="sap.m" text="Id" ></Text></header></Column>
<Column xmlns="sap.m"> <header> <Text xmlns="sap.m" text="Name" ></Text></header></Column>
<Column xmlns="sap.m"> <header> <Text xmlns="sap.m" text="Email" ></Text></header></Column>
<Column xmlns="sap.m"> <header> <Text xmlns="sap.m" text="Mobile" ></Text></header></Column>
</columns>
</Table>
</items>
</HBox> 
</items> <!-- sap.ui.core.Control -->
</VBox>
</content>
</Page>
</core:View>

Create crud_demo.controller.js. Write below code in it.

onInit: function() {
that = this;
// Create Model Instance of the oData service
var oModel = new sap.ui.model.odata.v2.ODataModel("/sap/opu/odata/sap/ZCRUD_DEMO_SRV");
sap.ui.getCore().setModel(oModel, "myModel");
},
oDataCall:function(oEvent)
{
// call oData service's function based on which button is clicked.
debugger;
var myModel = sap.ui.getCore().getModel("myModel");
myModel.setHeaders({
"X-Requested-With" : "X"
});
// CREATE******************
if ('Create' == oEvent.oSource.mProperties.text) {
var obj = {};
obj.id = that.getView().byId("uniqueid").getValue();
obj.name = that.getView().byId("nameid").getValue();
obj.email = that.getView().byId("emailid").getValue();
obj.mobile = that.getView().byId("mobid").getValue();
myModel.create('/userdataSet', obj, {
success : function(oData, oResponse) {
debugger;
alert('Record Created Successfully...');
},
error : function(err, oResponse) {
debugger;
alert('Error while creating record - '
.concat(err.response.statusText));
}
});
}
// READ******************
else if ('Read' == oEvent.oSource.mProperties.text) {
var readurl = "/userdataSet?$filter=(id eq '')";
myModel.read(readurl, {
success : function(oData, oResponse) {
debugger;
var userdata = new sap.ui.model.json.JSONModel({
"Result" : oData.results
});
var tab = that.getView().byId("userdatatable");
tab.setModel(userdata);
var i = 0;
tab.bindAggregation("items", {
path : "/Result",
template : new sap.m.ColumnListItem({
cells : [ new sap.ui.commons.TextView({
text : "{id}",
design : "H5",
semanticColor : "Default"
}), new sap.ui.commons.TextView({
text : "{name}",
design : "H5",
semanticColor : "Positive"
}), new sap.ui.commons.TextView({
text : "{email}",
design : "H5",
semanticColor : "Positive"
}), new sap.ui.commons.TextView({
text : "{mobile}",
design : "H5",
semanticColor : "Positive"
}), ]
})
});
},
error : function(err) {
debugger;
}
});
} 
// UPDATE******************
if ('Update' == oEvent.oSource.mProperties.text) {
var obj = {};
obj.id = that.getView().byId("uniqueid").getValue();
obj.email = that.getView().byId("emailid").getValue();
var updateurl = "/userdataSet(id='"
+ that.getView().byId("uniqueid").getValue() + "')";
 
myModel.update(updateurl, obj, {
success : function(oData, oResponse) {
debugger;
alert('Record Updated Successfully...');
},
error : function(err, oResponse) {
debugger;
alert('Error while updating record - '
.concat(err.response.statusText));
}
});
} 
// DELETE******************
if ('Delete' == oEvent.oSource.mProperties.text) {
var delurl = "/userdataSet(id='"
+ that.getView().byId("uniqueid").getValue() + "')";
myModel.remove(delurl, {
success : function(oData, oResponse) {
debugger;
alert('Record Removed Successfully...');
},
error : function(err, oResponse) {
debugger;
alert('Error while removing record - '
.concat(err.response.statusText));
}
});
}
}

Save, Deploy and Run the application. You should be able to run the application using below URL http://hostname:8000/sap/bc/ui5_ui5/sap/zcrud_demo/index.html.

Output:

OData and InterSystems IRIS

Yuri Marx P. Gomes — Wed, 12 Aug 2020 14:37:41 +0000

What is the OData

OData (Open Data Protocol) is an ISO/IEC approved, OASIS standard that defines a set of best practices for building and consuming RESTful APIs. OData helps you focus on your business logic while building RESTful APIs without having to worry about the various approaches to define request and response headers, status codes, HTTP methods, URL conventions, media types, payload formats, query options, etc. OData also provides guidance for tracking changes, defining functions/actions for reusable procedures, and sending asynchronous/batch requests (source: OData.org).

The OData use cases

Deploy data as REST Services with an interoperable format without development effort;
Allows BI, data visualization, ERP, CRM, ESB, Workflow tools and engines consume data using REST without development effort;
Virtualize corporate data in API Management tools;
OData advocates a standard way of implementing REST APIs that allows for SQL-like querying capabilities using these RESTful APIs. OData is essentially SQL for the web built n top of standard protocols – HTTP, JSON & ATOM – while leveraging the REST architecture style (progress.com);
OData has a broad adoption, see:

OData helps to implement FHIR: FHIR, or Fast Healthcare Interoperability Resources Specification, is a standard for exchanging healthcare information electronically. In order to make FHIR truly interoperable, it is recommended that systems use the rules specified by OData specification for the $search parameter. Further, FHIR also uses OAuth in order to establish a trusted relationship with the client for an extra layer of security (progress.com);
OData supports pagination, batch requests and different formats like JSON, ATOM, XML, etc.

OData and InterSystems IRIS

The InterSystems IRIS does not support OData but is possible use the OData Server for InterSystems IRIS to allows expose Persistent classes as REST.

Follow these instructions:

Clone the source code of the IRIS OData Server: git clone https://github.com/yurimarx/isc-iris-odata.git
Go to: isc-iris-odata folder
Execute: mvnw install (MS Windows) or ./mvnw install (linux or mac)
Execute: docker build -t odata:1.0.0 .
Execute: docker run -p 8080:8080 odata:1.0.0. Your OData Server started:

Start your InterSystems IRIS instance with any persistent class, in my ca
In your browser access: http://localhost:8080/. Set parameters on the screen:

This is parameters to my instance. Set the parameters to your IRIS instance. In namespace set your iris namespace, in the schema, the SQL Table schema, and in the port, the port to your JDBC database connection.
Press submit and RELOAD YOUR ODATA SERVER DOCKER INSTANCE TO APPLY PARAMETERS.
Access http://localhost:8080/odata.svc/ to see all persistent classes to your IRIS schema. In my case is:

To navigate to a persistent class browse: http://localhost:8080/odata.svc/<Persistent_Class> e.g.: http://localhost:8080/odata.svc/Animal
The OData server list Animal data, see:

To see in JSON format browse: http://localhost:8080/odata.svc/Animal?$format=application/json. See:

To see details about a row browse: http://localhost:8080/odata.svc/Animal(8)?$format=application/json
To delete send a DELETE in your postman with http://localhost:8080/odata.svc/Animal(8)
To insert send a POST in your postman with http://localhost:8080/odata.svc/Animal and a JSON body with property name and value pairs, like:

So you can do all CRUD operations with your persistent classes.
Many other features will be released in the future, if IRIS OData Server get the community adoption.

Thanks!