DEV Community: Kumologica

Serverless integration with Salesforce using Kumologica

ab73863 — Mon, 05 Apr 2021 14:34:15 +0000

Salesforce is a popular CRM platform used by many enterprises. Integration of salesforce modules with different enterprise systems are very common. There are multiple ways by which salesforce objects can be integrated .

SOQL query (Pulling the records)
Platform Events (Pushing the events)
CDC (Pushing the events)
OutboundMessages (Pushing the events using webhooks)

Most of the real time integrations with Salesforce works with PE(platform events), CDC and OutboundMessages. There are also solutions where SOQL query works in complementary with PE, CDC or OutboundMessages (Webhooks).

In this article we will see how to send a Salesforce outbound message to the target systems using Kumologica flow that runs on serverless infrastructure. For those who are new to Kumologica I would recommend to go through our articles and youtube videos to get an insight.

Concept

Before we begin let’s try to understand the working mechanism behind outbound messages in Salesforce.
OutboundMessages in salesforce is part of workflow in salesforce. The underlying mechanism is based on webhook notification. Whenever a change happens in any salesforce objects an associated outbound message is triggered by the workflow rule. The change can be an insert, update or a delete of an salesforce object. The outbound message is configured with a HTTP post endpoint. Salesforce outbound message will invoke the endpoint to send a SOAP payload having the appropriate object fields (eg : Case, Contact, etc).

Use Case

In this use case we will be using the Salesforce contact object. Whenever a new contact is created in Salesforce an event is triggered which will be sent to the target system by Kumologica flow. Here the target system is Amazon SQS queue.

The HTTP endpoint implementation will be done using Kumologica designer. The Kumologica flow will be then deployed on AWS Lambda. After the deployment we will copy the gateway endpoint which is then configured on the outboundMessage section in Salesforce.

Prerequisite

Salesforce account — For configuring the outbound message connecting contact module.
Kumologica Designer — Download the designer for building the flow.
AWS account — For running the Kumologica flow as lambda and for hosting the SQS queue.

Implementation

We will be creating ContactInboundService flow in Kumologica designer. This flow will expose the endpoint on the resource path /sfdc.

ContactInboundService flow

Steps

Open Kumologica Designer, click the Home button and choose Create New Kumologica Project.
Enter name (for example ContactInboundService), select directory for project.
Press Create Button.
Drag and drop EventListener node to the canvas the provide the following configuration.


Display Name :  POST /sfdc
Provider : AWS
Event Source : Amazon API gateway
Verb : POST
URL : /sfdc

Add a Logger node and wire with the EventListener node. Provide the following message.


'Request received - ' & msg.payload

Drag and drop the Set-Property node to the canvas and wire it with the logger node. Provide the following configuration.


Display Name :  ConvertToString
Rules :
   Set : msg.payload
   To  : J: $string(msg.payload)

Note: J : is the JSONata expression to be selected from the drop down.

Add the SQS node to canvas and wire it with the Set-Propery node. Provide the following configuration.


Operation : SendMessage
QueueUrl : <<SQS queue url>>
MessageBody : msg.payload

Note: Ensure that you have an SQS queue created in your AWS account and you have the queue url for configuring in the SQS node. Refer : https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-getting-started.html#step-create-queue

Add EventListener End node and wire with the SQS node. Provide the following configuration.


Display Name :  Success
Status Code : 200
Content-Type : application/json
Payload : {"status": "completed"}

Deployment

Select AWS icon on the right hand side vertical tab of Kumologica designer, select your AWS Profile.

Note: If you haven’t mapped the your local AWS profile with the designer then you may follow the below video to configure it.

Set profile video

Click Connect. If successfully connected will show the rest of the configuration options.
Set the Memory to 512mb and Timeout as 20 seconds.
Go to “Trigger” section under cloud tab and select the Amazon API Gateway trigger.

Post successful deployment copy the endpoint url from the Kumologica terminal. We will need this endpoint for configuring the OutboundMessage in Salesforce.


Url eg: https://ii48n22.execute-api.ap-southeast-2.amazonaws.com/test

Configuring OutboundMessage in Salesforce

Go to your salesforce Account and search for Outbound Messages.
On the Outbound Message screen , create a “New Outbound Message”
Select the object as “Contact” and Click Next. Provide the following details.

Click Save. Now you have created an Outbound Message.
Let’s map the Outbound Message to the workflow rule.

The above rule configuration ensures that anytime a contact is created with the description that starts with “ABC” the workflow will invoke the Outbound Message.
Now we will map the “ContactWebhook” OutboundMessage we have created in the earlier step to this workflow rule we are configuring.

From “Add Workflow Action” click on “Select Existing Action”.

From the type select “OutboundMessage”.

Now you will see the “ContactWebhook” in the Available Actions list. Add it and Save.

Now we are ready to rock n roll.

Testing

Go to the Salesforce contact module and create a new contact.
Go to the AWS account and look into the SQS queue. You will observe that a SOAP message is now available in the queue.

Migration from Mulesoft to Kumologica — Part 2

ab73863 — Sat, 12 Dec 2020 21:53:09 +0000

In our previous article “Migration from Mulesoft to Kumologica — Part 1” we took you through the basic nodes in Kumologica that can replace the Mulesoft processors. We also discussed about some of the popular patterns used in Kumologica and Mulesoft. If you haven’t got chance to read the article I would suggest to go through our part 1 article to get the basics ready for the migration exercise.

This is the second part of our four part series which will take a deep dive through a real-time use case of migrating a service that is developed on Mulesoft. The Mulesoft service described in this article will cover some of the patterns mentioned in our earlier article and we will learn how to migrate this service into Kumologica without negotiating the pattern.

For those who are new to Kumologica I would recommend to go through our articles and youtube videos to get an insight. Kumologica is one of the early players in this space which brings the benefits of the low-code integration capability of traditional integration like Mulesoft to the new world of Serverless. It provides a drag and drop visual designer and low code approach to ensure the greatest speed and flexibility to support a wide range of integrations compatible with most of the platforms and cloud providers.

Use case

An order processing service for an e-commerce platform is developed in Mulesoft Anypoint studio and deployed on CloudHub. The service consists of an API that accepts the order processing request (experience api) from the e-commerce UI and puts into Amazon SQS queue. Another service (process) that listens to the Amazon SQS queue to get the request message and send to two different service in parallel for processing i.e packaging service(out of scope) and logistic service(out of scope). In case of a failure the order processing request will be pushed into the re-processing queue in Amazon SQS. This design is typically based on API led approach of Mulesoft.

Order Processing Service (Mulesoft)

The experience API in Order processing service is named as exp-orderprocessing-service. The e-commerce platform UI invokes the exp-orderprocessing-service which accepts the request message and publish into the SQS queue (order_process_queue). The service transforms the request message into a canonical structure before publishing to the queue. In case of a failure the error the will be notified via email.

exp-orderprocessing-service

The process API in Order processing service is named as biz-orderprocessing-service. The service listens to the order_process_queue which receives the canonical message which will be transformed and send to packaging and logistics service in parallel for further processing. Mulesoft service uses scatter gather pattern processors to broadcast the message to the services in parallel. In case of a failure the payload will be submitted to re-process queue (order_reprocess_queue) and the error will be notified via email.

Migrating to Kumologica

In this exercise we have two Mulesoft service to migrate which are developed on Mule 3.9.0. The first service to migrate will be the experience API (exp-orderprocessing-service) and the second service is the process API (biz-orderprocessing-service).

Prerequisite

Kumologica designer installed in your machine. https://kumologica.com/download.html

Steps

exp-orderprocessing-service

Open Kumologica Designer, click the Home button and choose Create New Kumologica Project.
Enter name (exp-orderprocessing-service) and select directory for project.
press Create Button.
Drag and drop EventListener node to the canvas. From mulesoft flow HTTP listener copy the Name, Path and the Method. Open the EventListener settings and provide as given below.

Name = Name copied from Mulesoft HTTP listener
Path = Path copied from Mulesoft HTTP listener
Verb = Method copied from Mulesoft HTTP listener
Event Source : Amazon API gateway
Add Logger node and provide the message provided in Mulesoft flow.

Name = Name copied from Mulesoft Logger
Message = Message copied from Mulesoft Logger

Edit the MEL expression as given below.

**MEL expression in logger**
#[{"service name" : "exp-orderprocessing-service","object": "order","payload": **message**.**payload**,"logpoint": "Entry"}]

**Modified expression in Kumologica logger
**{"service name" : "exp-orderprocessing-service","object": "order","payload": **msg**.**payload**,"logpoint": "Entry"}

Add datamapper and provide the following.

Dataweave expression in Mulesoft

{
"tx_orderid": payload.OrderID,
"tx_product": {
"tx_productname": payload.Product.'ProductName',
"tx_productid": payload.Product.ProductID,
"tx_sku": payload.SKU,
"tx_desc": {
"Colour": payload.Description.Colour,
"Width": payload.Description.Width,
"Height": payload.Description.Height,
"Depth": payload.Description.Depth,
"Weight": payload.Description.Weight
},
"tx_qty": payload.Quantity
},
"t_processdate" : now(),
"t_deldate" : payload.DelDate,
"t_addr" : {
"unit" : payload.Address.Unit,
"strt" : payload.Address.Street,
"state" : payload.Address.State,
"country" : payload.Address.Country,
"pin" : payload.Address.Pin
}
}

Modified expression in Kumologica

{
"tx_orderid": msg.payload.OrderID,
"tx_product": {
"tx_productname": msg.payload.Product.'ProductName',
"tx_productid": msg.payload.Product.ProductID,
"tx_sku": msg.payload.SKU,
"tx_desc": {
"Colour": msg.payload.Description.Colour,
"Width": msg.payload.Description.Width,
"Height": msg.payload.Description.Height,
"Depth": msg.payload.Description.Depth,
"Weight": msg.payload.Description.Weight
},
"tx_qty": msg.payload.Quantity
},
"t_processdate" : now(),
"t_deldate" : msg.payload.DelDate,
"t_addr" : {
"unit" : msg.payload.Address.Unit,
"strt" : msg.payload.Address.Street,
"state" : msg.payload.Address.State,
"country" : msg.payload.Address.Country,
"pin" : msg.payload.Address.Pin
}
}
Add Amazon SQS node and provide the following

Name = Name copied from Mulesoft SQS connector
QueueUrl = Queue url copied from Mulesoft SQS connector
MessageBody = msg.payload
Add Logger node and follow procedure in step 5.
Add EventListener End node and provide the following. EventListener End node is the final node in any Kumolgica flow and this step is the only additional component when compared to Mulesoft flow.

Name : Completed
Payload : {"status" : "completed"}
Status Code : 200
Content-Type : application/json

For the exception handling

Add the catch node.
Add the logger node and wire to catch node and follow as in step 5.
Add the Email node and wire to logger node. Provide the following.

Name = Name copied from Mulesoft SMTP node
Host = Host copied from Mulesoft SMTP node
Port = Port copied from Mulesoft SMTP node
Username = Username copied from Mulesoft SMTP node
Password = Password copied from Mulesoft SMTP node
To = To copied from Mulesoft SMTP node
From = From copied from Mulesoft SMTP node
Cc = Cc copied from Mulesoft SMTP node
Body = msg.payload
Add EventListener End node and provide the following.

Name : Failure
Payload : {"status" : "Failed"}
Status Code : 500
Content-Type : application/json

biz-orderprocessing-service

Open Kumologica Designer, click the Home button and choose Create New Kumologica Project.
Enter name (biz-orderprocessing-service) and select directory for project.
press Create Button.
Drag and drop EventListener node to the canvas and provide the settings as given below.

Name : Name from Mulesoft SQS listener connector
Event Source : SQS

Note : SQS queue is attached to the biz-orderprocessing-service flow during the deployment under the cloud tab and hence queue url details is not provided in this node. We will be covering this in detail in our next article.

Add Logger node and provide the message provided in Mulesoft flow.

Name = Name copied from Mulesoft Logger
Message = Message copied from Mulesoft Logger

Edit the MEL expression as given below.

MEL expression in logger
#[{"service name" : "biz-orderprocessing-service","object": "order","payload": message.payload,"logpoint": "Entry"}]

**Modified expression in Kumologica logger
**{"service name" : "biz-orderprocessing-service","object": "order","payload": msg.payload,"logpoint": "Entry"}

Add the Set-Property node to store the payload. This is required for message re-processing incase of a failure.

Set
$vars.setpayload
from
msg.payload
Add the Scatter node and wire to Set-Property node.
Add two Datamapper node and wire both the datamapper node to the Scatter node. Follow the procedure as in step 6 of exp-orderprocessing-service.
Add two HTTP Req node and wire the each node to the datamapper node. From the mulesoft copy the details from HTTP requestor node from InvokePackagingService and InvokeLogisticsService. Provide the copied detail as given below .

Name : Name of the Mulesoft HTTP Requestor node
Path : Path of the Mulesoft HTTP Requestor node
Method : Path of the Mulesoft HTTP Requestor node
Header : Header of the Mulesoft HTTP Requestor node
Add the logger node after both HTTP Req node and follow as in step 5.
Add the Gather node to the canvas and wire both the logger nodes to the input terminal of gather node.
Add the logger node for Exit point. Follow the procedure as in step 5.
Add EventListener End node and provide the following. EventListener End node is the final node in any Kumolgica flow and this step is the only additional component when compared to Mulesoft flow.

Name : Completed
Payload : {"status" : "completed"}
Status Code : 200
Content-Type : application/json

For the exception handling

Add the catch node.
Add the logger node and wire to catch node and follow the procedure as in step 5.
Add the SQS node and provide the following.

Name = Name copied from Mulesoft SQS connector
QueueUrl = Queue url copied from Mulesoft SQS connector
MessageBody = vars.storepayload
Add the Email node and wire to logger node. Provide the following.

Name = Name copied from Mulesoft SMTP node
Host = Host copied from Mulesoft SMTP node
Port = Port copied from Mulesoft SMTP node
Username = Username copied from Mulesoft SMTP node
Password = Password copied from Mulesoft SMTP node
To = To copied from Mulesoft SMTP node
From = From copied from Mulesoft SMTP node
Cc = Cc copied from Mulesoft SMTP node
Body = msg.payload
Add EventListener End node and provide the following.

Name : Failure
Payload : {"status" : "Failed"}
Status Code : 500
Content-Type : application/json

Conclusion

This article has shown you how easy to move a Mulesoft flow to Kumologica without any changes to the flow pattern and with very minimal changes to the MEL expressions used in Mulesoft. Stay tuned for our next part in this series with more interesting use case and migration tricks.

Remember Kumologica is totally free to download and use. Go ahead and give it a try, we would love to hear your feedback. For more details and enterprise support on moving your Mulesoft flows to Kumologica connect with us .

Migration from Mulesoft to Kumologica— Part 1

ab73863 — Fri, 27 Nov 2020 11:18:35 +0000

In our previous article “A journey from MuleSoft to Serverless Integration” we discussed the need, benefits, use case and an example of moving a Mulesoft flow on to serverless based implementation using Kumologica. Based on the feedback we have received from our users we are introducing an article series demonstrating the migration of popular flow patterns and components from Mulesoft to Kumologica.

This is the first part of our four part series we are publishing which takes you through the steps for migrating your application from Mulesoft to a serverless integration using Kumologica. Mulesoft follows the EAI (Enterprise application Integration) patterns based on which the Mulesoft processors and connectors are aligned. We will discuss some of these patterns as we move through the series.

Flow elements

As a developer, before taking the initial steps in migrating the from Mulesoft it is important to understand the basic elements of Kumologica that can replace the Mulesoft processors. A Mulesoft flow consists of connectors and processors. In Kumologica this is called as nodes. Nodes in Kumologica can be connectors as well as processors.

Flows and subflows

In Mulesoft the flow consists of a source that can trigger the processing section which holds set of processors. eg. HTTP listener in source section of flow triggering the logger in processing section. The subflow unlike flow consists of only processing section.

Mulesoft flow and a subflow

In Kumologica the flow block is represented with an Eventlistener and Eventlistener End node. The Evenlistener node is the triggering section and all the processing nodes placed between the Eventlistener node and Eventlistener End node. The subflow equivalent in Kumologica is Processor In and Processor Out node.

Kumologica flow and subflow

Message routing

In Mulesoft, the popular message routing processors are choice, collection splitter, collection aggregator and scatter-gather.

Route based on condition

Choice router is the processor in Mulesoft for routing the flow based on a condition.

Mulesoft flow with choice block

In Kumologica we use switch node which routes the flow based on a given condition.

Kumologica flow with switch node

Split message to fragments

In Mulesoft a collection object (array) is split into individual records or fragments and send to next processor using collection splitter. The end of collection iteration is managed by collection aggregator.

Individual records send to database after split

In Kumologica an array of record is split into individual records or fragments and send to next node using split node and join node. Here the split is equivalent to collection splitter and join is equivalent to collection aggregator.

Individual records send to database after split

Sending message to multiple targets

In Mulesoft a of copy message is send to multiple targets and the response from each target is collected using scatter-gather processor.

Mule flow with scatter-gather processor

In Kumologica a copy of message is send to multiple targets the response from each target is collected using scatter and gather node.

Scatter gather node in Kumologica flow

Message Transformation

In Mulesoft, the popular message transformation processors are dataweave, JSON-Object, XML-Object etc. Dataweave is the heavily used data mapping processor in Mulesoft which following dataweave scripting expression language.

Dataweave editor

In Kumologica, Datamapper plays the role of data mapping. It follows JSONata based scripting. For more details on JSONata.

Datamapper editor

Custom logic implementation

In Mulesoft the custom logic is implemented primarily using Java component. In Kumologica this is achieved using the function node. The function node allows you the write the code in javascript.

Message structure

Mulesoft message structture

Kumologica message structure

Mule message structure and Kumologica message structure are almost similar structure with very minimal difference in hierarchy.

Accessing message structure

In Mulesoft message structure is accessed using MEL expression or using dataweave script. In Kumologica message structure is accessed using JSONata expression.

Accessing payload in Mulesoft

In mule 3.9 and lower
#[message.payload]
In mule 4.0 and above
#[payload]

Accessing payload in Kumologica

msg.payload

Accessing variables in Mulesoft

#[flowVars.<<variable name>>]

Accessing variables in Kumologica

vars.<<variable name>>

Conclusion

This article would have given you the basic understanding on the flow construct that we will be using and referring in our next article in the series where we will migrate a real time Mule flow to Kumologica and deep dive further into each flow elements.

How to split the records in Kumologica

ab73863 — Sun, 08 Nov 2020 07:07:02 +0000

In this article I am going to demonstrate on how to split a given input object or record in Kumologica. In order to split an object or an array or a string, Kumologica provides split and join node which are required to be used in combination. You may wire as many parallel path between split and a join node. Every path wired between split and join node will be invoked for each iterated or record splitted.

Split node supports

Iterating or lopping through an array.
Tokenizing a simple string based on provided character separator.
Splitting a object based on keys.

Join node is always used along with split node in order to close the loop or to combine the response from different flow path initiated from the split node.

Let's go through each of the above mentioned operations supported by the Split node.

Iterating or lopping through an array

When you have a requirement for a an array to be iterated or looped through and do some actions with the each record or object that has been iterated then you can choose this option.

Configuration in Split node in Kumologica

Splitting a simple string based on provided character separator

When you have a requirement for a string to be tokenized based on some separator then you may use this option.

Configuration in Split node in Kumologica

Splitting a object based on keys

When you have a requirement for iterating through an object based on its keys then you may choose this option.

Configuration in Split node in Kumologica

Flow demonstrating the split and join

References

https://docs.kumologica.com/docs/references/CoreRouting.html#split

https://docs.kumologica.com/docs/guide/GettingStarted.html

https://www.youtube.com/results?search_query=kumologica

Building a serverless IoT backend integration

ab73863 — Tue, 11 Aug 2020 14:28:24 +0000

Having a lightweight protocol which delivers messages over low bandwidth or unreliable networks is the foundation for a system that can facilitate the transports of data in near real time.

MQTT is one such protocol which designed to overcome following limitations:

Network limitation

Limited network bandwidth
High latency
Fragile connections

Device Limitation

Limited memory
Processing capabilities

MQTT is an open OASIS and ISO standard lightweight, publish-subscribe network protocol that transports messages between devices. The protocol usually runs over TCP/IP; however, any network protocol that provides ordered, lossless, bi-directional connections can support MQTT.

In this article we will be building a serverless backend service on AWS lambda which will integrate with a MQTT broker for publishing the passenger service request to the right group of taxi drivers. We will be building the passenger request API using Kumologica. Kumologica is a free low-code development tool to build serverless integrations. You can learn more about Kumologica in this medium article or subscribe to our YouTube channel for the latest videos.

Use case

In this use case we are having a passenger making a request from his mobile app for a taxi. The mobile app makes a call to the Kumologica passenger requester API which accepts the request and verifies in the database for the availability of the cab. Based on the availability status and the type of taxi requested by the passenger the flow will route the message to publish in the appropriate taxi topic on MQTT broker. The driver app will receive the message from the MQTT broker.

High level use case flow

Note: In this article we are not covering the section where the driver app accepts the request and sends confirmation to the passenger. If you are interested in this section please let us know and we will put another article covering this use case.

Prerequisite

Kumologica designer installed in your machine. https://kumologica.com/download.html
https://www.hivemq.com/public-mqtt-broker/
DynamoDB table with the following records.

DynamoDB table structure and data

Implementation

In order to implement the Passenger request API we will be following the below given logic

Flow logic

Open Kumologica Designer, click the Home button and choose Create New Kumologica Project.

Enter name (for example PassengerRequestAPI), select directory for project and switch Source into From Existing Flow …

2.Copy and Paste the following flow

3.press Create Button.

Install MQTT Node.

MQTT node is a contribution node and must be explicitly installed in Kumologica Designer. Press Add more nodes button located in the bottom left corner of designer. You will be presented screen with all contribution nodes that are currently available. Choose MQTT node, press Install button. After few seconds, plugin will be installed and you will be prompted to restart Kumologica Designer. MQTT node will be located under the new category: Platforms after restart.

Add more nodes panel

You should be seeing two flows as given below on the designer canvas.

The first flow is the API endpoint with the resource path /passenger/request. This flow accepts the request from the mobile app and drops the message to Amazon SQS queue (PassengerSQS). This is done as an event driven model in order to acknowledge back (to mobile app)quickly.

The second flow is having the logic to fetch the driver data from the DynamoDB based on the areacode (area_code primary key) which is passed as request by the passenger. The flow picks the message from the SQS queue and invokes the DynamoDB. The flow will then check the availability field in the dynamodb response data. If the availability is having the value as yes then the flow checks for the appropriate MQTT topic to be selected. This done by checking the vehicle_type field in the dynamodb response data. Based on the vehiclet_type the MQTT node is invoked to publish the message to the MQTT broker topic in Hive.

Note: The configuration for the Hive MQTT broker for this usecase is available in the HiveMQTT site

Note: To try out the above flow ensure to furnish the settings of MQTT node, SQS node and AWS DynamoDB node with appropriate connection configuration.

Deploy flow to AWS Lambda

Select CLOUD tab on the right panel of Kumologica designer, choose AWS Profile and press Deploy button. This will trigger build and deployment of flow into AWS Lambda. On the trigger section you need to add to triggers.

API Gateway
Amazon SQS

Trigger configuration for API gateway and SQS

Testing

Paste the below give payload in your postman or any rest client of your choice.

{
"areacode" : "2331",
"name" : "John",
"seat" : "4"
}

Provide the urls as https://<>/test/passenger/request. Method as POST and data format as application/json.

2.Open the HiveMQTT browser client and click connect. Once the status changes to green. Add the topic subscription as given below.

testtopic/3seater
testtopic/4seater
testtopic/5seater

All the 3 topic subscriptions are ready to fetch the messages from the appropriate topic.

3.Now click Send in the postman. Now when you check the HiveMQTT browser client you should see the event in the appropriate topic.

HiveMQTT browser client

Conclusion

This article presented how easy Kumologica Designer flow orchestrates and integrates with a MQTT broker platform using MQTT node. Within few minutes we have created and deployed the passenger request Kumologica flow.

Remember Kumologica is totally free to download and use. Go ahead and give it a try, we would love to hear your feedback.

In the meantime, if there is any specific use case that you would like to see in a future article please let us know in the comments section.

Serverless integration of JIRA using Kumologica

ab73863 — Fri, 10 Jul 2020 14:13:34 +0000

In this article we will create “Contact Us” HTML form that is present on most of websites and integrate it with Jira Service Desk using Kumologica Designer flow running on AWS Lambda.

If you want to learn more about Kumologica, I would highly recommend you to check out the articles or subscribe on the youtube channel

Prerequisites

Jira Service Desk

If you don’t have access to Jira software or Jira Service Desk you can create free account. Once account has been created you will be given your site url (ends with atlassian.net) for your Jira Service Desk Instance.

Next Step is to create API Key. Navigate to: Settings icon (in the top right corner) > Atlassian account settings > Security (in the left panel) > Create and manage API tokens >Create API Token. Enter label for token and press Create, you will be presented API token. Copy it as you will need this token to authenticate from Kumologica flow.

We will be using General service desk (GSD) example project that has been created for us. The GSD is a project id that we will be using in Kumologica flow.

Building orchestration flow

Once we have access to Jira configured we can start building our Kumologica flow.

Create New Project

Open Kumologica Designer, click the Home button and choose Create New Kumologica Project. Enter name (for example jira-demo), select directory for project and switch Source into From Existing Flow …

Copy and Paste the following flow:

[{"id":"main.flow","type":"tab","label":"main","disabled":false,"info":""},{"id":"test.flow","type":"tab","label":"test","disabled":false,"info":""},{"id":"eacbb9ef.69d5c8","type":"EventListener","z":"main.flow","name":"POST /issue","eventSource":"api","dynamodbOperation":"","apiMethod":"post","apiUrl":"/issue","albMethod":"any","albUrl":"","bucketName":"","event":"s3:ObjectCreated:*","x":122.5,"y":140,"wires":[["dcdb6a8a.2ba208"]],"caname":"aws-event-handler","category":"aws"},{"id":"b5fc6ccd.17dd7","type":"EventListener-End","z":"main.flow","name":"Success","statusCode":"200","headers":{"Content-Type":"application/json"},"payload":"{}","x":502.5,"y":140,"wires":[],"caname":"eventlistenerend","category":"aws"},{"id":"3aa032bd.826cae","type":"TestCase","z":"test.flow","name":"Test POST /issue","selectedTargetNode":"eacbb9ef.69d5c8","payload":"{\n  \"path\": \"/issue\",\n  \"headers\": {\n    \"Accept\": \"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\",\n    \"Accept-Encoding\": \"gzip, deflate, lzma, sdch, br\",\n    \"Accept-Language\": \"en-US,en;q=0.8\",\n    \"CloudFront-Forwarded-Proto\": \"https\",\n    \"CloudFront-Is-Desktop-Viewer\": \"true\",\n    \"CloudFront-Is-Mobile-Viewer\": \"false\",\n    \"CloudFront-Is-SmartTV-Viewer\": \"false\",\n    \"CloudFront-Is-Tablet-Viewer\": \"false\",\n    \"CloudFront-Viewer-Country\": \"US\",\n    \"Host\": \"wt6mne2s9k.execute-api.us-west-2.amazonaws.com\",\n    \"Upgrade-Insecure-Requests\": \"1\",\n    \"User-Agent\": \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.82 Safari/537.36 OPR/39.0.2256.48\",\n    \"Via\": \"1.1 fb7cca60f0ecd82ce07790c9c5eef16c.cloudfront.net (CloudFront)\",\n    \"X-Amz-Cf-Id\": \"nBsWBOrSHMgnaROZJK1wGCZ9PcRcSpq_oSXZNQwQ10OTZL4cimZo3g==\",\n    \"X-Forwarded-For\": \"192.168.100.1, 192.168.1.1\",\n    \"X-Forwarded-Port\": \"443\",\n    \"X-Forwarded-Proto\": \"https\"\n  },\n  \"pathParameters\": {\n    \"proxy\": \"issue\"\n  },\n  \"requestContext\": {\n    \"accountId\": \"123456789012\",\n    \"resourceId\": \"us4z18\",\n    \"stage\": \"test\",\n    \"requestId\": \"41b45ea3-70b5-11e6-b7bd-69b5aaebc7d9\",\n    \"identity\": {\n      \"cognitoIdentityPoolId\": \"\",\n      \"accountId\": \"\",\n      \"cognitoIdentityId\": \"\",\n      \"caller\": \"\",\n      \"apiKey\": \"\",\n      \"sourceIp\": \"192.168.100.1\",\n      \"cognitoAuthenticationType\": \"\",\n      \"cognitoAuthenticationProvider\": \"\",\n      \"userArn\": \"\",\n      \"userAgent\": \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.82 Safari/537.36 OPR/39.0.2256.48\",\n      \"user\": \"\"\n    },\n    \"resourcePath\": \"/{proxy+}\",\n    \"httpMethod\": \"POST\",\n    \"apiId\": \"wt6mne2s9k\"\n  },\n  \"resource\": \"/{proxy+}\",\n  \"httpMethod\": \"POST\",\n  \"queryStringParameters\": {\n    \"name\": \"me\"\n  },\n  \"stageVariables\": {\n    \"stageVarName\": \"stageVarValue\"\n  },\n  \"body\": {\n  \t\"name\": \"issue\",\n    \"message\": \"issue description\",\n    \"email\": \"aa@aa.com\",\n    \"issueType\": \"Submit a request or incident\"\n  }\n}","initialLoad":"false","x":262.5,"y":160,"wires":[["e0721cc7.653eb"]],"caname":"test-case","category":"testing"},{"id":"e0721cc7.653eb","type":"TestCaseEnd","z":"test.flow","name":"TestCaseEnd","x":425,"y":160,"wires":[],"caname":"test-case-end","category":"testing"},{"id":"dcdb6a8a.2ba208","type":"JIRA","z":"main.flow","name":"Create Issue","operation":"Issuecreate","issueNumber":"DP-5","host":"YOURACCOUNT.atlassian.net","password":"API TOKEN","username":"YOUR USERNAME","priority":"","description":"\"From: \" & msg.payload.email & \", Description: \" & msg.payload.message","issuetype":"msg.payload.issueType","project":"GSD","summary":"msg.payload.name","comment":"","commentId":"","transitionName":"Close Issue","x":302.5,"y":140,"wires":[["b5fc6ccd.17dd7"]],"caname":"jira","category":"platforms"}]

press Create Button.

Install Jira Plugin.

Jira node is a contribution node and must be explicitly installed in Kumologica Designer. Press Add more nodes button located in the bottom left corner of designer. You will be presented screen with all contribution nodes that are currently available. Choose JIRA node, press Install button. After few seconds, plugin will be installed and you will be prompted to restart Kumologica Designer. Jira node will be located under new category: Platforms after restart.

Edit flow

After installing JIRA node, the imported flow will look like below:

Click on the Create Issue Jira node to display its properties and change Jira Connection Details section:

Host URL: enter the url that has been created for your Jira service desk instance (ends with atlasian.net)
Username: the user name you setup when creating account or the username you use for your jira instance
API Token: token created in Prerequisites section

Press Save button to return to the flow.

We have put connection details directly into properties for simplicity. We recommend to use AWS Parameter store to store secrets and use Kumologica AWS SSM Node to retrieve them.

Deploy flow to AWS Lambda

Select CLOUD tab on the right panel of Kumologica designer, choose AWS Profile and press Deploy button. This will trigger build and deployment of flow into AWS Lambda. After short while you should see the API gateway url that has been created for your flow in the terminal:

Contact Us HTML form

Below is a sample “Contact Us” html form, save it in local drive. Change line 18 to use the API Url that was displayed in Kumologica Designer terminal after successful deployment to AWS Lambda. Don’t forget to append it with /issue to call right service.

<!-- Original bootsnipp https://bootsnipp.com/snippets/83Br -->
<html>
    <head>
        <link href="http://netdna.bootstrapcdn.com/bootstrap/3.1.0/css/bootstrap.min.css" rel="stylesheet" id="bootstrap-css">
        <script src="http://netdna.bootstrapcdn.com/bootstrap/3.1.0/js/bootstrap.min.js"></script>
        <script src="http://code.jquery.com/jquery-1.11.1.min.js"></script>
        <style>
            body {padding-top:20px;}
        </style>
    </head>
    <body>
        <div class="container">
            <div class="row">
            <div class="col-md-6 col-md-offset-3">
                <div class="well well-sm">
                 <!-- Replace value of action with the url of API that has been displayed in Kumologica Designer console -->
                <form class="form-horizontal" 
                      action="https://YOURAPIID.execute-api.YOURREGION.amazonaws.com/test/issue" 
                      method="post">
                <fieldset>
                    <legend class="text-center">Contact Us</legend>

                    <div class="form-group">
                        <label class="col-md-3 control-label" for="name">Summary</label>
                        <div class="col-md-9">
                            <input id="name" name="name" type="text" placeholder="Your name" class="form-control">
                        </div>
                    </div>

                    <div class="form-group">
                        <label class="col-md-3 control-label" for="email">Your E-mail</label>
                        <div class="col-md-9">
                            <input id="email" name="email" type="text" placeholder="Your email" class="form-control">
                        </div>
                    </div>

                    <div class="form-group">
                        <label class="col-md-3 control-label" for="issueType">Issue Type</label>
                        <div class="col-md-9">
                        <select class="form-control" id="issueType" name="issueType">
                            <option>Task</option>
                            <option>Ask a question</option>
                            <option>Submit a request or incident</option>
                        </select>
                        </div>
                      </div>

                    <div class="form-group">
                        <label class="col-md-3 control-label" for="message">Your message</label>
                        <div class="col-md-9">
                            <textarea class="form-control" id="message" name="message" placeholder="Please enter your message here..." rows="5"></textarea>
                        </div>
                    </div>

                    <div class="form-group">
                        <div class="col-md-12 text-right">
                            <button type="submit" class="btn btn-primary btn-lg">Submit</button>
                        </div>
                    </div>
                </fieldset>
                </form>
                </div>
            </div>
            </div>
        </div>
    </body>
</html>

Open html file in your browser, fill up details and press Submit button.

The browser should display empty json: {}. When you switch to your Jira Service Desk General service desk project you should see new issue created with following details:

As you can see, the properties set in Kumologica Jira node mapped Summary into Title, Your E-Mail and Your Message got concatenated into Description.

Conclusion

This article presented how easy Kumologica Designer flow orchestrates and integrates with Jira software using Jira node. Within few minutes we have setup Jira Service Desk, created and deployed Kumologica flow and finally created simple contact us html form that calls flow and creates issue in Jira.

Remember Kumologica is totally free to download and use.

In the meantime, if there is any specific use case that you would like to see in a future article please let us know in the comments section.

Resources

Code used in this article is available in Github repository:
Repo link

Feel free to check it out and change for your needs.

Splunk logging in lambda using low code approach

ab73863 — Tue, 30 Jun 2020 12:25:14 +0000

A centralised logging management solution like: Splunk, Datadog, Sumologic, etc. enables organisations to collect, analyse, and display logs through a single pane of glass.

In this article we will see how application logs can be sent to Splunk from lambda using Kumologica Splunk node.

Kumologica is a free low-code development tool to build serverless integrations. You can learn more in this medium article

Use case

In this use case we have a mobile app application that consumes a Mobile Channel API. Behind the mobile API we have an Order FulFilment API. The Mobile API invokes the Order FulFilment API for completing the process. When the mobile app sends the very initial request it has a Tracking id. The Tracking id traverses from the Mobile Channel API to Order FulFilment API and back.

The message passed across both applications needs to be traced based on the Tracking id in Splunk.

Prerequisite

Kumologica designer installed in your machine. https://kumologica.com/download.html
Splunk cloud Trial account

Implementation

For the above mentioned use case we will be creating two API’s , a Mobile Channel API and an Order FulFilment API.

Order fulfilment API

1 . Open the Kumologica designer and Create a New project. And provide the name of the project as OrderFulFilmentService

2 . Remove the default hello world flow.

3 . Drag and drop EventListener Node from the pallet to the canvas and open the EvenListener node settings and configure the following.

    Display Name : POST /order
    Event Source : Amazon API gateway
    Verb : POST
    URL : /order

4 . Add Splunk Node from Logging category.

By default Splunk Node will not be available as part of the default Logging category.

In order to install the Splunk node , Go to Add More Nodes Option on the bottom of the pallet and click Install Splunk. Once Installation is completed a restart of the Designer will be required. Open the Splunk Node settings and configure the following.

    Display Name : Log_Entry
    Host : hostname of your splunk trial account
    Port : 8088
    Message : {“TrackingID” : msg.payload.TxnID,”ServiceName”:”Order_FullFillment”, “Data”: msg.payload }
    Splunk HEC Token : Splunk HTTP Event collector Token

5 . Wire the EventListener Node to the Splunk Node.

6 . Drag and drop Datamapper node from the palette to the canvas.

We are using the Datamapper node to mimic the response after an Order processing.

Open the Datamapper settings and configure the following.

    Display Name : Order_ProcessResponse

    Mapping : {
                “TrackingID” : msg.payload.TxnID,
                “OrderID” : msg.payload.OrderID,
               “Status” : “Processed”
              }

If you are not familiar using the DataMapper node then would recommend to go through the following article.

7 . Wire the Splunk Node to the DataMapper Node.

8 . Add the EventListener End node from the palette to the canvas. Open the settings and configure the following.

    Display Name : Success
    Payload : msg.payload

With this we have completed the development of our Order FulFilment API.

Now let’s deploy the API as a lambda to your AWS account. If you are not familiar with the deployment of a Kumologica flow I would recommend to have a walkthrough on the following medium article.

Note : Copy the service url that you receive on the terminal after a successful deployment.

Mobile Channel API

1 . Open the Kumologica designer and Create a New project. And provide the name of the project as MobileChannelAPIService

2 . Remove the default hello world flow.

3 . Drag and drop EventListener Node from the pallet to the canvas and open the EvenListener node settings and configure the following.

    Display Name : POST /mobile
    Event Source : Amazon API gateway
    Verb : POST
    URL : /mobile

4 . Add Splunk Node from Logging category.

By default Splunk Node will not be available as part of the default Logging category.

In order to install the Splunk node , Go to Add More Nodes Option on the bottom of the Pallet and click Install Splunk. Once Installation is completed a restart of the Designer will be required. Open the Splunk Node settings and configure the following.

    Display Name : Log_Entry
    Host : hostname of your splunk trial account
    Port : 8088
    Message : {“TrackingID” : msg.payload.TxnID,”ServiceName”:”Mobile_Channel”, “Data”: msg.payload }
    Splunk HEC Token : Splunk HTTP Event collector Token

5 . Wire the EventListener node to the Splunk Node.

6 . Add the HTTP Req node from the web category in the palette to the canvas. Open the settings and configure the following.

Display Name : Invoke_OrderFulFilmentService
Method : POST
Url : your order service url. Configure this once Order FulFilment Service is deployed and you receive the url .
Return : a UTF-8 string
Response Timeout : 120000ms (default)
Authentication : None

7 . Wire the Splunk Node to the HTTP Req node.

8 . Add the EventListener End node from the palette to the canvas. Open the settings and configure the following.

    Display Name : Success
    Payload : msg.payload

With this we have completed the development of our Mobile Channel API.

Now let’s deploy both the services separately as two different lambda in your AWS account.

Testing

For testing purposes I am sending the following request as input to the Mobile API.

    {
    “OrderID”: “APQ-00001”,
    “TxnID” : “0948939-AWHDH”,
    “Item” : {
    “Name” : “ABC Laptop”,
    “Type” : “Mini-5”,
    “Qty” : 1
    },
    “Address” :{
    “Unit” : “9”,
    “Street” : “TTK Street”,
    “Road” : “Tower Road”,
    “State” : “NSW”,
    “Country” : “Australia”,
    “PIN” : 94839
    }
    }

After you invoke the mobile channel API and when you search in Splunk with the TxnID value as the search parameter

you would get the following log entries.

If you want to try out this application quickly you can import the loan broker flow from this project’s repo.

Summary

Thanks for reading and I hope you enjoyed this use case and appreciate how simple it is to implement an orchestration service with Kumologica. We would love to hear your feedback, and don’t hesitate to contact us if you have any questions.

To learn more about Kumologica checkout our official documentation at kumologica.com and subscribe to our youtube channel

Serverless Orchestration in lambda

ab73863 — Thu, 25 Jun 2020 04:33:03 +0000

Service orchestration is an architectural design where a single centralised service is responsible for coordinating and interacting among different services. They invoke different services and combine them together . The orchestration unit is abstracted and exposed as a single service.

Orchestration is usually accomplished through Enterprise Application Integration(EAI), which enables data integration, and the use of a central messaging engine such as an Enterprise Service Bus(ESB), which routes, transforms and enriches messages.

In this article we will see how orchestration can be achieved in a serverless world using Kumologica and AWS Lambda.

Kumologica is a free low-code development tool to build serverless integrations. You can learn more in this medium article or on our website.

Use case

To get a better sense of orchestration , we will create a loan broker service which will be making a loan request on behalf of a customer. The service will invoke the sub services which are involved in the loan request. These sub services include credit score from credit agencies, quotes from different banks etc. Each of these sub services are external services which need to be invoked in a sequence by the orchestration service and finally returns a list of quotes from potential lenders to the broker who made the original request.

Prerequisite

For the above use case we will be creating mock API’s in mockable.io for credit score check. Below is the sample mock response data for a credit score check.

    {
    "check_status" : "pass",
    "score" : 600,
    "SSN" : "ABHY888-999-HT",
    "Agency" : "ABC Provider",
    "Date" : "08-09-2019 10:00:00"
    }

2 . Below is the Bank quote Mock response for the Bank Quote mock API.

    [{
    "Bank_Name" : "ABC Corp",
    "Quote" :{
    "Min" :  "600000",
    "Max" :  "800000",
    "Currency" : "AUD"
    }
    },
    {
    "Bank_Name" : "HSKC",
    "Quote" :{
    "Min" :  "900000",
    "Max" :  "1000000",
    "Currency" : "AUD"
    }
    },
    {
    "Bank_Name" : "Westland",
    "Quote" :{
    "Min" :  "500000",
    "Max" :  "800000",
    "Currency" : "AUD"
    }
    },
    {
    "Bank_Name" : "CAN Bank",
    "Quote" :{
    "Min" :  "600000",
    "Max" :  "900000",
    "Currency" : "AUD"
    }
    }
    ]

Note : You may use any other services or mechanisms in order to create a mock service.

3 . Kumologica designer installed in your machine. https://kumologica.com/download.html

If you are new to Kumologica then I would suggest reading the following link to kick start. https://medium.com/@kumologica

4 . An AWS account to deploy your Kumologica service

Implementation

Open the Kumologica designer and Create a New project.
Remove the default hello world flow.
Drag and drop EventListener Node from the pallet to the canvas and open the EvenListener node settings and configure the following.

Display Name : GET /quote
Event Source : Amazon API gateway
Verb : GET
URL : /quote
Drag and drop HTTP Req node from the web category in the pallet to the canvas. Open the settings and configure the following.

Display Name : GetCreditCheck
Method : GET
Url : your mock url for credit check.
Return : a UTF-8 string
Response Timeout : 120000ms (default)
Authentication : None
Wire EventListener node to HTTP Req node.
Add JSON node to canvas for transforming JSON string to Javascript Object.
Wire HTTP Req node to JSON node.
Add Set-property Node to the canvas. This is to store the check_status field value from the credit check (http) response.
Drag and drop the switch node to the canvas and set the condition for verifying whether to proceed for Quote or Not.

Display Name: CreditStatus_Check
Set temp == ‘pass’
Set “otherwise”
Write the JSON node to Set-Property Node and Set-Property node to Switch Node.
Add another HTTP req node to the canvas and set the following.

Display Name : GetBankQuote
Method : POST
Url : your mock url for credit check
Return : a UTF-8 string
Response Timeout : 120000ms (default)
Authentication : None
Drag and Drop another Set-Property node to the canvas and set the msg.payload as {“message” : “Cannot process the quote”}
Wire the “pass” output terminal of Switch node to the HTTP req node (GetBankQuote) and wire the “otherwise” terminal to the second Set-Property node.
Drag and drop the EventListener End node to the canvas and set the Payload as msg.payload and leave other fields as default.
The loan broker service flow will look given below in the canvas.

If you want to try out this application quickly you can import the loan broker flow from the below given git url.

https://github.com/KumologicaHQ/kumologica-demos/tree/master/LoanBorkerApplication

Loan Broker service flow in Kumologica Designer Canvas

Now we are done with the implementation. Let’s deploy the flow to your AWS account.

On the cloud tab click on the Profile drop down menu and select your AWS profile. Ensure that you have the necessary permissions for your flow to be deployed as AWS lambda.
Select the API Gateway under the trigger section and leave all other fields as default.
Click Deploy.
Now you can see the flow is getting deployed to your AWS lambda. And once deployment is completed then you should the endpoint url for your Loan Broker service hosted on AWS API gateway.

Summary

To learn more about Kumologica checkout our official documentation at kumologica.com and subscribe to our video in youtube at https://www.youtube.com/channel/UCsTDAlhijUFpsfThJu3hF9g.