WxStreamSets - Integrate with StreamSets using webMethods

Typically, unlike application integration workflows, data integration pipelines have been configured to run on defined schedules (daily, weekly, monthly, etc.), and perform batch processing.

However, in today’s dynamic and fast-evolving operating environment, data integration pipelines are expected to run in real-time. That is why techniques like CDC (Change Data Capture), webhooks, APIs, etc. are supported by data integration platforms like StreamSets.

WxStreamSets is a webMethods integration package that leverages StreamSets REST API and enables you to integrate with StreamSets pipeline jobs

WxStreamSets is a webMethods integration package that leverages StreamSets REST APIs and enables you to trigger data integration pipelines in real-time.

With WxStreamSets, you can

• Start a StreamSets job
• Check the status of a StreamSets job
• Stop a StreamSets job

Useful links

For more information related to WxStreamSets and to download the package, visit:

SoftwareAG / WxStreamSets

WxStreamSets is webMethods Integration Server package to simplify integration with StreamSets.

WxStreamSets

WxStreamSets is a webMethods Integration Server package designed to simplify the integration with StreamSets.

Pre-requisites

• webMethods Integration Server
• StreamSets

Features

1. Ready to use flow-services to call StreamSets APIs.
2. Start StreamSets Job synchronously.

Installation

1. Download the latest release of WxStreamSets.zip from Releases section.
2. Copy WxStreamSets.zip to /${SAG_HOME}/IntegrationServer/instances/${INSTANCE}/replicate/inbound directory.
3. Login to Integration Server Admin Console http://${HOST_NAME}:${PORT}.
4. Go to Packages > Management in the left side menu and click Install Inbound Releases.
5. Choose WxStreamSets.zip from dropdown and click Install.

Generate StreamSets API Credentials

1. Login to StreamSets Control Hub.
2. Expand Manage option in the left-hand side naviagtion and choose API Credentials.
3. Click on Add button.
4. Provide the Name for credentials and click Save & Generate.
5. Copy the Credential ID and Token.

Usage

Following APIs in terms of flow-services are supported:

1. wx.streamsets.v1.jobRunner.services.startJobById - To start a specific job.
2. wx.streamsets.v1.jobRunner.services.getJobCurrentStatusById - To get the status of a specific job.
3. wx.streamsets.v1.jobRunner.services.stopJobById - To…

View on GitHub

Read full topic