DEV Community: KuFlow

KuFlow as a Temporal.io-based Workflow Orchestrator

KuFlow — Fri, 16 Dec 2022 09:27:01 +0000

KuFlow, among other things, orchestrates Temporal.io workflows so you can manage your workflows more easily. With KuFlow, you can create and run workflows in seconds and manage them all from one place.

What is KuFlow?

According to one of our developers, "From Temporal.io's point of view, KuFlow can be seen as a layer that builds on Temporal and provides authorization, authentication, data governance, and a collaborative user interface, among other features."

If you ask me, KuFlow is an orchestra conductor that directs the rhythm of business processes in an orderly manner and on time, making them "sound in harmony" so that they flow efficiently with the clear intention of adding value to the organization, and Temporal.io is part of the band in conjunction with humans and machines.

How KuFlow powers the use of Temporal.io

By interacting with the KuFlow user interface, you can trigger information inputs and outputs in the form of tasks for the desired business process. with an authorization system that guarantees that only the right people or systems have access to the information and functions necessary to perform interactions with the tasks.

Behind the scenes, where developers can do their magic, APIs can be used to connect to the different platforms in the environment, and if they don't have APIs, KuFlow can make use of RPA tools, such as robotframework or uiPath, and complete the processes.

The advantages of using KuFlow as a layer of empowerment of Temporal.io

KuFlow provides the role of supervisor to humans over machines, which execute all tasks: repetitive, long, short, boring, prone to failure, and many more. And it empowers him to decide if the result is as expected or not.

Therefore, the human is not replaced by the machine, but is freed from the time and effort that would be required to perform the tasks, giving them the possibility of using that time on issues of greater importance to the business, such as generating new income or searching for ways to save or improve management.

But KuFlow not only works with Temporal.io

With KuFlow it is also possible to work with serverless workflows apart from Temporal.io, we explain it in this blog entry, but in summary, almost as a no-code tool, the correct use It would be a rather low-code tool; in just a matter of minutes with our drag-and-drop tool, you can have a workflow that interacts with one or more users of the organization.

In conclusion

Temporal.io is a workflow platform based on microservices that offers a complete solution and KuFlow complements and enhances it. Adding security, resiliency, and collaboration, With KuFlow, developers can create robust and highly scalable workflows for their applications, allowing them to fully harness the power of temporal.io.

In terms of resiliency, KuFlow enables workflow users to detect and respond to events in real time, ensuring a higher level of service continuity.

Finally, we recommend you try KuFlow by creating a free account and finding out for yourself.

For more information, visit our technical documentation and our video tutorials.

No-code or Workflow as code? Better both

KuFlow — Tue, 29 Nov 2022 15:53:21 +0000

In KuFlow we know that some users don't need a fine-grain control of the workflows that they develop. Sometimes, the business process is so simple that you just want to establish a bunch of tasks, establish the relationship between them, and run the process.

For that kind of scenario, maybe you don’t need to implement the workflow as code and deploy it in a conventional way, so we have developed the KuFlow Diagram Tool that is based on these three pieces:

Diagram Maker, a new drag-and-drop tool designed to allow our users to implement simple workflows
Serverless workflow. Instead of reinventing the wheel, we decided to base this feature on an established and mature specification. Serverless workflow is a powerful open-source standard
Temporal as the main runtime engine that acts as the foundation for the workflow execution

How does it work?

First of all, you must define your workflow using our Diagram Maker Tool.

Thanks to the great work of AWS, we could build a minimalist UI, that fits our necessities.

But please remember! This is a first version of this tool. We are planning to add features to it, according to the feedback we receive from the community.

Once you have defined the workflow on a diagram, a JSON, following the Serverless workflow specification, is stored in KuFlow. With this specification, we can use all the great tools/libs that are developed.

Finally, our KuFlow Diagram Runner executes the JSON created after the diagram. This Runner is based on the great work of Tihomir Surdilovic (aka tsurdilo) that you can find at his repo (kudos, Tihomir!). Thanks to his awesome work, we have developed a very simple Serverless Workflow Runtime, that doesn't have all the features that the specification allows, but we think that it covers the core set of them as a first version.

The runtime is developed using Temporal, which is one of the main tools that we are currently using at KuFlow. Thanks to, all the workflow executions are robust: your application will be durable, reliable, and scalable.

How can I do it?

We are preparing a video tutorial with all the required steps (stay tuned on Twitter or subscribe to our YouTube channel!).

Meanwhile, you can try to create your own Serverless Workflow in KuFlow on your own, creating a new process definition and selecting “KuFlow Diagram Tool” as its Workflow Engine. Please remember that you can use our forum to ask about any issue or doubt you may find.

What's next?

Well, honestly, we are not very sure at this moment. We have thought about different paths for continuing the development: adding more features to the Diagram Tool, covering more details of the Serverless Workflow specification, building a DSL tool from scratch,... We are looking forward to getting some feedback from you, in order to establish the best priorities thinking in our community.

Please share your thoughts with us on Twitter!

No-code or Workflow as code? Better both

KuFlow — Tue, 29 Nov 2022 15:49:19 +0000

Diagram Maker, a new drag-and-drop tool designed to allow our users to implement simple workflows
Serverless workflow. Instead of reinventing the wheel, we decided to base this feature on an established and mature specification. Serverless workflow is a powerful open-source standard
Temporal as the main runtime engine that acts as the foundation for the workflow execution

How does it work?

First of all, you must define your workflow using our Diagram Maker Tool.

Thanks to the great work of AWS, we could build a minimalist UI, that fits our necessities.

But please remember! This is a first version of this tool. We are planning to add features to it, according to the feedback we receive from the community.

How can I do it?

We are preparing a video tutorial with all the required steps (stay tuned on Twitter or subscribe to our YouTube channel!).

What's next?

Please share your thoughts with us on Twitter!

How KuFlow supports Temporal as a workflows engine for our processes?

KuFlow — Tue, 15 Nov 2022 09:06:11 +0000

In such a diverse world, it would be boring to have a single way of doing things. That's why at KuFlow we support different ways to implement the logic of our processes and tasks. And in this post, we will talk about one of them, the orchestration through Temporal, which gives us a powerful way to manage our workflows.

We will not go into detail on the concepts that are handled in our platform, for that you can consult the documentation or other blog entries, but as a general idea, we can stay with the following. KuFlow is a platform that allows you to model business processes that involve a series of tasks to carry out their achievement. These tasks can be performed by humans or by systems and are basically activities to perform actions and/or collect information. We also need to establish a way to control the performance of these tasks, their order, their asynchronous or non-asynchronous nature, or their ability to recover from errors. In other words, we need a workflow to orchestrate our process. At KuFlow we call WorkFlow Engine for the different types of implementations we support. In this case, Temporal is the Workflow Engine.

Why use Temporal for your KuFlow processes?

The use of Temporal gives us several benefits, the main one being the ability to implement the resilience of our operations in a comfortable and reliable way. For example, let's imagine that a task of one of our processes makes a call to a third-party API. If this API is not available at the time of its call, the task fails and our process could be blocked. In this case, we would like the workflow of our process to be retried later and have to forget about all the scaffolding required to do so. Also, if we have already issued modification requests to other external APIs, we would like that when recovering the process, the consumption of these APIs would be deterministic and would not fail because we have consumed them twice. We achieve all this by implementing our workflows with Temporal.

How does Temporal integrate with KuFlow?

In order to simplify as much as possible the infrastructure of our customers, in KuFlow we have our own Temporal cloud avoiding the need to deploy this infrastructure by customers. Having a Temporal cloud is not our core business and in the future, we may also offer the possibility to use other clouds such as Temporal Cloud or any other.

To implement a Workflow in a process using Temporal, we need to develop what in KuFlow we call Worker. A worker is nothing more than an application that runs on our customers' servers and interacts with the KuFlow API and the Temporal API available in our cloud. This approach allows for a hybrid cloud architecture that provides flexibility when accessing the different services that you want to consume in the Worker. Thus, for example, a Worker deployed on the customer's servers would have access to the customer's private services in a secure and reliable way in order to extract data and make decisions based on their needs.

To facilitate the development of this Worker, a library that facilitates the consumption of the KuFlow API through Temporal Activities is provided. Together with the different SDKs provided by Temporal, the construction of a Worker is, in most cases, child's play. The various languages for which these libraries are provided are updated frequently.

Security

One of the most important aspects for us is everything related to security. And for this reason, apart from a powerful RBAC system that is deployed on the customer's resources, we use a series of authentication mechanisms to access the different APIs that we offer. To facilitate the management of access and credentials to these APIs, KuFlow defines the concept of 'Application'. By definition, an 'Application' is composed of credentials and access certificates. In turn, these applications are used as candidates or "principals" to grant roles and permissions to the resources you want.

How is authentication and authorization achieved with the Temporal API?

Schematically, to authenticate we will use Mutual TLS and to get authorization we will use a JWT bearer token. Obtaining the certificates and credentials is done from the KuFlow App in a simple way, without the need to create and upload the certificates manually. Armed with this data we can now connect our workers to the KuFlow cloud. The authentication mechanism follows the following flow: On the one hand, it is necessary to obtain a JWT bearer token that will allow us authorization in Temporal, for this a request is issued to the KuFlow API Rest (authenticated by BasicAuth) that returns the token. With this token, the communications to Temporal's API GRPC are made, besides using the certificates to encrypt the communication and to achieve the authentication with this API.

For more information, please refer to the documentation.

To deal with problems such as carrier token renewal and to make the developer's experience as comfortable as possible, this mechanism is implemented in the libraries we provide as support for Workers. An example of token renegotiation can be seen in this code extracted from the kuflow-engine-client library in Java.

public class KuFlowAuthorizationTokenSupplier implements AuthorizationTokenSupplier {

    private static final Logger LOGGER = LoggerFactory.getLogger(KuFlowAuthorizationTokenSupplier.class);

    private static final double EXPIRE_PERCENTAGE = 0.1;

    private static final Duration EXPIRE_MAX_DURATION = Duration.ofMinutes(10);

    private final AuthenticationApi authenticationApi;

    private volatile String token;

    private volatile Instant tokenExpireAt;

    public KuFlowAuthorizationTokenSupplier(AuthenticationApi authenticationApi) {
        this.authenticationApi = authenticationApi;
    }

    @Override
    public String supply() {
        String token = this.requestToken();

        return "Bearer " + token;
    }

    private String requestToken() {
        String token = this.token;
        Instant tokenExpireAt = this.tokenExpireAt;
        if (isTokenNonExpired(token, tokenExpireAt)) {
            return token;
        }

        synchronized (this) {
            token = this.token;
            tokenExpireAt = this.tokenExpireAt;
            if (isTokenNonExpired(token, tokenExpireAt)) {
                return token;
            }

            AuthenticationResource authentication = new AuthenticationResource();
            authentication.setType(AuthenticationTypeResource.ENGINE);
            authentication = this.authenticationApi.createAuthentication(authentication);

            Duration expireDuration = Duration.between(Instant.now(), authentication.getExpiredAt());
            expireDuration = Duration.ofSeconds((long) (expireDuration.getSeconds() * EXPIRE_PERCENTAGE));
            if (expireDuration.compareTo(EXPIRE_MAX_DURATION) > 0) {
                expireDuration = EXPIRE_MAX_DURATION;
            }

            this.token = token = authentication.getToken();
            this.tokenExpireAt = tokenExpireAt = Instant.now().plus(expireDuration);

            LOGGER.debug("Regenerated JWT Temporal authorization token. Expired at: {}", tokenExpireAt);

            return token;
        }
    }

    private static boolean isTokenNonExpired(String token, Instant tokenExpireAt) {
        return token != null && tokenExpireAt != null && Instant.now().isBefore(tokenExpireAt);
    }
}

What we have done here is to implement the AuthorizationTokenSupplier interface provided by the Temporal SDK to use our KuFlow API client library to negotiate a token when needed. In the same way, we implemented the GrpcMetadataProvider interface with the purpose of adding as metadata the token provided by the previous implementation:

public class AuthorizationGrpcMetadataProvider implements GrpcMetadataProvider {
  public static final Metadata.Key<String> AUTHORIZATION_HEADER_KEY =
      Metadata.Key.of("authorization", Metadata.ASCII_STRING_MARSHALLER);

  private final AuthorizationTokenSupplier authorizationTokenSupplier;

  public AuthorizationGrpcMetadataProvider(AuthorizationTokenSupplier authorizationTokenSupplier) {
    this.authorizationTokenSupplier = authorizationTokenSupplier;
  }

  @Override
  public Metadata getMetadata() {
    Metadata metadata = new Metadata();
    metadata.put(AUTHORIZATION_HEADER_KEY, authorizationTokenSupplier.supply());
    return metadata;
  }
}

This metadata provider class for the GRPC connection to Temporal, will be the one we will use later when we configure our WorkflowServiceStub in the Worker.

WorkflowServiceStubsOptions.newBuilder().addGrpcMetadataProvider(
    new AuthorizationGrpcMetadataProvider(new KuFlowAuthorizationTokenSupplier(this.authenticationApi))
);

As the last step to successfully connect to Temporal we need to create an SSL context that will be used in the negotiation with MutualTLS. For this, we need to extend the SslContext class of the SDK. The implementation depends on your needs but an example of it can be found in the following code snippet available in our example repositories on GitHub.

 private SslContext createSslContext() {
        MutualTlsProperties mutualTls = this.applicationProperties.getTemporal().getMutualTls();
        if (StringUtils.isBlank(mutualTls.getCert()) && StringUtils.isBlank(mutualTls.getCertData())) {
            return null;
        }

        if (
            StringUtils.isNotBlank(mutualTls.getCert()) &&
            (StringUtils.isBlank(mutualTls.getKey()) || StringUtils.isBlank(mutualTls.getCa()))
        ) {
            throw new KuFlowEngineClientException("key and ca are required");
        }

        if (
            StringUtils.isNotBlank(mutualTls.getCertData()) &&
            (StringUtils.isBlank(mutualTls.getKeyData()) || StringUtils.isBlank(mutualTls.getCaData()))
        ) {
            throw new KuFlowEngineClientException("keyData or caData are required");
        }

        try (
            InputStream certInputStream = this.openInputStream(mutualTls.getCert(), mutualTls.getCertData());
            InputStream keyInputStream = this.openInputStream(mutualTls.getKey(), mutualTls.getKeyData());
            InputStream caInputStream = this.openInputStream(mutualTls.getCa(), mutualTls.getCaData())
        ) {
            KeyStore trustStore = KeyStore.getInstance(KeyStore.getDefaultType());
            trustStore.load(null, null);
            X509Certificate certificate = (X509Certificate) CertificateFactory.getInstance("X509").generateCertificate(caInputStream);
            trustStore.setCertificateEntry("temporal-ca", certificate);

            TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
            trustManagerFactory.init(trustStore);
            TrustManager trustManager = trustManagerFactory.getTrustManagers()[0];

            return SimpleSslContextBuilder.forPKCS8(certInputStream, keyInputStream).setTrustManager(trustManager).build();
        } catch (KeyStoreException | NoSuchAlgorithmException | CertificateException | IOException e) {
            throw new KuFlowEngineClientException("Unable to configure mTLS", e);
        }
    }

Finally, like the GRPC metadata provider, we add this SSL context to our WorkflowServiceStubs options.

public WorkflowServiceStubs workflowServiceStubs() {
    Builder builder = WorkflowServiceStubsOptions.newBuilder();
    builder.setTarget(this.applicationProperties.getTemporal().getTarget());
    builder.setSslContext(this.createSslContext());
    builder.addGrpcMetadataProvider(
        new AuthorizationGrpcMetadataProvider(new KuFlowAuthorizationTokenSupplier(this.authenticationApi))
    );

    WorkflowServiceStubsOptions options = builder.validateAndBuildWithDefaults();

    return WorkflowServiceStubs.newServiceStubs(options);
}

At this point it is useful to remember that when configuring a Temporal-backed process in the KuFlow APP, it is possible to download a ready-to-use Worker template that uses these connection configuration mechanisms, so you have a working example just a few clicks away. Similarly, much more information and integration examples can be found in our GitHub repository and in our documentation.

Notes on the Temporal cloud deployed in KuFlow

As discussed earlier in the article, the purpose of offering a Temporal deployment is to facilitate integration for our customers. At this point, it is important to highlight some issues: Our Temporal deployment is a slightly modified version of the one available in the official Temporal repositories. Basically, security and authorization modifications have been added. As a result, some API methods are not available. For example, you will not be able to use our Temporal deployment to execute Workflows that are not launched from KuFlow processes, which is necessary to avoid abuse. Or in the same way, it is also not possible to cancel Workflows if not from the KuFlow App. Other APIs may not be available, unlike a normal Temporal installation. However, all the APIs that guarantee the correct functioning of your workers are enabled. If you need any of the currently disabled APIs, you can request them by contacting us.

Some other modifications made allow you, for example, to know from the App if there is any worker currently online for your process.

In the same way, we try to keep our Temporal deployments as up-to-date as possible, if you have any questions about this, please contact us.

Conclusion

In this article, we have superficially explored the use of Temporal as a workflow engine for our KuFlow processes. This solution provides us with a number of advantages over the other workflow engines we support, such as access to private services, allowing greater flexibility when implementing Worker. In the same way, the resilience and scalability offered by Temporal makes it the most recommendable option when the processes are highly complex and need to orchestrate different external APIs.The following table shows a summary of the main features of each of the supported Workflows Engine:

Try KuFlow for free, creating your own free account and start automating your company.

Originally published in KuFlow's Blog - How KuFlow supports Temporal as a worfkows engine for our processes?