DEV Community: Riza Farheen

Building an Agentic Workflow: Orchestrating a Multi-Step Software Engineering Interview

Riza Farheen — Mon, 21 Apr 2025 10:46:26 +0000

As language models evolve, so does the way we build intelligent systems. One emerging pattern is the agentic workflow—a coordinated sequence of tasks in which an AI agent interacts with external tools and APIs, makes decisions, and executes complex flows autonomously.

But intelligence alone isn’t enough. Large language models excel at reasoning and response generation, yet they fall short when it comes to real-world execution. Managing state, coordinating asynchronous tasks, handling tool invocation, and ensuring reliability across multiple steps remain key challenges. That’s where orchestration comes in.

By combining LLMs with a robust workflow engine, we can design resilient, multi-step systems that bridge reasoning with action. In this guide, we’ll walk through an agentic workflow that automates a full software engineering technical interview—from candidate intake to interview questions, evaluation, and report delivery.

Example: Agentic interview app

Let’s build an application that mimics a human-led coding interview, where:

The candidate enters their details.
An LLM agent conducts the interview by generating a series of leetcode-style programming questions, which the candidate responds to in a sequence.
The agent scores answers and provides feedback.
A formatted transcript of the interview is generated.
A final report is emailed to the candidate.

This multi-step process involves reasoning (via OpenAI), external tools (Google Docs and SendGrid), and orchestration (Orkes Conductor).

To build it, we’ll use the following components:

Agentic Interview App flow

Frontend (Next.js)–Provides the user interface for candidate interaction. It captures user input (name, language, email) and streamlines the interview conversation.
Backend (Python)–Acts as the trigger point for the workflow. It starts the Conductor workflow and serves as middleware to process frontend user input to the backend.
OpenAI–Provides the reasoning layer. Generates questions, interprets responses, and guides the flow through hints and feedback.
Google Cloud (Docs API)–Generates and formats the interview transcript as a sharable document.
SendGrid–Send the final report and feedback to the candidate via email.
Orkes Conductor–Orchestrates the entire workflow: managing state, coordinating tasks, handling API integrations and logic.

Now, let’s walk through how these tools come together to form the agentic interview application.

Building an agentic interview app

You can follow this tutorial using the free Developer Playground. Create an account to get started.

We’ll build an interview app powered by the following Conductor workflow:

Interview Agentic Workflow in Orkes Conductor

After gathering the candidate details, the interview will begin. Conductor’s Start Workflow task then starts the Core Interview Loop workflow and generates a coding question using OpenAI. It then enters a loop to collect and evaluate the candidate’s responses (with support for hints or simplifications), finally scoring the answer and storing a structured evaluation as a scorecard.

Core Interview Loop Workflow in Orkes Conductor

Now, here’s how you can create an agentic interview application on your own.

Step 1: Clone the repository

The source code for this example project is available in this GitHub repository. Clone it to your local machine:

git clone https://github.com/conductor-oss/awesome-conductor-apps/

Open the project and navigate to the interview app folder:

cd python/interview_agentic_app

Step 2: Get credentials and set up environment variables

This application interacts with multiple tools and APIs. You’ll need to configure a few credentials before running it locally.

Access key and secret from Conductor

Get the access key and secret from Developer Playground by creating a new application from Access Control > Applications. Set the Application role to Worker and Metadata API.

Generate the access key and secret.

Generating access key and secret from Orkes Conductor

Open your project in the IDE of your choice and set the following environment variables:

export CONDUCTOR_SERVER_URL=https://developer.orkescloud.com/api
export CONDUCTOR_AUTH_KEY=<YOUR_CONDUCTOR_AUTH_KEY>
export CONDUCTOR_AUTH_SECRET=<YOUR_CONDUCTOR_AUTH_SECRET>

OpenAI API key

Generate your API key from the OpenAI platform, then set the environment variable:

export OPENAI_API_KEY=<YOUR_OPENAI_KEY>

Google authentication credentials

Google authentication credentials are required to access Google APIs. Ensure Google Docs API is enabled on your GCP project.

Create a service account in your Google Cloud console and download the JSON key file. It will be named something like project-name.json.

Creating service account key from GCP console

Next, we need to stringify the contents of the JSON file using the following command:

python3 -c 'import json; print(json.dumps(json.load(open("<PATH-TO-JSON-FILE>/project-name.json"))))'

Copy the output (the stringified JSON), then set the environment variables:

export GOOGLE_SERVICE_ACCOUNT_JSON='<PASTE_THE_STRING_HERE>'
export ENV=prod

SendGrid API key

SendGrid automatically sends the final interview summary and feedback to the candidate via email.

Start by creating an account on SendGrid, then create and verify the sender email address. Next, generate an API key from SendGrid.

Generating API key from SendGrid

Now, set the environment variable for your API Key:

export SEND_GRID_API_KEY=<YOUR_SEND_GRID_API_KEY>

Then, set the environment variable for your verified sender email:

export SEND_GRID_EMAIL_ADDRESS=<YOUR_SEND_GRID_VERIFIED_EMAIL_ADDRESS>

Finally, update your workflow JSON to use this sender email. Navigate to python/interview_agentic_app/resources/interviewAgenticWorkflow.json file.

Go to line 878 and replace the from.email value:

//workflow JSON

 "from": {
                  "email": "<SENDGRID_VERIFIED_SENDER_EMAIL>" //Line 878
                },
                "subject": "Thank you ${workflow.variables.name} for interviewing with Orkes",

Go to line 996 and replace the from.email value:

//workflow JSON 
                "from": {
                  "email": "<SENDGRID_VERIFIED_SENDER_EMAIL>" //Line 996
                },
                "subject": "Thank you ${workflow.variables.name} for interviewing with Orkes",

Step 3: Start the backend server

This step launches the Python-based backend that listens for incoming requests and triggers the Conductor workflow.

Set the Python path:

export PYTHONPATH=/[PATH_TO_REPO]/awesome-conductor-apps/python/interview_agentic_app

Create and activate a virtual environment:

python3 -m venv venv
source venv/bin/activate

Install required dependencies:

pip3 install -r requirements.txt

Start the backend server:

cd workflow
python app.py

Step 4: Start the frontend server

This starts the Next.js frontend, where candidates can enter their details and take the interview in an interactive chat UI.

Open a new terminal and navigate to your project:

cd python/interview_agentic_app

Next, navigate to the frontend directory:

cd interview-chat

Install dependencies:

npm install --legacy-peer-deps

Start the frontend server:

npm run dev

The app should now be running locally.

Step 5: View the interview app in action

To start the interview, open it in your browser at http://localhost:3000, this launches the interview experience locally.

Agentic Interview App UI

In your Developer Playground, you’ll notice that the required OpenAI integrations, AI prompts, and workflow definitions are automatically created.

Resources created in Developer Playground

As the candidate proceeds, the Interview Agentic Workflow is triggered. Each interview question is handled by a separate workflow, Core Interview Loop. So, for a three-question interview, three workflows run in sequence using Conductor’s Start Workflow task.

Triggered workflows in Developer Playground

Once the interview is complete, a thank you email with personalized feedback and scores is instantly sent to the candidate via SendGrid API.

Thank you email triggered via SendGrid on interview completion

In parallel, a fully formatted Google Docs transcript is generated and stored in Google Drive. The interviewer and candidate are granted access, and a link is shared in another email.

Interview transcript generated and stored in Google Docs

That’s it—you’ve now built a fully automated coding interview for a Software Engineer, from candidate intake to evaluation and reporting.

With the core workflow in place, you can easily tailor it to your own needs: customize the prompts, add more questions, or plug it into tools of your choice.

This agentic interview app isn’t just for demos—it’s built for production. You can deploy the backend on platforms like Render and the frontend on Vercel, with your Conductor workflows running in the Orkes Cloud.

What next?

Now that you’ve built and tested your agentic interview workflow, here are a few ways you can take it further:

Add a progress tracker–Show candidates how they’re doing after each question to make the experience more engaging.
Plug into ATS or HRIS–Send the final report straight to your ATS or HR dashboard for easier review.
Extend beyond coding–Replace coding questions with behavioral prompts or design discussions to adapt the workflow for non-technical interviews.

Wrapping up

This tutorial walked you through building a complete agentic workflow—one that goes beyond single-shot prompts to orchestrate a real-world, multi-step interview process. Using OpenAI for reasoning, Google Docs for formatting, SendGrid for communication, a modern frontend and backend stack, and Orkes Conductor at the core, you now have a robust foundation for deploying agentic workflows in production.

Whether you’re building technical assessments, onboarding flows, or multi-agent decision systems, Conductor gives you the structure to scale—with full control over inputs, logic, and integrations.

Explore more AI-powered workflows built with Conductor

Try building and running your first agentic workflow using the free Developer Playground. Drop by our Slack community and show us what you’ve built.

SendGrid Webhooks for Email Nurturing Workflows: A Step-by-Step Guide

Riza Farheen — Mon, 17 Mar 2025 09:17:41 +0000

Did you know the average person receives around 121 emails per day? With inboxes constantly flooded, how can your email stand out and earn that crucial click? The key lies in personalization and timing–sending the right email at the right moment based on user engagement.

Email nurturing is a powerful B2B strategy to build relationships with potential clients by guiding them through time-based campaigns. Whether it’s encouraging them to take an offer, sign up for a demo, or schedule a consultation, well-timed follow-ups can significantly improve lead conversion.

SendGrid’s webhooks help automate this process by tracking real-time email events such as opens, clicks, bounces, and more. When integrated with a platform like Orkes Conductor, the Unified Application Platform, these events can trigger automated and personalized responses.

This guide demonstrates how to leverage SendGrid webhooks to create an automated email nurturing workflow.

Building an email nurturing flow

To illustrate how email nurturing works, let’s use the example of a product launch email campaign.

When the campaign is sent, we consider two key scenarios: subscribers who open the email and those who click a link. To further nurture interest, follow-up emails are customized based on subscriber engagement.

Here’s the workflow to achieve this:

emailNurturing workflow in Orkes Conductor

The Wait for Webhook task listens for incoming event payloads from SendGrid webhooks, while the Switch task determines the follow-up action based on whether the event is an open or a click.

The Wait task introduces a grace period before follow-up emails are sent via an HTTP task using SendGrid’s API.

Here’s an overview of the system we are going to build:

Create an email nurturing workflow in Conductor.
Get the API key from the SendGrid account.
Store the SendGrid API key as a Secret in Conductor.
Create a webhook in Conductor.
Configure a webhook event in SendGrid.
Modify and run Conductor workflow.
Create a Single Send in SendGrid.
Automate the flow using Conductor’s Workflow Scheduler.

Follow the tutorial using the free Developer Playground. Sign up for an account to get started.

Step 1: Create an email nurturing workflow

In Orkes Conductor, application flows can be built as JSON workflows using SDKs, APIs, or Conductor UI.

In this tutorial, let’s create the workflow using Conductor UI.

To create the workflow:

Go to Definitions > Workflows from the left navigation menu on your Conductor cluster.
Select + Define workflow.
Select the Code tab on the right and paste the following code:

{
 "name": "emailNurturing",
 "description": "An email nurturing flow",
 "version": 1,
 "tasks": [
   {
     "name": "webhook",
     "taskReferenceName": "webhook_ref",
     "inputParameters": {
       "matches": {
         "request[0].event": "${workflow.input.input}"
       }
     },
     "type": "WAIT_FOR_WEBHOOK"
   },
   {
     "name": "switch_event",
     "taskReferenceName": "switch_ref",
     "inputParameters": {
       "switchCaseValue": "${webhook_ref.output.request[0].event}"
     },
     "type": "SWITCH",
     "decisionCases": {
       "click": [
         {
           "name": "wait_before_click_follow_up",
           "taskReferenceName": "wait_ref_1",
           "inputParameters": {
             "duration": "10 seconds"
           },
           "type": "WAIT"
         },
         {
           "name": "send_click_follow_up_email",
           "taskReferenceName": "http_1_ref",
           "inputParameters": {
             "uri": "https://api.sendgrid.com/v3/mail/send",
             "method": "POST",
             "accept": "application/json",
             "contentType": "application/json",
             "encode": true,
             "body": {
               "personalizations": [
                 {
                   "to": [
                     {
                       "email": "${webhook_ref.output.request[0].email}"
                     }
                   ],
                   "subject": "🌟 Ready to Unlock the Full Potential of AI Agent?"
                 }
               ],
               "from": {
                 "email": "<SENDGRID-VERIFIED-EMAIL>"
               },
               "content": [
                 {
                   "type": "text/plain",
                   "value": "Hi,\n\nThank you for checking out the AI Agent—we noticed you took a closer look, and we’re excited to help you take the next step! 🚀\n\nOur AI Agent is designed to make your life easier, with a variety of powerful features that will save you time and boost your productivity:\n\n- Automate Repetitive Tasks: Let AI handle the routine stuff so you can focus on the bigger picture.\n- Personalized Assistance: Get 24/7 support tailored to your needs.\n- Seamless Integration: Easily connect AI Agent with the tools and platforms you already use.\n\n🛍 Here’s Your Special Offer:\n- 10% off when you sign up in 7 days.\n- Exclusive Early Access to all features, before they’re made available to the public.\n\n👉 Ready to get started? https://example.com\n\nWe truly believe the AI Agent can transform the way you work and would love for you to experience its benefits firsthand. If you have any questions or need assistance, don’t hesitate to reach out—we’re here to help!"
                 }
               ]
             },
             "headers": {
               "Authorization": "Bearer ${workflow.secrets.SendGrid}"
             }
           },
           "type": "HTTP"
         }
       ],
       "open": [
         {
           "name": "wait_before_open_follow_up",
           "taskReferenceName": "wait_ref",
           "inputParameters": {
             "duration": "10 seconds"
           },
           "type": "WAIT"
         },
         {
           "name": "send_open_follow_up_email",
           "taskReferenceName": "http_ref",
           "inputParameters": {
             "uri": "https://api.sendgrid.com/v3/mail/send",
             "method": "POST",
             "accept": "application/json",
             "contentType": "application/json",
             "encode": true,
             "body": {
               "personalizations": [
                 {
                   "to": [
                     {
                       "email": "${webhook_ref.output.request[0].email}"
                     }
                   ],
                   "subject": "Still Thinking About AI Agent? Here's What You Need to Know!"
                 }
               ],
               "from": {
                 "email": "<SENDGRID-VERIFIED-EMAIL>"
               },
               "content": [
                 {
                   "type": "text/plain",
                   "value": "Hi,\n\nWe noticed that you opened our email about the AI Agent—but haven’t had a chance to explore it fully yet. We wanted to make sure you don’t miss out on how this AI-powered tool can transform the way you work!\n\nHere’s a quick reminder of what AI Agent can do for you:\n\n- Automate Time-Consuming Tasks: Let the AI take over repetitive processes so you can focus on more important things.\n- 24/7 Personal Assistant: Get support whenever you need it—no matter the time of day.\n- Integrate with Your Workflow: Effortlessly connect AI Agent to your favorite tools for a seamless experience.\n\n💡 Why Act Now?\n- 10% off if you sign up in 7 days.\n- Exclusive Early Access: Be one of the first to harness the power of our AI Agent before it’s available to everyone!\n\n👉 **Don’t miss out: https://example.com\n\nWe truly believe that AI Agent will be a game-changer for you, and we'd love for you to experience it firsthand. If you have any questions or need more details, just reply to this email!\n\nLooking forward to hearing from you!\n\nBest regards,"
                 }
               ]
             },
             "headers": {
               "Authorization": "Bearer ${workflow.secrets.SendGrid}"
             }
           },
           "type": "HTTP"
         }
       ]
     },
     "defaultCase": [],
     "evaluatorType": "value-param",
     "expression": "switchCaseValue"
   }
 ],
 "inputParameters": [
   "input"
 ],
 "schemaVersion": 2
}

Rename the workflow Name to something unique.
Select Save > Confirm.

Now that the workflow is set up, the next step is to get the API keys from your SendGrid account.

Step 2: Get the API key from the SendGrid account

To send follow-up emails, Conductor’s HTTP task calls SendGrid's Send Message API. This API request is authenticated using the SendGrid API key.

To get the API key:

Steps to generate and secure your SendGrid API key

Sign in to SendGrid.
Go to Settings > API Keys from the left navigation menu.
Select Create API Key, and generate the key with Full Access permission.
Copy and store the key.

Now that you have the API key, the next step is to store it as a secret in Conductor.

Step 3: Store the SendGrid API key as a Secret in Conductor

Conductor’s Secrets feature provides a secure way to store sensitive information that workflow definitions should not expose. By storing these values as secrets, you can reference them within workflows without revealing the actual values. This ensures better security and helps maintain the confidentiality of critical data.

Go to Definitions > Secret on your Conductor cluster to create a secret. Ensure that the Secret name is SendGrid, matching the secret name used in the workflow definition.

Storing the SendGrid API key as a secret in Orkes Conductor

Your API key is now securely stored with Conductor.

Returning to your workflow, you can verify that this secret authorizes the SendGrid API request within the HTTP task.

Authenticating API requests without exposing keys

Step 4: Create a webhook in Conductor

The next step is to create a webhook in Conductor that will listen for events coming from SendGrid.

To create a webhook:

Go to Definitions > Webhook from the left navigation menu on your Conductor cluster.
Select + New webhook.
Configure the following parameters:

In Webhook name, enter a unique name for the webhook.
In Workflows to receive Webhook event, select the workflow created in Step 1.
Select the Source platform as SendGrid.
Select Save.

Creating a webhook in Orkes Conductor

An unverified webhook URL will be generated, which is to be copied to the SendGrid webhook.

Step 5: Configure an event webhook in SendGrid

Next, you must set up an event webhook in SendGrid to send events to your Conductor webhook.

To create an event webhook in SendGrid:

Go to Settings > Mail Settings from the left menu on your SendGrid Console.
In Webhook Settings > Event Webhooks, select Create new Webhook.
Configure the following parameters:

Provide a Friendly Name.
In Post URL, enter the unverified webhook URL generated in Conductor
In Actions to be posted, select Opened and Clicked.
In Signature verification, toggle on Enable Signed Event Webhook.

Select Save.

Setting up a SendGrid Event Webhook to track email opens and clicks

Next, copy the Verification key from SendGrid and paste it into the Conductor webhook.

Copying SendGrid verification key to Orkes Conductor webhook

To verify the webhook URL, go back to SendGrid and select Test Integration. This sends a test event to the Conductor webhook, verifying the URL.

Webhook verification in Conductor

Step 6: Modify and run Conductor workflow

Before running the workflow, you need to modify it to include the verified SendGrid sender email, which will be used when sending the campaigns.

To modify the workflow definition:

Open your workflow definition, and select send_open_follow_up_email task.
Update the From email address to your SendGrid account’s verified sender email.

Modifying workflow

Repeat Step 2 for the send_click_follow_up_email task.
Save the workflow.

Now, let's run the workflow.

The Wait for Webhook’s task input matches are wired to the workflow input:

   "inputParameters": {
       "matches": {
         "request[0].event": "${workflow.input.input}"
       }
     }

Since the workflow is designed to send follow-up emails on "opened" or "clicked" events, the SendGrid webhook event payloads are open and click, respectively.

For testing purposes, we’ll start with an open event.

Go to the Run tab and provide the input parameter.

{
 "input": "open"
}

Select Run workflow.

The workflow is now running and waiting for events from SendGrid.

Step 7: Create a Single Send in SendGrid

Next, let's send the SendGrid campaign. Campaigns are scheduled in production, but here, we’ll create a Single Send to test the workflow.

Prerequisites:

To create a single send:

Go to Marketing > Single Send from the left menu on your SendGrid console.
Select Create a Single Send.
Draft your email content.
Configure the Single Send settings.

Creating a single send in SendGrid

Ensure the From Sender email matches the one used in the HTTP task.
Review the email content and select Review Details and Send.

This schedules the email to be sent immediately, typically within 2 minutes.

An end user's first touchpoint in the nurture journey

When the recipient opens the email (one of the events configured in the SendGrid webhook), the event triggers and is sent to the Conductor.

SendGrid logs an interaction, triggering the webhook event to Conductor

Checking the workflow execution, you can verify that the Wait for Webhook task is completed and the workflow proceeds with the “open” case.

Workflow triggered by Webhook event

The wait task is configured for 10 seconds for testing purposes. In a production environment, the follow up emails are ideally sent after 3-4 days, so the Wait task must be adjusted accordingly.

After 10 seconds, a follow-up email is sent via the SendGrid API, and the workflow completes successfully.

A personalized response triggered by user behavior

Similarly, you can trigger the workflow for a “click” event.

Step 8: Automate using Workflow Scheduler

The final step is automating the workflow to run in sync with your SendGrid campaigns.

Tip: Before automation, update the workflow's wait period to 3 or 4 days, depending on when you want to send the follow-up email.

For example, let’s say your product launch email is scheduled on the last day of the month.

Now, we need to create a workflow schedule that will trigger the workflow simultaneously with both inputs (“open” and “click”), automating the entire process.

To trigger the workflow simultaneously for an “open” event, create a Workflow Schedule and use cron expressions to run it on the last day of the month.

Workflow scheduled for the last day of the month for “open” event

Similarly, another schedule can be created for the “click” event using Conductor's clone functionality. You can clone the schedule and update the workflow input accordingly.

Clone and tweak schedules for flexible automation

Here’s the cloned schedule for the “click” event:

Workflow scheduled for the last day of the month for “click” event

With the workflow schedulers in place, both the "open" and "click" workflows will be triggered along with the corresponding email campaigns from SendGrid.

Congratulations! You've successfully created an automated email nurturing flow for a product launch campaign. This workflow can now be reused for future product launches, ensuring a streamlined process. You can also leverage features like Workflow Versioning to manage different email content for A/B testing.

Feel free to modify the workflow further to suit your needs—whether by adding more follow-up emails or customizing actions based on user behavior.

youtube.com

Summary

Using SendGrid webhooks in Conductor, you’ve created an automated email nurturing workflow that responds to user interactions. This reusable flow streamlines future product launches and campaigns, enabling you to focus on optimizing strategies instead of managing responses manually.

–

Conductor is an enterprise-grade Unified Application Platform for process automation, API and microservices orchestration, agentic workflows, and more. Check out the full set of features, or try it yourself using our free Developer Playground.

Ensuring 99.999% Uptime: Why Service Uptime Monitoring is Crucial

Riza Farheen — Tue, 25 Feb 2025 11:13:25 +0000

How often have you been frustrated by a website that’s down, taking forever to return online? If it happens too frequently, would you trust that platform again? Probably not.

With businesses running 24/7, downtime is more than just an inconvenience–it’s a direct hit to customer trust and revenue. Studies reveal that businesses can lose thousands of dollars for every hour of downtime. This is why businesses strive for “five-nines” (99.999%) uptime, with an average of less than 6 minutes of downtime annually.

Major outages can bring businesses to a standstill, leading to frustrated customers. This is where uptime monitoring becomes crucial to minimize potential losses. By proactively detecting downtime, businesses can prevent significant disruptions, keep things running smoothly, and build reliable services.

In this blog, we’ll explore why service uptime monitoring is essential for modern businesses and how Orkes Conductor can help implement application flows to monitor HTTP endpoint availability.

What is service uptime monitoring?

Service uptime monitoring is a process of monitoring and managing critical services, such as a website or application, to ensure they remain accessible 24/7.

With continuous monitoring, businesses can receive automated alerts when disruptions occur, allowing them to take immediate action. A robust system can notify relevant team members about any detected downtimes via multiple communication channels, ensuring the right team members can respond promptly.

Why use service uptime monitoring?

The demand for fast and reliable online services has always been higher, especially with the growing shift toward digital services. For instance, the increasing trend of digital payments is rapidly replacing traditional cash transactions.

According to Visa Perspective’s Top trends shaping 2025 payments:

Source: Visa Perspective

The payment industry is just one example of the growing shift toward online services. Industries like e-commerce and others are also witnessing increased consumer demand for digital platforms. As more businesses move toward online services, ensuring uptime monitoring becomes vital. By keeping track of service status, you can identify potential issues—such as outages, performance slowdowns, or errors—before they escalate into major problems.

Here are a few reasons why uptime monitoring is a must for businesses today:

1. Prevent reputation damage

Outages can severely damage your organization’s reputation. Frustrated users experiencing downtime may leave negative reviews, vent on social media, or decide to switch to a competitor.

2. Meet Service Level Agreements (SLAs)

Many businesses establish Service Level Agreements (SLAs) to meet specific performance standards. Failure to detect an outage can lead to missed SLAs, potentially resulting in contract violations and penalties.

3. Minimize loss of revenue

Downtime can directly impact revenue for businesses that rely on online sales and transactions. Customers may be unable to access services or complete purchases, leading to missed sales opportunities.

Consistent service uptime monitoring is essential to overcome these challenges. It ensures your services are quickly restored, minimizing disruptions and enhancing the user experience.

Building a service uptime monitoring with Orkes Conductor

Orkes Conductor, the leading workflow orchestration platform, can be leveraged to build a service uptime monitoring system that fits your business operations. Unlike off-the-shelf monitoring tools, Orkes Conductor gives you full control over uptime checks, allowing you to customize logic to fit your exact business needs. Let’s walk through building the service uptime monitoring flow.

To get started, simply create a free account in Developer Playground and follow along with the tutorial. This example is available as a ready-to-use template. For more details, check out the Availability Monitoring for HTTP Endpoints documentation to learn how to import and use it.

Service uptime monitoring flow

Let’s break down the logic for the service uptime monitoring system:

The system checks the service endpoint for availability.
It retrieves the server’s timestamp and status code.
If downtime is detected, a notification is sent via the preferred channel.

Now, let's build the workflows that will orchestrate this monitoring system. Here’s an overview of the service uptime monitoring system we are going to build:

Create a workflow to monitor the endpoints.
Create a separate workflow for sending downtime notifications and call it as a sub-workflow.
Automate the monitoring workflow to run every 2 minutes.

Part 1: Availability monitoring

The first part of the service uptime monitoring system checks the endpoint’s availability. This process can be accomplished by creating a Monitor-HTTP-Endpoint-Availability workflow.

Step 1: Check the endpoint availability

The first step is to check if the HTTP(s) endpoints are available. To check the availability, create an HTTP task in your Conductor workflow. Configure the endpoint method, URL, and other parameters. When the workflow is executed, it will call the endpoint and retrieve the response.

Use an HTTP task in Orkes Conductor to call an endpoint

Step 2: Retrieve the timestamp

If the server is down, it’s essential to know when the downtime was detected. To achieve this, use an Inline task that extracts the timestamp when the response was received.

Use an Inline task in Orkes Conductor to extract the timestamp

Step 3: Evaluate the server’s status code

Use the Switch task in your Conductor workflow to evaluate the response from the HTTP task. If the response indicates success, the workflow will terminate. If failure is detected, the workflow will proceed to the next part, which involves notifying the appropriate team.

Use the Switch task in Orkes Conductor to evaluate the response from the previous HTTP task

The sample workflow input includes the notification method (e.g., SMS), the endpoint URL to be monitored, and the sender/receiver information for notifications.

{
  "notification_type": "SMS",
  "endpoint_url": "https://example.com/",
  "notification_from": "14XXXXXXXXX",
  "notification_to": "12XXXXXXXX"
}

Here is the full Monitor-HTTP-Endpoint-Availability workflow.

Monitor-HTTP-Endpoint-Availability workflow in Orkes Conductor

The notification step is incorporated as a Sub Workflow, which facilitates best practices in modularization.

Using the Sub Workflow task to call the notification workflow

Sub Workflows in Conductor simplifies workflow management, making it easy to reuse common workflows across multiple processes.

The notification system is a typical flow within an organization when building applications. Creating a dedicated workflow for your organization’s payment system that integrates all notification methods allows you to easily adapt your notification flow to different application processes using the Sub Workflow functionality in Conductor.

Part 2: Notifying users

Once the endpoint is monitored and downtime is detected, the next part is to notify the corresponding team members to take action. This is where the second part of the application flow is to be implemented. For this second part, let’s build a Notify-Channels workflow that sends notifications via the preferred channel.

Step 1: Check the preferred notification method

Let's add a Switch task to a new workflow to assess the input and determine the preferred notification method.

Using the Switch task to evaluate the switch condition based on workflow inputs

The Switch task evaluates the workflow input and proceeds along the appropriate path.

Step 2: Send notification via the preferred channel

With the chosen notification type, the next step is to send the service downtime alert through the selected channel. In this workflow, we’ve defined four cases: SMS, email, Slack, and PagerDuty.

Here’s an example of a flow that sends notifications via SMS using Twilio as the provider.

The notification flow for alerting through SMS via Twilio in Orkes Conductor

The workflow is set up to send notifications via SMS using Twilio, emails through SendGrid, Slack channel alerts, and PagerDuty notifications.

The notification flow for alerting through multiple channels

The notification method here is not restricted to just four channels. Adding more switch cases allows you to incorporate other notification systems to accommodate the organization’s preferences.

Here is the full Notify-Channels workflow.

Notify-Channels workflow in Orkes Conductor

Part 3: Automating the monitoring process

The final part is automating the monitoring process to check the endpoint availability every 2 minutes to achieve 99.999% uptime.

For this, create a Workflow Scheduler in Conductor, using cron expressions to set the workflow to run every 2 minutes. Once configured, Conductor will automatically trigger the workflow every 2 minutes without requiring any further action from the user.

Monitor HTTP Endpoints Availability workflow scheduled to run every 2 mins

The ability to schedule workflows using cron expression greatly streamlines the automation of your workflows. Conductor further enhances this experience by supporting different time zones, avoiding the need for manual time conversions. Yet another outstanding feature of Scheduler is its user-friendly interface, which displays upcoming scheduled runs, allowing you to verify and confirm your schedules easily.

While this uptime monitoring system is a simple version, you can use Orkes Conductor to extend the flow further to suit your specific application needs.

This workflow is available as an out-of-box template, allowing you to receive alerts via PagerDuty, Slack, SMS, and email. For more details, check out the Availability Monitoring for HTTP Endpoints documentation to learn how to import and use the template.

Summary

Uptime monitoring of services is crucial for a business to thrive, as it directly impacts revenue. By leveraging tools like Orkes Conductor, you can build an automated monitoring system to track your services 24/7 to achieve 99.999% uptime. This, in turn, can decrease the customer churn rate and ultimately enhance customer satisfaction and loyalty.
–

Build an AI Application for Document Classification: A Step-by-Step Guide

Riza Farheen — Thu, 06 Feb 2025 06:43:08 +0000

This is Part 2 of the AI App Development series, which will demonstrate how to build an application that uses LLMs to automate document classification. Check out Part 1 for a use case on summarizing an article.

In a modern data-driven world, organizations handle an ever-growing volume of documents, ranging from invoices and contracts to reports and records. Manually sorting and categorizing these documents is now a thing of the past—a daunting task that is time-consuming, resource-intensive, and prone to human error. Misclassifications can lead to operational inefficiencies, compliance risks, and increased costs.

AI-powered document classification automates this process, enabling organizations to handle large volumes of documents quickly and accurately.

This blog will demonstrate how to build an AI application that classifies your documents using Orkes Conductor.

Building a document classifier application flow

Document classification is the process of classifying documents into predefined categories based on their content. It is widely used across industries to streamline the sorting, categorizing, and processing of large volumes of documents.

In this tutorial, we’ll create an AI application flow that:

Identifies PDF documents from a given URL.
Classifies them into predefined categories such as W2 forms, driving licenses, pay stubs, employment verification letters, or mortgage applications.
Handles errors for unsupported file types.

Let’s break down the workflow logic.

The workflow begins with a Switch task that evaluates whether the input document URL points to a PDF file (i.e., has a .pdf extension). Based on the evaluation, it then follows one of the two cases:

defaultCase–Triggered if the input file is not a PDF. The workflow proceeds to an Inline task that generates an error message indicating the unsupported file type, followed by a Terminate task that halts the workflow and displays the error message.
PDF–Triggered if the input file is a PDF. The workflow then moves to a Get Document task that retrieves the PDF content. If the file is text-based, this task extracts the text from the document.

The next step involves another Switch task that evaluates the output from the Get Document task. If the document is text-based, the defaultCase is executed. If the document is non-text-based, the workflow routes to the ocr_processing case, which uses Optical Character Recognition (OCR) to process the image content in the PDF.

An OCR-based Worker task (Simple) then extracts the image content from the PDF. This task requires an external worker, set up outside the Conductor, to poll for the task.

Following that, an Inline task formats the extracted text, preparing it for the document classification process. The classification step involves an LLM Text Complete task that uses a prompt to categorize the document into different categories. The task leverages a prompt template created within Conductor to determine the file’s type,

Finally, the workflow concludes with an Inline task that extracts, formats, and publishes the classification results.

Step 1: Create a document classifier workflow

To follow the tutorial, let’s use the free Developer Playground. Create an account to get started.

You can create application flows as JSON workflows in Conductor using SDKs, APIs, or Conductor UI. For easier testing, let’s create a workflow in the Conductor UI.

First, let's define the worker task that will be part of the workflow.

To create a task definition:

Go to Developer Playground.
In the left navigation menu, select Definitions > Task.
Select + Define task.
In Name, enter extractTextFromPDF.
In Description, enter a brief explanation of the task’s purpose.
Select Save > Confirm Save.

Next, let's create a workflow that includes this task.

To create the workflow:

Go to Definitions > Workflow from the left navigation menu.
Select + Define workflow.
Select the Code tab on the right and paste the following JSON code:

{
 "name": "document_classifier_workflow",
 "description": "LLM Powered PDF Document Classification Workflow",
 "version": 1,
 "tasks": [
   {
     "name": "check_for_pdf",
     "taskReferenceName": "check_for_pdf_ref",
     "inputParameters": {
       "url": "${workflow.input.document_url}"
     },
     "type": "SWITCH",
     "decisionCases": {
       "PDF": [
         {
           "name": "get_document_task",
           "taskReferenceName": "get_document_task_ref",
           "inputParameters": {
             "url": "${workflow.input.document_url}",
             "mediaType": "application/pdf"
           },
           "type": "GET_DOCUMENT"
         },
         {
           "name": "check_for_output",
           "taskReferenceName": "check_for_output_ref",
           "inputParameters": {
             "textOutput": "${get_document_task_ref.output.result}"
           },
           "type": "SWITCH",
           "decisionCases": {
             "ocr_processing": [
               {
                 "name": "extractTextFromPDF",
                 "taskReferenceName": "extractTextFromPDF_ref",
                 "inputParameters": {
                   "pdfUrl": "${workflow.input.document_url}"
                 },
                 "type": "SIMPLE"
               }
             ]
           },
           "evaluatorType": "graaljs",
           "expression": "(function () {\n   if($.textOutput === \"\\n\"){\n    return \"ocr_processing\"\n   }\n   return \"defaultCase\"\n  }())"
         },
         {
           "name": "evaluate_text",
           "taskReferenceName": "evaluate_text_ref",
           "inputParameters": {
             "expression": "(function () {\n  if($.getDocumentText === \"\\n\"){\n    const output = $.ocrText.results.map((item) => \n    \"PageNumber: \"+item.page+\"\\nContent: \"+item.text\n  ).join('\\n\\n');\n  \n    return output\n  }\n  return $.getDocumentText;\n})();",
             "evaluatorType": "graaljs",
             "getDocumentText": "${get_document_task_ref.output.result}",
             "ocrText": "${extractTextFromPDF_ref.output}"
           },
           "type": "INLINE"
         },
         {
           "name": "classify_using_llm",
           "taskReferenceName": "classify_using_llm",
           "inputParameters": {
             "promptName": "your-prompt-template",
             "promptVariables": {
               "text": "${evaluate_text_ref.output.result}"
             },
             "llmProvider": "your-llm-provider",
             "model": "your-llm-model"
           },
           "type": "LLM_TEXT_COMPLETE"
         },
         {
           "name": "extract_classification",
           "taskReferenceName": "extract_classification",
           "inputParameters": {
             "expression": "(function(){ \n  let llmResult = new String($.llmResult).trim();\n  if(llmResult === 'NO_MATCH') {\n    return \"This document couldn't be classified by the LLM\";\n  }\n  return \"LLM classified this document as: \" + llmResult;\n})();",
             "evaluatorType": "graaljs",
             "llmResult": "${classify_using_llm.output.result}"
           },
           "type": "INLINE"
         }
       ]
     },
     "defaultCase": [
       {
         "name": "generate_error",
         "taskReferenceName": "generate_error_ref",
         "inputParameters": {
           "expression": "(function(){ \n\n  return \"The filename supplied: \" + $.filename + \" is an unsupported file type. Please try again with a .pdf file\";\n})();",
           "evaluatorType": "graaljs",
           "filename": "${workflow.input.document_url}"
         },
         "type": "INLINE"
       },
       {
         "name": "terminate_task",
         "taskReferenceName": "terminate_task_ref",
         "inputParameters": {
           "terminationStatus": "TERMINATED",
           "terminationReason": "Unsupported file type",
           "workflowOutput": {
             "result": "${generate_error_ref.output}"
           }
         },
         "type": "TERMINATE"
       }
     ],
     "evaluatorType": "graaljs",
     "expression": "$.url.toLowerCase().trim().endsWith(\"pdf\") ? \"PDF\" : \"NOT_SUPPORTED\";"
   }
 ],
 "inputParameters": [
   "document_url"
 ],
 "schemaVersion": 2
}

Change the workflow Name to something unique.
Select Save > Confirm.

Now that your workflow is set up, the next step is configuring an OCR worker to extract content from non-text-based PDFs.

Step 2: Set up OCR worker

In this example, we will set up the OCR worker project using Node.js.

Prerequisites:

Node.js version 20.18.2 or later.

To connect your Node.js worker with Conductor, you need access keys from an application in Orkes Conductor.

To generate the access keys:

Go to Developer Playground.
In the left navigation menu, select Access Control > Applications.
Select + Create application.
Enter a Name for the application, and select Save.
In Application roles, enable the Worker role.
In Access Keys, select + Create access key.
Copy the Key Id and Key Secret and store them securely.

Note: If you're following this tutorial with Developer Playground, you can skip the steps below. However, if you're using your own Conductor cluster, you must enable additional permissions for tasks and workflows:

In your application’s Permission section, select + Add permission.
In the Workflow tab, search for your workflow and turn on READ and EXECUTE permissions.
In the Task tab, search for the task extractTextFromPDF and turn on READ and EXECUTE permissions.
Select Add permissions.

To set up a Node.js worker:

Create a folder for your project.
Open a terminal and navigate to the folder. Then, run:

npm init -y

Run the following command to install the required dependencies:

npm install @io-orkes/conductor-javascript @napi-rs/canvas pdfjs-dist tesseract.js

Add an index.js file in the project root and paste the following worker code:

import Tesseract from "tesseract.js";
import https from "https";
import { createCanvas } from "@napi-rs/canvas";
import { getDocument } from "pdfjs-dist/legacy/build/pdf.mjs";
import {
  orkesConductorClient,
  TaskManager,
} from "@io-orkes/conductor-javascript";

async function extractTextFromPDF(pdfUrl) {
  try {
    const pdfBuffer = await new Promise((resolve, reject) => {
      https
        .get(pdfUrl, (response) => {
          if (response.statusCode !== 200) {
            reject(new Error(`HTTP ${response.statusCode}`));
            return;
          }

          if (!response.headers["content-type"]?.includes("application/pdf")) {
            reject(new Error("Not a PDF file"));
            return;
          }

          const chunks = [];
          response.on("data", (chunk) => chunks.push(chunk));
          response.on("end", () => resolve(Buffer.concat(chunks)));
        })
        .on("error", reject);
    });

    const pdfDoc = await getDocument(new Uint8Array(pdfBuffer)).promise;
    const results = [];

    for (let pageNum = 1; pageNum <= pdfDoc.numPages; pageNum++) {
      const page = await pdfDoc.getPage(pageNum);
      const viewport = page.getViewport({ scale: 2.0 });
      const canvas = createCanvas(viewport.width, viewport.height);

      await page.render({
        canvasContext: canvas.getContext("2d"),
        viewport: viewport,
      }).promise;

      const {
        data: { text },
      } = await Tesseract.recognize(canvas.toBuffer("image/png"), "eng");

      results.push({ page: pageNum, text: text.trim() });
    }

    return results;
  } catch (error) {
    throw new Error(`PDF processing failed: ${error.message}`);
  }
}

// Conductor worker configuration

async function startWorker() {
  const clientPromise = orkesConductorClient({
    keyId: "YOUR-KEY-ID",
    keySecret: "YOUR-KEY-SECRET",
    serverUrl: "YOUR-CONDUCTOR-CLUSTER-URL/api",
  });

  const client = await clientPromise;

  const pdfWorker = {
    taskDefName: "extractTextFromPDF",
    execute: async ({ inputData: { pdfUrl }, taskId }) => {
      try {
        if (!pdfUrl) {
          return {
            status: "FAILED",
            outputData: { error: "PDF URL is required" },
          };
        }

        const results = await extractTextFromPDF(pdfUrl);

        return {
          status: "COMPLETED",
          outputData: {
            results,
            processedPages: results.length,
          },
        };
      } catch (error) {
        return {
          status: "FAILED",
          outputData: { error: error.message },
        };
      }
    },
  };

  const manager = new TaskManager(client, [pdfWorker], {
    options: { pollInterval: 100, concurrency: 1 },
  });

  console.log("Starting PDF extraction worker...");
  manager.startPolling();

  // Handle graceful shutdown
  process.on("SIGINT", () => {
    console.log("Stopping worker...");
    manager.stopPolling();
    process.exit(0);
  });
}
startWorker();

In the worker code, replace YOUR-SECRET-ID and _YOUR-SECRET-KEY _ with previously copied Key Id and Key Secret.
Replace “YOUR-CONDUCTOR-CLUSTER-URL/api” with your cluster URL. For Developer Playground, use https://developer.orkescloud.com/api.
Run the worker:

node index.js

Now that your workflow and worker are ready, it’s time to get it up and running by adding your LLM integration.

Step 3: Add your preferred LLM integration

Orkes Conductor offers integration with many LLM providers. The next step is to add an integration in the Conductor cluster with your preferred LLM provider.

To add an integration:

Go to Integrations from the left navigation menu on your Conductor cluster.
Select + New integration.

In AI/LLM section, choose your preferred LLM provider and select + Add.

The configuration parameters vary depending on the AI/LLM platform being integrated. Refer to the respective integration guides to obtain the parameters.

Enter the API keys and other parameters based on the chosen LLM provider.
Select Save.

The LLM integration has been added. The next step is to add a specific model. It's important to choose a model that aligns with your use case.

To add a model to your integration:

In the Integrations page, select the + button next to your newly-added integration.
Select + New model.
Enter the Model name and Description.
Ensure that the Active toggle is switched on and select Save.

The integrations are now set up and ready to use. Let's create an AI prompt that will utilize this integration to classify the documents into appropriate categories.

Step 4: Create a prompt template using Orkes’s AI prompt builder

A prompt plays a crucial role in guiding the LLM model to act in a specific manner. For our document classification workflow, the prompt should be carefully crafted to enable the model to identify and categorize the document into appropriate groups.

To create an AI prompt:

Go to Definitions > AI Prompts.
Select + Add AI prompt.
In Prompt Name, enter a unique name for your prompt, such as “doc_classifier_prompt_yourname”.
In Model(s), select the integration you added in the previous step. The UI drop-down lists the integration along with the model names. Make sure to choose the right one.
Enter a Description of what the prompt does.
In Prompt Template, enter your prompt, which will classify the document into appropriate categories. For example:

We have a document that was scanned using OCR. The content is ${text}. You need to classify the document based on the provided OCR content. The document could be one of these: W2, Drivers License, Pay stub, Employment Verification Letter, or Mortgage Application. Suppose the provided content does not match with any of those documents. In that case, you must reply NO_MATCH, and in the following line, you must give a human-understandable message about the result and why that determination was made in under three sentences. If the provided content matches, return the values found, including the document type. If the social security number is part of the values, obfuscate the first five digits.

In the prompt, we have defined ${text} as a variable that will be derived from the output of the previous task. This will become clearer once we incorporate this prompt into the workflow.

Select Save > Confirm save.

This saves your prompt template. Orkes Conductor provides a built-in testing interface to test and validate your prompt before adding it to the workflow. In our example, we have defined the ${text} variable to be derived from the output of the preceding text extraction task. For example, here’s what a test run looks like with sample data.

Step 5: Edit workflow with your LLM integration and AI prompt

Now that everything is in place, let’s bring it all together and get the workflow running.

Let’s go back to the workflow created:

In Developer Playground, go to Definitions > Workflow and select the workflow created previously.
In the classify_using_llm task, replace the following values:

Replace your-llm-provider with your integrated LLM provider.
Replace your-llm-model with your chosen model.
Replace your-prompt-template with your prompt.
Add the promptVariable `text’ parameter as ‘${evaluate_text_ref.output.result}’.

You can see that the text variable is set to ${evaluate_text_ref.output.result}, which is the output result of the evaluate_text task that retrieves the content of the PDF document.

Select Save > Confirm.

Now, let's test run your workflow:

Select Run tab and provide the document URL to be classified as the input parameter.
Select Run Workflow.

// example input params { "document_url": "https://image-processing-orkes.s3.amazonaws.com/test-w2-form-full-text.pdf" }

Upon running the workflow, you will be directed to the workflow execution page. If you select the Workflow Input/Output tab, you will see the category in which the document has been classified.

Here is a sample test run results with a W2 document passed as the input:

Here is a sample test run results with a pay stub passed as the input:

Ta-da! You have successfully created an application flow that accurately classifies your documents.

You can either develop your application from the ground up using Conductor or wire the necessary application flow into your existing applications through Conductor SDKs or APIs.

Let’s see what happens when an image file with a .jpg extension is passed as input.

The workflow will terminate, and you’ll receive an output message indicating that the file type is unsupported.

This workflow is currently designed to work with input URLs with a .pdf extension. But what if your input is an image file or the document falls into a category not listed, requiring manual verification without disrupting the workflow?

No worries—Orkes Conductor has got you covered! We have developed an advanced version of this workflow to handle these scenarios seamlessly. Ready to explore it? Stay tuned for the upcoming tutorials.

In the meantime, if you have any ideas for improving this workflow to support additional file types, feel free to join our community on Slack.

In summary, Orkes Conductor stands out as the best choice for creating application flows. It offers flexibility in prompt customization and prioritizes data privacy, ensuring your data stays secure while allowing private LLM configurations with Conductor.

Change Data Capture (CDC) in Event-Driven Microservices

Riza Farheen — Wed, 23 Oct 2024 08:38:55 +0000

Change Data Capture (CDC) is a technique for identifying and capturing changes in a database and replicating them in real time to other systems. In an event-driven architecture, CDC captures changes from a source database and transforms them for easy use by target systems. This process is crucial for maintaining consistency and functionality across all data-dependent systems.

In this blog, we’ll explore CDC in event-driven microservices and how to send workflow state changes to a stream from Orkes Conductor, the leading orchestration platform.

What is Change Data Capture (CDC)?

Traditional data movement typically uses an ETL (Extract, Transform, Load) tool to extract source data, transform it, and load it into a target database. This process often operates within a defined cache window, which limits how frequently data is moved.

On the other hand, a CDC system captures and moves data changes in real time. This keeps systems in sync and provides a reliable method for data replication, allowing for zero-downtime cloud migrations, making CDC well-suited for modern cloud architecture.

To understand the concept of CDC, let’s consider a traditional ETL (Extract, Transform, Load) pipeline as shown below:

A key challenge here is managing continuous updates in the source data, which can lead to inconsistencies between the source and target systems. These challenges make maintaining data accuracy and timely updates difficult across the entire ETL pipeline.

CDC addresses these challenges by capturing real-time changes in the source database, allowing only the changed data to be replicated in the target system. When combined with an event-driven architecture, CDC enables changes to be propagated as events, which can trigger real-time processing and synchronization in the target system. This reduces latency and ensures the target system remains up-to-date without needing complete data extractions.

CDC in Event-driven Microservices

CDC has become an essential strategy in event-driven microservices architecture, bridging traditional databases and modern cloud-native systems.

While CDC captures data modifications (such as inserts, updates, and deletes) at the database level, it needs a messaging system to propagate those changes to relevant microservices. The most efficient way to accomplish this is to treat the changes as events within an event-driven architecture and send them asynchronously.

Here’s an overview of the CDC process with an event bus:

The process can be split into three phases:

Change Detection and Event Generation
Change Event Ingestion
Change Event Propagation

Change Detection and Event Generation

In this phase, a log-mining component monitors the database’s transaction logs for changes whenever a microservice interacts with the database (e.g., creating a new order or updating customer data). Detected changes are formatted as individual events (e.g., "OrderCreated" or "CustomerUpdated"), timestamped, and published to the event bus. Each database operation results in a corresponding change event, ensuring that downstream consumers receive the necessary data updates.

Change Event Ingestion

In this phase, the event bus receives and stores the change events. Events are typically organized into topics/queues, each mapped to a unique table in the source database.

Change Event Propagation

The generated event is published to a messaging platform using a publish/subscribe (pub/sub) model. This allows downstream systems or microservices to subscribe to specific topics based on their needs. This ensures that the events are transmitted to required consumers without tight coupling or direct dependencies.

Let's now look into how to send change data capture streams from Orkes Conductor to external systems.

CDC with Orkes Conductor

Orkes Conductor is a powerful tool for developers to build distributed applications or manage orchestration. You can easily stream workflow state changes to external systems with CDC-enabled workflows.

Enabling the workflow status listener in a Conductor workflow allows you to detect and capture workflow state changes in real time. These changes are ingested into the integrated message brokers in Conductor, which propagate them to the configured topics or queues in the eventing systems.

Steps to enable CDC in Conductor workflow:

Set Up Eventing Integration in Orkes Conductor
Configure CDC Settings in Conductor Workflow
Run Workflow
Verify Changes in the Eventing System

To test this functionality, you can use Orkes Playground, a free developer sandbox.

Step 1: Set Up Eventing Integration in Orkes Conductor

The first step is to integrate with the supported eventing systems in Orkes Conductor. Supported integration includes:

Configuration parameters vary depending on the message broker you’re integrating with. Customize these parameters accordingly, and set up the integration based on the specific message broker you’re using. [Note: You only need to add the integration; no event handler setup is required.]

Once the integration is added, note the sink name, which will be in the format:

Integration Type:Config Name:Queue Name

For example, if using AMQP with a test config called amqp-test and a queue named queue-a, the sink would look like this:

amqp:amqp-test:queue-a

Step 2: Configure CDC Settings in Conductor Workflow

Once the eventing integration is ready, configure your workflow to capture CDC events. Conductor workflows can be created using APIs or from Conductor UI.

To create workflows using Conductor UI,

Go to Definitions > Workflows from the left menu on the Conductor cluster.
Select + Define Workflow and create your workflow with the necessary tasks.

In the workflow JSON, set the following fields:

Set "workflowStatusListenerEnabled" to true.
Set "workflowStatusListenerSink" to the integration sink added in the previous step.

"workflowStatusListenerEnabled": true,
"workflowStatusListenerSink": "amqp:amqp-test:queue-a"

You can also enable it directly in the Conductor UI by switching on the "Workflow Status Listener" in the Workflow tab.

When adding the sink from the UI, the “Workflow listener sink” drop-down field lists only the integrations added to the cluster. The topic or queue name must be added manually. For example, if an AMQP integration is added with the name “amqp-test” the drop-down shows:

Choose the integration and add the queue name to ensure the workflow listener sink is updated as follows:

"workflowStatusListenerSink":"amqp:amqp-test:add-queue-name-here"

Now, save the workflow definition.

Step 3: Run Workflow

After the CDC settings are configured, you're ready to run the workflow. Workflows can be executed in different ways—using SDKs, APIs, or Conductor UI.

To test it out, you can quickly run it from Conductor UI:

Select the Run Workflow button from the left menu.
Choose the workflow name and version.
Select Run Workflow at the top-right corner.
Click the workflow execution ID generated to view the execution details.

Once the execution begins, the details are captured and sent to the configured eventing system upon any workflow state change. To be more precise, an event is triggered when the workflow state transitions from ‘Running’ to any other state.

Step 4: Verify Changes in the Eventing System

The final step is verifying that the configured eventing system reflects the workflow status changes. The settings for each eventing system vary, so ensure that the message is received under the configured topic or queue.

That's it! You've now enabled CDC in Conductor workflow to stream workflow state changes to external systems.

Conclusion

In conclusion, CDC is crucial for maintaining real-time data consistency across microservices by capturing and propagating changes as events. This ensures that all services are synchronized with up-to-date information, enabling low-latency and reducing the need for batch data transfers. CDC is essential for modern event-driven architectures, providing a foundation for scalable, reliable, and efficient data replication.

Orkes Conductor further aids in optimizing event-driven systems by seamlessly enabling CDC on workflows. By allowing real-time workflow state change detection and event propagation, Conductor ensures smooth orchestration between services. This allows developers to leverage the power of CDC for efficient performance in their microservice-based applications. The enterprise-grade Orkes Conductor offering is available as Orkes Cloud.

–

Guide to Prompt Engineering

Riza Farheen — Thu, 27 Jun 2024 10:23:01 +0000

This is part one of a series exploring prompt engineering, what it is, its significance in app development, and how to build LLM-powered applications effectively.

The AI landscape underwent a transformative change with the public release of OpenAI’s GPT-3 model in late 2022, opening up a world of possibilities and sparking widespread experimentation.

One of the most critical aspects of leveraging GPT-3 and similar models lies in the art of creating prompts. Crafting precise and well-defined prompts is essential; a sloppy prompt can lead to irrelevant or meaningless outputs, limiting AI applications' potential. As users navigated the early days of GPT-3 experimentation, they quickly recognized that the key to unlocking the full capabilities of these models lies in writing well-crafted prompts.

This blog explores the fundamentals of prompt engineering and its crucial role in application development and provides insights into interacting with and building LLM-based applications.

What is Prompt Engineering?

Prompt engineering is an emerging engineering discipline defined as the practice of writing inputs for AI tools to produce desirable outputs.

Before delving into the fundamentals of prompt engineering, let’s take a look at what prompts are.

What is a Prompt?

A prompt is an input or question you provide to an AI model like ChatGPT. For example, when you initially used ChatGPT, the questions you asked were your prompts. Sometimes, the initial prompt might not yield the desired result, so you refine it by providing clearer instructions and setting specific expectations. This improved prompt might have helped in getting a better response.

In this context, better inputs to a model can produce better results. Therefore, prompts are the inputs provided to AI models, and prompt engineering is the practice of crafting these prompts effectively.

While prompt engineering is relatively new, its origins can be traced back to the history of Natural Language Processing (NLP). NLPs are subcategories of Artificial Intelligence (AI) that specifically address how computers interact with human language. Within NLP, large language models (LLMs) represent a significant advancement in Generative AI (GenAI). These models are trained on millions and trillions of data, enabling them to generate something new based on given inputs.

Effective prompts are vital in prompt engineering. The prompts guide the GenAI models in creating relevant and accurate responses that align with the user's expectations. This helps users interact with GenAI models more intuitively, creating a smoother experience.

Significance of Prompt Engineering in App Development

AI-based applications are increasingly dominating the market compared to conventional applications as businesses evolve by incorporating AI-powered components into their applications. A critical aspect of these AI-powered applications is their ability to communicate effectively with large language models (LLMs). This communication is facilitated through the creation of effective prompts.

Prompt engineering can be integrated into AI-powered applications for better user interactions:

Prompt engineers design and refine prompts to elicit the best possible outputs from LLMs, minimizing errors and enhancing the reliability of the application.
Acting as a vital link between end users and LLMs, prompt engineers craft prompts that accurately interpret user inputs and generate appropriate responses. This ensures seamless interaction between the application and its users.
Prompt engineers conduct experiments using diverse inputs to construct a prompt library that application developers can utilize in different situations. This library becomes an invaluable resource, providing out-of-box solutions and reducing developers' time on prompt creation.

Next, let's explore how these prompt engineering techniques can be implemented in AI app development.

Implementing Prompt Engineering in App Development

During prompt design, the outputs generated can vary significantly depending on the specific parameters configured for the LLM.

LLM Parameters

Here are the common LLM parameters you may encounter while using different providers.

Temperature

Temperature indicates the randomness of the model’s output. A higher temperature makes the output more random and creative, whereas a lower temperature makes the output more stable and focused. Higher temperatures can be used for generating creative content, such as social media posts or drafting emails. Lower temperatures are better suited for use cases like text classification, which require a more focused output.

TopP

TopP, also known as nucleus sampling, allows the prompt engineer to control the randomness of the model’s output. It defines a probability threshold and selects tokens whose cumulative probability exceeds this threshold.

Max Tokens

Max token determines the maximum number of tokens to be generated by the LLM and returned as a result. This parameter helps prevent long or irrelevant responses and controls costs. In contrast, the Temperature and TopP parameters help define the output's randomness, while they don’t limit its size.

Context Window

Content window determines the number of tokens the model can process as inputs when generating responses. Increasing the context window size enhances the performance of LLMs. For example, GPT-3 handles up to 2,000 tokens, while GPT-4 Turbo extends this to 128,000 tokens, and Llama manages 32,000 tokens. As of the latest update, Google's Gemini 1.5 Pro ships with a default context window size of 128,000 tokens, with plans to allow a selected group of developers and enterprise customers to experiment with a context window of up to 1 million tokens through AI Studio and Vertex AI in a private preview phase.

Stop Sequences or Stop Words

Stop sequences (also known as stop words) help prevent the model from generating content containing specific sequences, such as profanity or sensitive information.

Frequency Penalty and Presence Penalty

Frequency penalty is a parameter used to discourage LLM models from generating frequent words, thereby helping to prevent repetitive text. On the other hand, the presence penalty encourages LLM models from generating words that have not been recently used.

LLM parameters are crucial in creating effective prompts, which are integral to developing AI-based applications. Adjusting these parameters ensures that the LLMs produce the desired outcomes. Therefore, understanding and carefully configuring these parameters is essential for prompt creation.

How to Create Effective Prompts

We have covered some basic LLM parameters. Now, let's look into key considerations for crafting effective prompts.

Understanding context

Before crafting the prompt, it's crucial to clearly understand its purpose. Are you seeking information, encouraging creativity, or solving a specific problem? Understanding this ensures the prompt aligns with your goals.

Writing clear instructions

Clear and precise instructions are key to eliciting useful responses. When creating a prompt:

Provide sufficient context or background information to help the model comprehend the scenario or issue accurately.
Avoid ambiguity by clearly specifying the desired outcome from the response. For example, explicitly state if creativity is preferred.
If applicable, define a persona for the model, outlining character traits or roles for the response.
Outline any necessary steps or criteria the model should follow to complete the task.
Offer examples of the desired outputs whenever possible. This gives the model a concrete reference point, helping produce relevant responses.
Specify the expected length or format of the response, whether it's a brief summary, a detailed analysis, or a specific word count.
Ensure prompts are framed neutrally to avoid suggesting biased answers.

Testing or fine-tuning prompts

Before finalizing, it's important to test the prompt to ensure it effectively guides the models toward the desired outcome. Fine-tuning may be necessary based on initial responses.

Good Prompts Vs. Bad Prompts

Let’s contrast generic examples of good and bad prompts in real-life scenarios:

Example 1 - Recipe Recommendation

The first prompt generated generic breakfast ideas without specific details on ingredients and preparation steps. In contrast, the second prompt included clear instructions, such as preferring Lebanese food, noting an egg allergy, and requesting easy preparation within 30 minutes. As a result, the responses were well crafted to suit these requirements.

Example 2 - Vacation Planning

The first prompt provided a generic plan, while the second offered a detailed itinerary tailored to the specific requirements.

Example 3 - Generating Code

The first prompt is a generic request for CSS code to align elements when building an HTML-based website. In contrast, the second prompt provides specific context, outlines the issues encountered, and requests a resolution.

A good prompt provides clear context, issues, and expectations:

Using Orchestration Tools for Building LLM-powered Applications

Having explored the fundamentals of prompts and best practices for creating them, it's time to dig deeper into how orchestration tools like Orkes Conductor streamline the process of building LLM-powered applications by orchestrating the interaction between distributed components.

Let’s say we have an existing application and want to plug in AI features. This can be achieved using Orkes Conductor through the following steps:

The initial step involves integrating LLM models with Orkes Conductor. Orkes Conductor facilitates integration with various LLM providers and their hosted models, such as OpenAI, Azure Open AI, Vertex AI, Gemini AI, AWS Bedrock, etc. Furthermore, the integration also includes Vector Databases such as Pinecone, Weaviate, MongoDB, etc.

The subsequent step involves creating AI prompts within Orkes Conductor. These prompts can be created and stored directly within the Conductor console. They include an interface for real-time testing, allowing for iterative refinement of prompts before integrating into workflows.

The AI prompts can also be stored as a prompt library in Orkes Conductor, which developers can later leverage when building AI-powered applications.

Once the LLM model integration and prompts are set up, the subsequent step involves creating a workflow using AI agents in Orkes Conductor. These AI agents currently support tasks like text or chat completion, generating embeddings, retrieving embeddings, indexing text/documents, conducting searches within indexes, etc. Depending on the application's specific needs, LLM tasks can be incorporated into the workflow.

The final step is to execute the workflow. This can be initiated through various methods such as code execution, APIs, Conductor UI, schedulers, or Webhook events. Additionally, workflows can be triggered in response to events from external systems like Kafka, SQS, and others.

Summary

In this guide, we've explored the essentials of prompt engineering and its significance in app development. Prompt engineering, as an emerging discipline, plays a critical role in ensuring effective communication between users and AI models. By understanding the basics of prompts, their parameters, and best practices for crafting them, developers can easily create more intuitive and responsive AI-driven applications with Orkes Conductor.

As AI continues to evolve, mastering prompt engineering will become increasingly vital for developers aiming to integrate AI capabilities into their applications. Check out Part 2 of this series, where we dive into experimenting and putting prompt engineering tactics into practice.

–

AI Orchestration Meetup [April 2024] - Recap

Riza Farheen — Mon, 13 May 2024 10:28:21 +0000

With the emergence and fast-growing pace of Generative Artificial Intelligence (GenAI) technologies impacting industries from entertainment to healthcare, recent research suggests that the AI orchestration market size is projected to reach approximately $35 billion by 2031. This growth highlights the increasing need for advanced AI systems capable of effectively handling various AI algorithms, data streams, and workflows. This, in turn, is driving significant progress in different sectors worldwide, marking a transformative shift in the global economy.

So, what exactly is AI orchestration? AI orchestration is the process of managing the interaction, deployment, and integration of various AI models within a workflow or system. These AI models add a human touch to the system when it comes to making complicated decisions. It can help rebalance IT/LOB (Line of business) in decision-making roles, reorganize operational systems, redefine customer acquisition and sales processes, and more.

Yet, with so many AI models entering the market, how can we effectively utilize these models in our applications? Integrating the LLM models and prompts into the application as code by conventional methods eventually creates new monolithic modules that are difficult to deploy, debug, and maintain. However, AI orchestration decouples models, prompts, and vector databases to bring a customized RAG (Retrieval Augmented Generation) framework that can be set up in minutes.

How do you seamlessly integrate AI orchestration into your application? That’s where Orkes, the leading workflow orchestration platform, comes into play. Recently, we organized a developer meet-up to share more about AI orchestration and how to leverage it while building distributed applications. This blog recaps the key takeaways from the meetup and then covers some basic concepts on GenAI, AI orchestration, and its implementation with Orkes Conductor.

Recap from AI Orchestration Bengaluru Meetup

In collaboration with Google for Developers, the Orkes DevRel team hosted the meetup in Bengaluru, India. The event attracted attendees from diverse backgrounds, including software engineering, system administrators, AI research, backend engineering, data science, and more. They represented companies like ANZ, Accenture, HP, IBM, Dell, Wipro, Amazon, Slice, and Gojek.

The meetup featured 3 sessions:

Manan Bhatt (Founding Engineer at Orkes) delivered a session on seamlessly integrating Google Vertex AI with Orkes Conductor to build RAG frameworks.
Riza Farheen (Developer Advocate at Orkes) demonstrated the power of Orkes Conductor's GenAI capabilities through a demo on automating document classification.
Nikhil Rana (Cloud Consultant at Google Cloud) explored the latest updates from Google Gemini, emphasizing its powerful models and features.

A big thank you to Sangeeta Gupta, Community Programs Manager at Orkes, and Srividhya S Subramanian, Solution Architect at Orkes, for their efforts in hosting and networking with attendees.

One of the attendees from the meetup, Prasoon Kumar, quoted:

And here’s what our community shared about the meetup:

#ai #orchestration #distributedapplications #bengaluru #meetup #geminiai… | Riya Negi

✨Exciting Update!✨ I recently had the privilege of attending an enlightening meetup organized by Orkes, focusing on AI Orchestration & Distributed…

linkedin.com

Now, let’s look into some of the basic GenAI and AI Orchestration concepts covered in the meetup.

Understanding AI Orchestration

Before delving into the specifics, let's grasp the significance of AI.

Do you recall the internet buzzing with excitement when ChatGPT was announced in November 2022? During that period, we experimented with creating Conductor workflows using ChatGPT, and the results were amazing.

And now, in May 2024, how often do you use tools like ChatGPT? For instance, finance professionals rely on AI tools to aggregate and summarize financial news, providing timely insights and market trends. Similarly, travelers utilize them to generate personalized travel ideas and itineraries based on their preferences, helping them plan trips efficiently.

Whether it's assisting with research, generating creative content, or aiding decision-making processes, tools like ChatGPT are increasingly integrated into daily processes across diverse sectors. The technology behind tools like ChatGPT is called Generative AI (GenAI).

What is Generative AI (GenAI)?

Generative AI, or GenAI for short, is a type of artificial intelligence capable of producing human-like content from pre-trained models in response to prompts. It can generate content such as text, images, audio, videos, and more.

Given the vast array of AI technologies emerging, why should we specifically focus on GenAI? To understand this, let's examine the key differences between traditional/conventional AI and GenAI.

In conventional AI systems, algorithms are designed to follow a set of predefined rules. In contrast, GenAI is the next generation of artificial intelligence, capable of creating something new from the inputs.

Consider the example of traditional AI, where you play chess with a computer. The computer knows the rules; it can predict the moves and make its own based on a predefined strategy. However, it doesn’t invent new ways to play chess; it chooses from strategies it was programmed with.

With GenAI, you can provide the input, and the AI model generates something new with similar characteristics, surpassing the capabilities of traditional AI models.

What is AI Orchestration & its Benefits?

Do you recall Kevin Scott’s (Microsoft's CTO) keynote at MSBuild 2023? (The Orkes team were also at MSBuild 2023—check out the highlights if interested.) In his keynote, Scott shared the importance of orchestration in building GenAI apps.

Scott remarked: “We really are setting the pace of innovation in the field of AI right now. And I think even for us, it’s been surprising to see how much of the zeitgeist is being captured by things like ChatGPT and applications that people are building on top of these large foundation models.”

Integrating AI models and prompts into the applications by code can result in monolithic modules that are difficult to deploy, debug, and maintain. However, AI orchestration allows easy management of such interaction, deployment, and integration. AI orchestration can decouple models, prompts, and vector databases to create a framework that can be integrated into the application in minutes.

AI orchestration offers a multitude of benefits, including:

Scalability- Makes it easier to scale applications to handle increasing user demands. This is useful in scenarios where AI models need to process a lot of data.
Efficiency - Streamlines the deployment and management of AI components, increasing efficiency in development and operation processes.
Flexibility - Enables the integration of diverse AI models and components, allowing developers to leverage the best tools and algorithms for specific tasks.
Consistency - Provides standardized processes for deploying and managing AI models, helping maintain consistency across different projects and teams.
Optimization - AI orchestration tools often include features for monitoring performance metrics and optimizing resource utilization, which helps identify improvement opportunities.
Cost Savings - Reduce operational costs associated with AI development and deployment by efficiently utilizing resources and automating repetitive tasks. This can he lp companies achieve higher ROI on their AI investments.

Let’s look at a recent Gartner poll from Oct 2023:

One notable highlight from the survey is that software development stands out as the area with the highest adoption or investment rate in GenAI.

Now, the question arises: How can you integrate GenAI capabilities into your application development strategy?That’s where a tool like Orkes Conductor can be leveraged. Orkes is the leading workflow orchestration platform built over the popular open-source Conductor OSS.

With Orkes Conductor introducing AI agents in Nov 2023, it’s now easier to bridge the gap between AI models and distributed applications.

AI Orchestration with Orkes Conductor

Let’s say you've got an existing application and want to add AI capabilities to it.

With Orkes Conductor, it's easy to build a workflow using AI agents and seamlessly integrate it as a new feature into your application.

If you're starting from scratch, Orkes Conductor can be your go-to tool for building the entire application while leveraging its GenAI capabilities.

Now, let’s look at the major steps involved in creating a workflow with AI agents in Orkes Conductor:

Integrate LLM models with Orkes Conductor
Create AI Prompts
Create Workflow including AI Agents
Execute Workflow

Step 1 - Integrating LLM Models with Orkes Conductor

The advancements in GenAI shed new prominence on large language models (LLMs), now equipped with billions and trillions of parameters, opening up a new era in the GenAI landscape.

Orkes Conductor supports integration with the following LLM providers and their hosted models:

For Retrieval Augmented Generation (RAG), you need to integrate Orkes Conductor with vector database providers as well.

Vector databases play a crucial role in RAG by indexing and storing large amounts of data in vector form, enabling efficient retrieval at the required flow within the application. Whether it's answering user queries, providing personalized recommendations, or generating content, the integration of Orkes Conductor with vector database providers enhances the capabilities of AI-powered applications, delivering more accurate and engaging results to users. We currently support integration with providers such as Pinecone and Weaviate.

The added integrations can be shared widely with the required team through role-based access control (RBAC) functionality within Orkes Conductor.

Step 2 - Creating AI Prompts

The next step is to create AI prompts in Orkes Conductor that interact with the integrated LLM models.

Prompts are the instructions provided to the LLM models, and a Prompt Engineer creates them to elicit the most optimal response. With Orkes Conductor, prompt engineers can design prompts, which app developers can utilize while creating their application flows.

Step 3 - Creating Workflows with AI Agents

With LLM model integration and prompts in place, the next step is to create a workflow with the AI agents.

In Orkes Conductor, the workflows can be built using code in your preferred language, JSON, or the Conductor UI. The AI Agents currently supported are:

Depending on the application requirements, the LLM tasks can be added to build our GenAI-powered application.

Step 4 - Executing Workflows

The final step is to execute the workflow. The workflow can be invoked using different methods such as code, APIs, UI, scheduler, and Webhook events or in response to events from external eventing systems such as Kafka, SQS, and more.

And voila! Your GenAI-powered application is up and running. Check our documentation if you are curious about a basic example of a translation workflow covering all these steps.

Summary

We wrapped up a fantastic developer meetup in India on AI Orchestration & Distributed Applications. To check out Orkes Conductor’s GenAI functionality and capabilities, explore our free trial today.

Stay updated on our upcoming events, and if you'd like to participate as a speaker, sponsor, or collaborator at Orkes events, contact us at devrel@orkes.io.

Automating Slack Greetings to Community with Orkes Conductor

Riza Farheen — Wed, 17 Apr 2024 07:36:56 +0000

Imagine a situation where you joined a community channel and felt lonely, not knowing where to start—has it ever happened to you?

What if your community members, too, feel the same about joining your community platforms? It’s a make-or-break moment when it comes to community engagement.

In this blog, we’ll explore how Orkes Conductor, the leading workflow orchestration platform built on the popular open-source Conductor, can help you automate the process of welcoming new community members to your dedicated Slack channel.

Creating a Community Welcome Bot

Before beginning with the bot creation process, you need to ensure that:

You have a Slack workspace.
You have signed up for Orkes Playground, a free developer sandbox from Orkes, to test Conductor in real-time. We’ll be using Playground to test the welcome bot in this blog.

So, here’s a quick overview of how our Conductor community welcome bot will work:

A webhook event is sent to Orkes Conductor whenever a new member joins the community.
On receiving the event, the application flow begins, and the new member gets a message from the designated person (for example, the Head Of Community) welcoming them to the community with relevant greetings.
The workflow waits a few minutes, and a message is sent to help them get started.

And Boom! You’ve automated the welcome greetings to all the newly joining community members. Sounds simple, doesn't it?

As a result, when your community keeps growing, you don’t have to worry about sending personalized messages to every new member.

All you need to do is create an application flow in Orkes Conductor, and Conductor automates this process.

Creating Workflows

Let’s look at the application flow {workflow} now:

View in Playground

You can get the JSON file for the workflow in our GitHub repository.

The workflow begins with a switch task, an operator in Orkes Conductor. In Orkes Conductor, operators are tasks equivalent to common programming language constructs.
In our example, we have added two switch cases evaluating an input parameter named switchCaseValue; one gets executed whenever a new community member joins (the value equals team_join). For all other cases, the default case will be executed. This is a good example of how to validate elementary input data from the workflow executor and logically branch to an error management flow.
The workflow then moves to an HTTP task, yet another operator in Orkes Conductor. It sends a POST request to the Slack Webhook with the user information. Through this task, a message would be posted to the required channel on new members joining.

           "name": "http_task_1",
           "taskReferenceName": "http_task_ref_1",
           "inputParameters": {
             "uri": "https://hooks.slack.com/services/XXXXXX",
             "method": "POST",
             "connectionTimeOut": 3000,
             "readTimeOut": "3000",
             "accept": "application/json",
             "contentType": "application/json",
             "body": {
               "text": "A new user has joined our community slack - ${workflow.input.event.user.profile.display_name}."
             }
           },
           "type": "HTTP",

It is followed by another HTTP task, which is used to welcome the new user to the community.

     "name": "http_task_2",
           "taskReferenceName": "http_task_ref_2",
           "inputParameters": {
             "http_request": {
               "uri": "https://slack.com/api/chat.postMessage",
               "method": "POST",
               "connectionTimeOut": 3000,
               "readTimeOut": "3000",
               "accept": "application/json",
               "contentType": "application/json",
               "headers": {
                 "Authorization": "Bearer ${workflow.secrets.slack_welcome_token}"
               },
               "body": {
                 "channel": "${workflow.input.event.user.id}",
                 "text": ":wave: Hi there!  I would like to welcome you to our DevRel community on Slack.  We're thrilled to have you join our vibrant community! :blush: Whether you're here to collaborate, share knowledge, or simply connect with like-minded individuals, you're in the right place. Take a moment to introduce yourself in the #introduce-yourself channel. Share your interests, what brought you here, or anything else you'd like the community to know! ",
                 "as_user": true
               }
             }
           },
           "type": "HTTP",

Then, the workflow pauses for a specific period and sends another message with tips to get started. The wait period can be configured using a Wait task. Adding a Wait task can be useful to make the flow less “robotic” and give users a natural perception of things happening behind the scenes.

          "name": "wait_for_sometime",
           "taskReferenceName": "wait_for_sometime",
           "inputParameters": {
             "duration": "2 minutes"
           },
           "type": "WAIT",

The final task in the workflow is an HTTP task that sends messages to the user with utility links to help them get started.

Creating Webhook

New user sign-up information is received as an event within Orkes Conductor through Webhooks. Webhooks are used to trigger workflows.

Therefore, create a Webhook by navigating to Definitions > Webhook from the left menu on your Conductor cluster, and set the webhook parameters as shown below:

Provide a Webhook name and choose the Source platform as Slack.
Under the field “Start workflow when webhook event comes”, choose the workflow we’ve created in the previous step.
When saving the webhook, an unverified URL will be generated. It gets verified when this URL is copied to your Slack app.

You might wonder why we did not add the workflow under the field “Workflow to receive webhook event”. In this case, the workflow must be started only after the webhook event is received, i.e., whenever a new community member joins.

Creating Slack App

The next step is to create a Slack app.

Navigate to Slack Apps API and click Create an App.

Choose the option From scratch and provide an app name and the Slack workspace to which the application will be integrated. Click Create App.

Next, activate the incoming webhooks by navigating to Features > Incoming Webhooks. Turn on the option Activate Incoming Webhooks.

Then, from Features > Event Subscriptions, turn on Enable Events.

Under the field Request URL, provide the webhook URL created previously. At this step, the URL becomes verified in your Conductor cluster.
Under Subscribe to bot events, click Add Bot User Event & add team_join (remember, this is the value evaluated by our switch in our workflow).
Save the changes.

Navigate to Features > App Home, and under the Messages tab, enable the option Allow users to send slash commands and messages from the messages tab.

Then, you need to enable permissions for your Slack app by navigating to Features > OAuth & Permissions.

Under Scopes > Bot Token Scopes, add the following permissions to those added by default.
channels:read
chat:write
chat:write.public
groups:history
groups:read
im:history
mpim:history
mpim:read
Under Scopes > User Token Scopes, add the following permissions.
channels:history
channels:read
chat:write
groups:history
groups:read
im:history
mpim:read
users:read

Install App to Slack Workspace

Now that you have created your Slack app let’s install the app:

For this:

Navigate to Settings > Install App, and click Install to Workspace.
Under “Community Welcome App for Slack requires a channel to post to as an app”, choose the channel you want to notify when a member joins the workspace.

Once the app is installed, OAuth tokens will be generated.

To try this workflow in your Slack workspace, you need to make some changes to the following configurations in the sample workflow.

Create a secret in Orkes Conductor from the left menu.
Provide a name for your secret and save the User OAuth token generated (see above image) when the app was installed in your workspace. In this example, we have saved the secret as slack_welcome_token and used it in the workflow definition, as shown below:

"headers": {
                 "Authorization": "Bearer ${workflow.secrets.slack_welcome_token}"
               },

Note: Ensure you replace the slack_welcome_token in your workflow definition with the secret name you created.

From your Slack APIs, copy the Webhook URL (refer to the above image) and replace the POST URL in the first HTTP task as shown below:

Now save the workflow definition.

Executing Community Bot App

Whenever a person joins your Slack community, the webhook event will be received, and the workflow will be triggered.

From the newly-joined Community User perspective:

The user will receive a welcome message from the Workspace admin (the one who created the Slack app).

You can customize the message according to your needs. These messages are sent using the HTTP task.

Here’s the welcome message received by the user.

And it can be changed as needed by updating the “text” in http_task_2.

               "body": {
                 "channel": "${workflow.input.event.user.id}",
                 "text": ":wave: Hi there!  I would like to welcome you to our DevRel community on Slack.  We're thrilled to have you join our vibrant community! :blush: Whether you're here to collaborate, share knowledge, or simply connect with like-minded individuals, you're in the right place. Take a moment to introduce yourself in the #introduce-yourself channel. Share your interests, what brought you here, or anything else you'd like the community to know! ",
                 "as_user": true
               }

After this, we have added a wait period of 2 minutes using the wait task. So, the user will receive the second message after the wait period like this:

It can be updated as needed by changing the “text” in http_task_3.

"body": {
                 "channel": "${workflow.input.event.user.id}",
                 "text": "Welcome aboard! Here's a link to get started: [https://orkes.io/content/getting-started/first-workflow-application]. These will help you get the most out of your experience here. Feel free to ping us for any help. ",
                 "as_user": true
               }

From the Slack Admin perspective:

Once a new community user has joined, a message will be posted on the slack-community-join-updates channel (which is the channel we chose to install the application on).

The message can be configured via the http_task_1, where you can update the “text” like this:

"body": {
               "text": "A new user has joined our community slack - ${workflow.input.event.user.profile.display_name}."
             }

This ensures that the admins are notified when someone joins.

Tip: Consider posting these updates in a private channel, as they contain details other community members may not need to know.

Now, you can check on Orkes Conductor to see that the workflow has been completed successfully.

“That’s how we welcome new members to our community Slack channel at Orkes. Join and witness the magic for yourself now.”

This is a simple example with just a few tasks. Of course, you can customize this app to fit your needs, adding additional tasks and changing the flow as per your requirements.

Here’s a video guide summarizing the example we’ve discussed:

youtube.com

Summing Up

We have used our own platform for welcoming our community members & many other use cases like:

Meanwhile, don’t forget to give us a 🌟on our Conductor OSS repo!

Conductor is an enterprise-grade Unified Application Platform for process automation, API and microservices orchestration, agentic workflows, and more. Check out the full set of features, or try it yourself using our free Developer Playground.

What’s new in Conductor Python SDK?

Riza Farheen — Mon, 12 Feb 2024 06:20:36 +0000

The Conductor Python SDK has undergone a major update, including new feature additions. 🎉

If you want to try it out quickly, pip install with the following:

python3 -m pip install conductor-python

The SDK needs Python 3.9+ to work with.

Here’s the quick link to Python SDK - https://github.com/conductor-sdk/conductor-python.

If you are curious to explore the new enhancements in the Python SDK, read on!

Key Highlights

Support for AI Orchestration with language models, prompt templates and vector databases

Recently, we introduced support for building Gen AI applications by integrating with popular AI/LLM models and vector databases, & prompt templates to work with Orkes AI Agents. These features are now available in Conductor Python SDK.

Check out the AI SDKs here.

For example, here’s a snippet from an AI example:

@worker_task(task_definition_name='get_friends_name')
def get_friend_name():
    name = os.getlogin()
    if name is None:
        return 'anonymous'
    else:
        return name


def start_workers(api_config):
    task_handler = TaskHandler(
        workers=[],
        configuration=api_config,
        scan_for_annotated_workers=True,
    )
    task_handler.start_processes()
    return task_handler

Get the complete code here.

We currently support integrations with LLM providers such as OpenAI, Azure OpenAI, Vertex AI & Hugging Face, and vector DB’s such as Pinecone & Weaviate.

This is just the beginning. Orkes team is working on adding integrations with more LLM providers & vector DBs. So, keep an eye out for our upcoming announcements.

Improved way to create dynamic workflows in Python

By using the improved ConductorWorkflow class and its associated methods, you can easily create complex workflows for specific use cases, enhancing the scalability of Python-based automation projects.

from conductor.client.worker.worker_task import worker_task


@worker_task(task_definition_name='get_user_email')
def get_user_email(userid: str) -> str:
    return f'{userid}@example.com'


@worker_task(task_definition_name='send_email')
def send_email(email: str, subject: str, body: str):
    print(f'sending email to {email} with subject {subject} and body {body}')

Get the complete code here.

Support to Unit Test Workflows & Replay Workflows

Introduced support for unit testing workflows, ensuring reliability and robustness throughout the development process and enhancing the SDK's quality and stability.

Also, enabled the ability to replay workflows, facilitating debugging, analysis, and optimization, providing you with greater visibility and control over the workflows.

Check out here.

Quality of Life Improvements

We have incorporated numerous quality-of-life improvements, such as dedicated clients for various resource types, worker configuration flexibility, enhanced documentation, performance optimizations, and out-of-box examples, enhancing the overall developer experience.

Out-of-Box Examples

We have made it easier by adding many ready to use examples, which you can test here.👇

Python SDK Examples

Try it out & feel free to contribute more to Conductor OSS.

Get Started with Python SDK

If you want to get started with Python SDK, check out these steps:

Install SDK - Create a virtual environment to build your package:

virtualenv conductor
source conductor/bin/activate

Get Conductor Python SDK - If you want to try it out quickly, pip install with the following:

python3 -m pip install conductor-python

The SDK needs Python 3.9+ to work with.

Setup SDK - Point the SDK to the Conductor Server API endpoint.

export CONDUCTOR_SERVER_URL=http://localhost:8080/api

(Optionally) If you are using a Conductor server that requires authentication. Obtain the key and secret from the conductor server.

export CONDUCTOR_AUTH_KEY=your_key
export CONDUCTOR_AUTH_SECRET=your_key_secret

Start Conductor Server

docker run --init -p 8080:8080 -p 5000:5000 conductoross/conductor-standalone:3.15.0

After starting the server navigate to http://localhost:5000 to ensure the server has started successfully.

Happy Coding! 🎉

Meanwhile, don’t forget to follow the Conductor OSS - the open-source project & support us by giving a ⭐.

Join our community Slack if you have any queries.

Orkes Monthly Highlights - January 2024

Riza Farheen — Wed, 31 Jan 2024 11:22:42 +0000

Welcome to the first edition of 2024 Orkes Monthly Highlights.

We look forward to an exciting 2024 with more meetups, conferences, and major product updates. As we begin a new year, we thank our ever-growing community of developers for all your contributions and support.

Check out our Conductor OSS repository, give a star 🌟, fork & start contributing!

Let’s have a look into the happenings of January 2024 and a deep dive into the exciting event updates for February 2024.

Community Updates

Release of Conductor OSS

After taking full stewardship of Conductor OSS, Orkes achieved a significant milestone with the initial release of Conductor OSS (version 3.16.0), marking a pivotal moment in our open-source strategy.

Startups in Conversation: Building with AI - Microsoft for Startups

Viren Baraiya joined as a panelist at “Startups in Conversation: Building with AI” at Microsoft for Startups, where he delved into the latest AI-based Orchestration launched in Orkes Conductor.

Open Source Startup Podcast

Jeu George (CEO, Orkes) was featured in the Open Source Startup Podcast - E122: How Orkes is Building a Great Company on Conductor. In this podcast, Jeu takes you behind the scenes of the Conductor's journey, from inception at Netflix to its journey to open-source and formation of Orkes.

Tune In

Join our Upcoming Events

AWS & Orkes Developer Meetup | Bay Area

Feb 7, 2024: San Francisco, US

Our first Bay Area Meetup, in collaboration with AWS, is happening on Feb 7, 2024, from 01:30 PM to 04:30 PM at 475 Sansome Street, 10th Floor, San Francisco, CA 94111. We're thrilled to be surrounded by fellow developers, software architects, and tech enthusiasts. We’ll delve into modern distributed applications with Conductor, the leading open-source orchestration platform. We have an exciting lineup of speakers, including a few prominent industry experts. Don't miss out on this exciting opportunity!

Reserve your spot

AWS & Orkes Developer Meetup | Seattle

Feb 21, 2024: Seattle, US

Join us for our first in-person meetup in Seattle, in collaboration with AWS, on Feb 21, 2024, from 01:30 - 03:30 PM at AWS, Seattle. Dive into how Conductor powers contemporary distributed applications with informative sessions on implementing resilient application executions and creating GenAI-driven solutions using Conductor and AWS. Don't let this opportunity pass you by!

Block your slot now

AI Webcast | Build and Scale RAG Applications using Orkes Gen AI Orchestration

Feb 8, 2024: Virtual

Join us for our first technical webinar of the year on Feb 8, 2024, at 09:00 AM PT. Discover the potential of Gen AI Orchestration with Orkes Conductor as we explore application development. Learn directly from the Orkes CTO and the creator of Netflix Conductor on how to integrate Conductor with AI/LLM models, vector databases, and effective prompt engineering techniques. Secure your spot now for this invaluable opportunity!

Conductor Virtual Meetup | Feb’24

Feb 13, 2024: Virtual

The February edition of the Conductor Virtual Meetup is scheduled for Feb 13, 2024, from 09:00 AM - 10:30 AM PT. This meetup is an opportunity to dive deep into the potential of Conductor, uncover the strategies for scalable application development, get insights from industry experts leveraging Conductor, and engage in vibrant discussions with fellow tech enthusiasts. See you there!

Reserve your spot

Vue.js Nation Workshop

Feb 15, 2024: Virtual

Secure your spot for an exciting two-hour virtual workshop at Vue.js Nation Conference on Feb 15, 2024, starting at 10:00 AM ET. Join Olivier Poupeney, Head of DevRel at Orkes, as he delves into the intricacies of building stateful BFF (Back-end for front-end) using Conductor, the leading open-source orchestration platform, and gain invaluable insights to enhance your front-end development journey with Conductor.

Count me in

dojo.live Tech Talk

Feb 6, 2024: Virtual

Join Orkes CTO and co-founder Viren Baraiya at dojo.live on Feb 6, 2024, at 10:00 AM PT for an exclusive session on the incredible potential of integrating Gen AI capabilities into your applications!

FOSS Overflow

Dec 16, 2023 - Feb 10, 2024: Bhilai, India

Join us at FOSS Overflow, a month-long hackathon from Dec 16, 2023 - Feb 10, 2024, at IIT Bhilai, India, organized by OpenLake and GDSC IIT Bhilai. This program aims to introduce students to the realm of open-source by guiding them through creating real-world open-source projects with the support of experienced mentors.

Hack This Fall Hackathon

Feb 9-11, 2024: Gujarat, India

This February 2024, we are excited to partner with Hack This Fall Hackathon Season 4 for a 36-hour in-person coding challenge from Feb 9-11, 2024, at Karnavati University BBA New campus, Gandhinagar, Gujarat, India.

This hackathon aims to foster a beginner-friendly environment where people of diverse backgrounds can collaborate and create meaningful projects with Orkes Conductor. Orkes team will be there to help you kickstart and build exciting projects with Conductor.

Check out now

Recap of January 2024 Events

Virtual Conductor Meetup Jan’24

Jan 9, 2024: Virtual

We began 2024 with the Virtual Conductor Meetup on Jan 9, 2024. We were fortunate to have Viren Baraiya, CTO & Co-Founder of Orkes & Olivier Poupeney, Head of DevRel, as the speakers.

The meetup began with Olivier’s talk on leveraging Conductor as an integration platform for building large-scale distributed applications.

Using Conductor as an Integration Platform | Virtual Conductor Meetup - Jan'24 - YouTube

In this video, our Head of DevRel, Olivier Poupeney, shares his insights on how you can leverage Conductor as an integration platform for your large scale ap...

youtube.com

It was followed by Viren’s talk on the importance of application flows' durability for conquering modern distributed systems' development.

Durable Executions - Why Does it Matter for Applications | Virtual Conductor Meetup-Jan'24 - YouTube

In this video, our CTO and creator of Netflix Conductor, Viren Baraiya shares insights on the importance of durability of application flows for distributed s...

youtube.com

FOSS United Bangalore

Jan 13, 2024: Bengaluru, India

We had an incredible time at the FOSS (Free and Open Source Software) United Bangalore, a gathering of free, open-source software developers, enthusiasts, and communities. Manan Bhatt, Founding Engineer at Orkes, shared an insightful session on the fascinating realm of orchestration in Retrieval Augmented Generation (RAG) with Orkes Conductor.

Orkes x TechHacks

Jan 19-21, 2024: Punjab, India

January was an electrifying month with the TechHacks hackathon, held from Jan 19-21, 2024, in Punjab, India. This gathering united many innovative minds and witnessed terrific projects built over Conductor. Additionally, we had Altaf Alam Ansari, Software Engineer at Orkes, as an in-person mentor at the event.

During workshops, mentorship sessions, and networking events, participants engaged in vibrant discussions, fostering an environment rich in creativity and innovation.
Here's the feedback shared by our wonderful community about the event:

Vue.js Nation Conference 2024

Jan 24-25, 2024: Virtual

At the Vue.js Nation Conference 2024, we made a significant impression with our "Accelerating front-end development with stateful BFFs" session led by Olivier Poupeney, Head of DevRel at Orkes. Featuring Conductor, the leading open-source orchestration platform, the talk showcased Conductor's transformative potential, encouraging attendees to reimagine their methodologies and cultivate a unified outlook for the future of front-end development with Conductor.

Microservices & Distributed Applications Meetup | Boston

Jan 30, 2024: Boston, US

In partnership with GDG Boston, we organized the Microservices & Distributed Applications Meetup in Boston, offering a valuable opportunity to dive into the practical aspects of building distributed applications using Conductor.

Abhishek Gupta, Software Engineering Lead at Orkes, delivered a comprehensive session on developing distributed applications with the Python SDK, catering to Python enthusiasts and opening doors for Conductor SDK exploration.

Olivier Poupeney, Head of DevRel at Orkes, shared insightful strategies for integrating orchestration capabilities into applications using Conductor.

Announcing Initial Release of New Conductor OSS Repository

Riza Farheen — Mon, 22 Jan 2024 09:16:46 +0000

Authored by Olivier Poupeney

This week, Orkes announced the availability of the initial release of Conductor OSS. This is the inaugural release since we took ownership of the future and direction of Conductor’s community version.

This first release (tagged as v3.16.0) of Conductor OSS marks a significant milestone in Orkes’s Open Source strategy. Member of CNCF, Linux Foundation, and more recently FINOS, having a seat at the FINOS’s technical oversight committee, we aim to not only nurture and grow the Conductor community but also interconnect it with complementary communities in order to empower open source-based innovations where Conductor can be a key architectural component.

The enthusiasm from developers around the world to contribute to this release and participate in the contribution effort demonstrates the strength of Conductor to become the leading Open-Source application orchestration platform.

The takeaways of this release are detailed below:

Updates in v3.16.0

Features & Enhancements

Spring boo3 upgrade by @LuisLainez in #10.
Permissive Task Capability by @ivakoleva in #3: Introduced permissive task capability.
Added the task update API by reference name by @boney9 in #15.

Bug Fixes

Bug fix by @math29 in #4: Resolved an issue where the wrong unit was used in SetVariable error messages.
Typo Fix by @Young-Zen in #6: Corrected a typo in WorkflowTaskTypeConstraint.
Security Update by @scottcarter87 in #5: Addressed and resolved open vulnerabilities in Conductor.
Dependency Error Fix by @Young-Zen in #11: Fixed an Elasticsearch dependency error.
Bug fix by @haricane8133 in #7.

Build and Testing

Classifier Update by @s50600822 in #23: Updated the classifier in alignment with Gradle 5.1 changes.

Contributors - Our Community Stars

We extend a heartfelt thank you to our newest contributors, who have played a pivotal role in shaping this release:

@LuisLainez - Luis Miguel Lainez Reche, Senior Software Engineer at Atlassian
@math29 - Mathieu Allain, Software Engineer at Amadeus
@s50600822 - Hoa Phan, Software Developer
@Young-Zen
@scottcarter87
@ivakoleva - Broadcom
@haricane8133 - Hari Rajesh, SDE at PayPal
@boney9 - Boney Sekh
@c4lm - Vasily Pankov, Senior Software Engineer at Orkes
@saksham2105 - Saksham Solanki, Software Engineer at Orkes

Your dedication and hard work are truly appreciated. 🙌 Together, let's continue to push the boundaries and shape the future of Conductor.

Show Your Support

We invite you to express your support by starring our Conductor OSS repository on GitHub. Your stars fuel our motivation to innovate and grow.

🌟Star, Fork & Start Contributing

Happy Coding!

As we celebrate this milestone, we're eager to see the incredible projects and innovations that will be powered by Conductor OSS. Happy coding to all our contributors, users, and the entire Conductor community!

To preview what's coming in the subsequent releases, check out our Conductor Roadmap.

Orkes Monthly Highlights - December 2023

Riza Farheen — Fri, 22 Dec 2023 11:58:48 +0000

Welcome to the December edition of Orkes Monthly Highlights.

December closes the 2023 chapter as a very intense year for the Conductor developer community. Contributions, hackathons, and participation in numerous developer conferences, we have been delighted to meet and exchange with developers throughout the year. As today Conductor starts a new era with its new home, the community is more active and engaged than ever.

If not done yet, Star 🌟 the new repo and show your support to Conductor and Open Source!

Let's quickly dive into the events from December 2023 and get an exclusive glimpse of what January 2024 has in store.

Community Updates

A New Home for Conductor OSS

We have some exciting updates to our Conductor Community regarding the future of Conductor OSS.

Starting December 13, 2023, Orkes assumed a more prominent role in stewarding the Conductor OSS project. This transition comes as Netflix refocuses its efforts on its internal Conductor fork. Read our official announcement blog for more details.

Conductor OSS now has a new home, and Orkes will continue facilitating collaboration across the active open-source community. Send PRs directly to this repo and check the contribution guidelines of Conductor.

Recap of December 2023 Events

APIDays Paris

Dec 7, 2023: Paris, France

We had an incredible time at the APIDays Paris on Dec 7, 2023. Our Head of DevRel, Olivier Poupeney, delivered an insightful session on the groundbreaking impact of Retrieval Augmented Generation (RAG) across diverse applications.

The session delved into a practical demonstration showcasing the application of RAG in automating market data within the financial services industry. Olivier provided a live demo highlighting the latest advancements in Gen AI capabilities integrated into Orkes Conductor.

In case you missed it, the recording is now available.

Microservices and Orkes Conductor Meetup Paris

Dec 8, 2023: Paris, France

The second edition of the Microservices and Orkes Conductor Meetup in Paris occurred on Dec 8, 2023. The event showcased insightful sessions by industry experts, offering attendees valuable opportunities to network and learn.

Featured speakers included Daniel Phiri from Weaviate, Clément Aubert from Product Live, and Olivier Poupeney from Orkes. The meetup was a blend of fun and information, fostering connections and engagement within the Paris tech community.

A heartfelt thank you to everyone who joined us. Your active participation made it an exceptional event filled with enlightening discussions, groundbreaking insights, and unparalleled networking opportunities.

Orkes x Netflix | Conductor Meetup

Dec 13, 2023: Virtual

Co-organized by Orkes and Netflix Engineering, the Conductor Meetup for Q4 was hosted virtually on Dec 13, 2023. We were privileged to have Ankit Agarwal from Fynarfin, & Viren Baraiya, and Olivier Poupeney from Orkes as featured speakers.

The meetup started with Viren’s talk on building event-driven distributed applications with Conductor.

It was followed by Ankit’s talk on how Mifos implemented a Digital Payment Goods (DPG) Compliant for orchestrating their microservice-driven Payment Hub.

Finally, the meetup concluded with Olivier’s session on AI-enabled Orchestration and Retrieval Augmented Generation (RAG).

Join our Upcoming Events

Virtual Conductor Meetup

Jan 9, 2024: Virtual

The first virtual meetup of the year is scheduled for Jan 9, 2024, from 09:00 AM - 11:00 AM PT. This meetup is an opportunity to dive deep into the potential of Conductor. We have a great panel of industry experts lined up to share their expertise and knowledge on how they are using Conductor to build their scalable, resilient, and distributed platforms.

Register now

FOSS Overflow

Dec 16, 2023 - Feb 10, 2024: Bhilai, India

Join us at FOSS Overflow, a month-long hackathon from Dec 16, 2023 - Feb 10, 2024, at IIT Bhilai, India, organized by OpenLake and GDSC IIT Bhilai. This program aims to introduce students to the realm of open-source by guiding them through the creation of real-world open-source projects with the support of experienced mentors.

Register now

dojo.live Tech Talk

Jan 10, 2024: Virtual

Join our CTO and co-founder, Viren Baraiya, at dojo.live on Jan 10, 2024, at 10:00 AM PT for an exclusive session on the incredible potential of integrating Gen AI capabilities into your applications!

Register now

TechHacks Hackathon

Jan 19-21, 2024: Punjab, India

Orkes is teaming up with Techacks 4.0 for an amazing hackathon from Jan 19 - 21, 2024, at Chitkara University, Punjab, India. Engage with a vibrant community of hackers, collaborate on innovative projects, and unleash your creativity in a supportive and inspiring environment. Don't miss this opportunity to connect, learn, and create something extraordinary.

Register now

HackHiveHackathon

Jan 19-21, 2024: Indore, India

In January 2024, be part of an exciting 36-hour in-person hackathon, HackHive, taking place from Jan 19-21 in Indore, India. HackHive is organized by TechHunterssss, a community fostering unity among tech enthusiasts. Its goal is to create a space where individuals passionate about technology can converge, learn, network, interact, and share experiences, cultivating their tech interests in a community-driven environment.

Register now

HackNITR Hackathon

Jan 27, 2024: Odisha, India

We are excited to partner with HackNITR 5.0, one of the largest student-run hackathons in India. Hop on the bandwagon and join us on Jan 27, 2024, at NIT Rourkela, Odisha, India. It’s an incredible opportunity put forward by tech enthusiasts from the Google Developer Student Club (GDSC) - NIT Rourkela, and OpenCode. Let your creativity soar, collaborate, and craft innovative solutions for an unforgettable experience. Exciting prizes await - reserve your spot now.

Register now

Hack This Fall Hackathon

Feb 9-11, 2024: Gujarat, India

This February 2024, we are excited to partner with Hack This Fall Hackathon Season 4 for a 36-hour in-person coding challenge from Feb 9-11, 2024 at Karnavati University BBA New campus, Gandhinagar, Gujarat, India.

Register now

Latest Resources

Integrating Orkes Conductor with SQS

Recent Blog Posts

Opsgenie Alerting with Orkes Conductor

In the realm of SaaS and SLAs, maintaining a solid commitment to stringent uptime standards is critical. For instance, ensuring a mandated 98% application uptime requires swift detection and mitigation of downtime. Orkes Conductor, coupled with Opsgenie, swiftly addresses these needs, ensuring adherence to SLA commitments by promptly detecting and resolving application downtime and strengthening the provider-client relationship.

Read more

Community Spotlight

Using Conductor OSS for Orchestrating DPG Compliant Payment Hub - Ankit Agarwal

This month, we would like to spotlight one of our community members, Ankit Agarwal from OpenMF, on leveraging Conductor OSS (formerly Netflix Conductor) to orchestrate the Payment Hub to be Digital Public Goods (DPG) Compliant. Read the article to learn how OpenMF opted for Conductor OSS (formerly Netflix Conductor) to orchestrate their microservice-based Payment Hub.

Read now

Like Ankit, if you would like to share your innovation based on Conductor with the community, please reach out to us at devrel@orkes.io.

Wishing everyone joyous holidays filled with warmth and cheer!!!

DEV Community: Riza Farheen

Building an Agentic Workflow: Orchestrating a Multi-Step Software Engineering Interview

Example: Agentic interview app

Building an agentic interview app

Step 1: Clone the repository

Step 2: Get credentials and set up environment variables

Access key and secret from Conductor

OpenAI API key

Google authentication credentials

SendGrid API key

Step 3: Start the backend server

Step 4: Start the frontend server

Step 5: View the interview app in action

What next?

Wrapping up

SendGrid Webhooks for Email Nurturing Workflows: A Step-by-Step Guide

Building an email nurturing flow

Step 1: Create an email nurturing workflow

Step 2: Get the API key from the SendGrid account

Step 3: Store the SendGrid API key as a Secret in Conductor

Step 4: Create a webhook in Conductor

Step 5: Configure an event webhook in SendGrid

Step 6: Modify and run Conductor workflow

Step 7: Create a Single Send in SendGrid

Step 8: Automate using Workflow Scheduler

Summary

Ensuring 99.999% Uptime: Why Service Uptime Monitoring is Crucial

What is service uptime monitoring?

Why use service uptime monitoring?

Building a service uptime monitoring with Orkes Conductor

Service uptime monitoring flow

Part 1: Availability monitoring

Part 2: Notifying users

Part 3: Automating the monitoring process

Summary

Build an AI Application for Document Classification: A Step-by-Step Guide

Building a document classifier application flow

Step 1: Create a document classifier workflow

Step 2: Set up OCR worker

Step 3: Add your preferred LLM integration

Step 4: Create a prompt template using Orkes’s AI prompt builder

Step 5: Edit workflow with your LLM integration and AI prompt

Change Data Capture (CDC) in Event-Driven Microservices

What is Change Data Capture (CDC)?

CDC in Event-driven Microservices

CDC with Orkes Conductor

Step 1: Set Up Eventing Integration in Orkes Conductor

Step 2: Configure CDC Settings in Conductor Workflow

Step 3: Run Workflow

Step 4: Verify Changes in the Eventing System​

Conclusion

Guide to Prompt Engineering

What is Prompt Engineering?

What is a Prompt?

Significance of Prompt Engineering in App Development

Implementing Prompt Engineering in App Development

LLM Parameters

How to Create Effective Prompts

Good Prompts Vs. Bad Prompts

Example 1 - Recipe Recommendation

Example 2 - Vacation Planning

Example 3 - Generating Code

Using Orchestration Tools for Building LLM-powered Applications

Summary

AI Orchestration Meetup [April 2024] - Recap

Recap from AI Orchestration Bengaluru Meetup

#ai #orchestration #distributedapplications #bengaluru #meetup #geminiai… | Riya Negi

Understanding AI Orchestration

What is Generative AI (GenAI)?

What is AI Orchestration & its Benefits?

AI Orchestration with Orkes Conductor

Step 1 - Integrating LLM Models with Orkes Conductor

Step 2 - Creating AI Prompts

Step 3 - Creating Workflows with AI Agents

Step 4 - Executing Workflows

Summary

Automating Slack Greetings to Community with Orkes Conductor

Creating a Community Welcome Bot

Creating Workflows

Creating Webhook

Step 4: Verify Changes in the Eventing System