Workflow orchestration means that one component owns the sequence of steps, the state and the failures of a multi-step process. That component is the orchestrator. For a business process inside one .NET product, run the orchestrator as a library in your app. Keep its state in your own database. Data teams use the same term for a second job. For them, the term means running data pipelines on tools such as Apache Airflow.
Mike Lukinov, co-founder of Optimajet, wrote What Is Workflow Orchestration. The post gives a rule for where to run the orchestrator. If the process belongs to one product and its data should stay in that product's database, put the orchestrator inside the product. The post also says when a separate server or a cloud service is the better place.
Workflow Engine by Optimajet is a .NET workflow engine that runs as a library inside your app. The post compares Workflow Engine with Temporal, Camunda 8, AWS Step Functions and the data pipeline tools. It also lists the vendor pages used for these comparisons, with the date each page was read.
Workflow Engine Free needs no license key. Its license covers personal use and internal commercial use, with attribution. It runs up to 10 workflow definitions (schemes) and 4 execution threads. A public web app or a SaaS product needs the Subscription, SaaS or Enterprise Edition of Workflow Engine NEO. Workflow Engine NEO is a separate product based on Workflow Engine. It adds an HTTP API and multitenancy. The facts about Workflow Engine were checked against version 23.0.0 and its documentation on 27 October 2026.
What an orchestrator does
For a business process inside an application, the orchestrator is software called a workflow engine. Workflow Engine by Optimajet is one product of that kind. Take a purchase request. Someone submits it, and a manager approves it within a week. Then the system places the order through a supplier's API. Each request is a case. A case is one business item that moves through the process. Without an orchestrator, each service calls the next one. The state of the request is spread across several tables. A scheduled job searches for requests that got stuck. With an orchestrator, each request has one record. There is one place to ask where the request is.
An orchestrator has five duties:
- Sequence. It decides which step runs next, and which steps can run in parallel. In Workflow Engine, the sequence lives in the process definition, which is called a scheme. A scheme is an XML file.
- State. It saves the current step and the data of each case between steps. In Workflow Engine, the state is a row in the application's own database, so the case survives a restart.
- Waiting. It pauses a case until a person acts, a timer fires or another system answers. In Workflow Engine, a waiting case holds no execution thread.
- Failure handling. It retries a step, times it out, moves the case to another path, or hands the case to a person. In Workflow Engine, the exception handlers of a step can retry a failed call and then move the case to manual review.
-
History. It records each move, the time of the move, and who or what caused it. In Workflow Engine,
GetProcessHistoryAsyncreturns that history.
Two jobs share the name
Data pipeline orchestration moves datasets on a schedule. Application workflow orchestration runs business cases that wait for people. It is the job of a workflow engine, such as Workflow Engine. The two jobs need different tools.
| Data pipeline orchestration | Application workflow orchestration | |
|---|---|---|
| What moves | Datasets, from source to warehouse | One business case, such as an order, a request or a loan |
| Typical shape | A graph of tasks with no loops (a DAG). It runs on a schedule, and each run takes minutes to hours. | A process instance that can wait days for a person |
| Who acts | Mostly machines | Machines and people |
| When a step fails | Rerun the task, or backfill (start the runs that were missed) | Retry, time out, take another path, or route the case to a person. The case is kept. |
| Typical tools | Apache Airflow, Dagster, Prefect | Workflow engines such as Temporal, Camunda, AWS Step Functions and Workflow Engine by Optimajet |
For a nightly ETL chain (extract, transform, load), pick a data tool. Apache Airflow, Dagster and Prefect define pipelines in Python and are built for that job. Kubernetes uses the word orchestration for a third thing. Container orchestration keeps containers running in the state you declare. It knows nothing about approvals.
What an embedded engine does with waits, deadlines and failures
An embedded engine is a workflow engine that runs as a library inside your application. In Workflow Engine, the engine's object inside the application is WorkflowRuntime. It runs on the application's own database. The purchase request goes from the Draft step to the Review step, where a manager approves or rejects it. Then it goes to the Ordering step, which calls the supplier's API, and then to Ordered.
A restart. If you stop the app while the request waits for approval, the request is still in Review when the app starts again. A waiting request has the status Idled. It holds no execution thread. All it holds is its row in the database. The restarted application is a new operating-system process. It loads the request with the same step, the same deadline and the same two commands, Approve and Reject. A command is what a person executes to move a request to another step.
A deadline. A timer on the Review step can take its value from a process parameter. A process parameter is a value stored with the request. So each request carries its own review window. When the timer fires, a transition moves the request from Review to Escalated. By default, the engine checks timers about once a second. It does this only while WorkflowRuntime is started. If a timer's deadline passes while the app is down, the timer fires after the app starts again.
A failure. A step runs code called an action. The step also declares what happens when its action fails. When the manager executes Approve, the request moves into Ordering. So the supplier call runs inside the Approve command. If that call throws an HttpRequestException, the exception handlers of the Ordering step can retry it twice. After that, they move the request to a step for manual review. Without handlers, the Approve command fails with a ProcessExecutionException. The request stays in Review with the status Error. Nothing is lost, but someone has to go and find the request.
An embedded engine still leaves four things for you to handle:
- A crash in the middle of a step. By default, the engine does not rerun an interrupted step. The process gets the status Error or Terminated. Your recovery policy decides what happens next.
- Side effects. A real order needs an idempotency key. This key is a value that makes a repeated call harmless.
- The next step for a person. Escalation and manual-review steps need commands for the manager. Without them, a case stops there.
- More than one server. To run several servers on one database, you need either Workflow Engine Complete or Workflow Engine NEO. Workflow Engine Complete is a paid Edition of Workflow Engine.
When you need an orchestrator, and when you do not
An orchestrator is one more component to learn and to keep up to date. For small jobs, it costs more than it saves. One job on a schedule needs only a cron job or a hosted service. One unit of work with a retry policy needs only a queue and a background worker. Two services that work together with no waits and no people need only a direct call with a retry policy.
Mike Lukinov's rule of thumb is that you need an orchestrator when one of these is true:
- A step waits for a person, or a process takes days.
- One process touches three or more systems.
- Someone keeps asking where a case is stuck, and nobody can answer quickly.
- The process changes more often than you want to release code.
Where the orchestrator runs
For an application workflow, the orchestrator runs in one of three places. The first is a separate server, such as Temporal. The second is a cloud provider's service, such as AWS Step Functions. The third is a library in your app, such as Workflow Engine by Optimajet. The place you choose decides where the process state lives and what you operate. It also decides what it costs to move away later.
| A separate server | A cloud provider's service | A library in your app | |
|---|---|---|---|
| Examples | Temporal, Camunda 8, Orkes Conductor | AWS Step Functions, Azure Durable Functions, Azure Logic Apps | Workflow Engine by Optimajet, Elsa Workflows, Workflow Core |
| Where state lives | In the server's own storage | In the provider's store. With Durable Functions, in a backend you choose. | In your application's database |
| What you operate | A cluster, its storage and its upgrades, unless you buy the vendor's cloud plan | Nothing extra, within the provider's limits | Your application and its database. The database also holds the engine's tables. They get migrations on major upgrades. Workflow Engine Free runs on one server. |
| How your code talks to it | Through an SDK. Your workers poll the server for tasks. | Through the provider's SDK or API | Through method calls in the same process |
| Cost model | Temporal (MIT) and Conductor OSS (Apache 2.0) are free to self-host. Camunda 8 Self-Managed needs a license in production. Each vendor also sells a cloud plan. | You pay for what you use. Step Functions Standard charges per state transition. Logic Apps Consumption charges per action. | Elsa Workflows and Workflow Core use the MIT license. Workflow Engine Free covers personal and internal commercial use, with attribution. A public web app or a SaaS product needs an Edition of Workflow Engine NEO. There is no fee per execution. |
| Moving away | You rewrite the definitions. Temporal workflows are code written against its SDK. Camunda 8 models are BPMN (a standard process notation) with Camunda's extensions. Conductor definitions use Conductor's own JSON format. | You rewrite the definitions. Step Functions uses the Amazon States Language. Durable Functions workflows are code written against its SDK. | You rewrite the schemes, which are in the engine's own XML format. The process data is already in your database. |
| Fits best | Many services in several languages, or one process that starts thousands of parallel tasks | An application that already lives on one cloud | One product that keeps its own processes and data |
In Mike Lukinov's view, a separate server is worth its operating cost when services in several languages share the same processes, or when one process starts thousands of parallel tasks. That is where Temporal is strong. A cloud service is the fastest way to start when the application already lives on that cloud. Its cost grows with every execution. For one product whose data stays in its own database, the library is the simplest place. The reason is that you add the library to an application you already run.
The post takes its facts about these products from the following pages:
- Temporal: the .NET SDK (Temporalio 1.20.0), workers and license, read on 14 and 27 October 2026.
- Camunda: the licenses, read on 14 October.
- Orkes Conductor: the C# SDK (conductor-csharp 3.0.0), read on 14 October.
- Azure Durable Functions: the storage providers, read on 11 October.
- AWS Step Functions and Azure Logic Apps: the Step Functions pricing and Logic Apps pricing pages, read on 27 October.
- Elsa Workflows 3.9.0, read on 11 October, and Workflow Core 3.21.0, read on 1 October.
The full post also compares orchestration with automation, job scheduling and choreography. It sorts the tools into seven categories, from data pipeline orchestrators to robotic process automation. The categories are in the tools section of What Is Workflow Orchestration.
Top comments (0)