A process starts a multi-step job. Step 1 validates the customer. Step 2 creates an internal record. Step 3 moves money, changes a subscription, sends an irreversible request to a partner, or does whatever your business considers serious. Then the process dies.
Not politely.
The container gets killed. The deploy rolls. The host disappears. The worker restarts halfway through the thing it was doing.
Now the question is simple and horrible:
What happens when it runs again?
If step 3 already happened, restarting from step 1 may duplicate something expensive. If you skip too much, you may leave the customer in a half-activated state. If you retry blindly, the external provider may accept the same request twice.
This is the shape of a lot of fintech work. Account opening, subscription activation, credit disbursement, repayment settlement, KYC refreshes, scheduled sweeps, ledger posting. They are not one database transaction. They are small business stories pretending to be function calls.
the old answer was discipline
At a fintech on the American continent, the public architecture story was event-driven: Kafka, services, async processing, consistency through events, and careful thinking around idempotency. That style makes sense in a bank-sized environment.
But the boring part of the work was never just "publish an event." The real work was deciding what each step meant, which message was the source of truth, how to identify duplicates, how to make consumers safe to replay, and how to compensate for something that had already crossed a boundary.
You ended up with sagas, state machines, idempotency keys, outbox tables, retry topics, reconciliation jobs, audit trails, dashboards, alerts, and runbooks.
That is not a criticism. In serious financial systems, this discipline is part of the job. You cannot hand-wave around money by saying "the queue is eventually consistent" and then go for lunch.
But there is an operational cost to owning the mechanism. Every team has to remember the same failure modes. Every new workflow has to rebuild the same scaffolding. Every incident asks: did the step finish, did the event publish, did the retry duplicate it, and is the customer state now real?
The painful thing is that most of this machinery is not product-specific.
The compensation logic is product-specific. The step boundaries are product-specific. The decision that a transfer can be retried but a partner call needs an idempotency key is product-specific. But "remember completed step 3 after the worker crashes" is plumbing.
And plumbing eventually becomes a product.
dbos and temporal change the default
This is why frameworks like DBOS and Temporal are worth paying attention to, even if you do not adopt either tomorrow.
Temporal's model is built around workflows and activities. The Temporal service stores a durable event history for each workflow execution. When a worker crashes or restarts, workflow code can be replayed from history, and completed activities are not simply repeated. The service has a record of what happened.
DBOS takes a different product shape. DBOS Transact is positioned as an open-source library that runs inside your application. The current DBOS site still says Transact uses your existing Postgres database to store and recover workflow state and execution history. Their paid model is around DBOS Conductor, tooling, support, and hosting options, with Pro and Teams plans priced around checkpoint usage.
That difference matters.
Temporal asks you to run, or pay for, a workflow service. Temporal Cloud pricing is consumption-based, mostly around actions and storage. The minimum Cloud plan is currently listed at $100/month.
DBOS is making a more Postgres-native bet. The open-source library gives you durable workflows in application code, while Conductor adds operational tooling like monitoring, recovery, versioning, alerts, and support.
The interesting bit is that both package the same formerly internal capability:
This multi-step job survives process death.
That sentence used to imply bespoke platform work. Now it can start with a framework.
the hard part did not disappear
There is a trap here, and it is exactly the trap that appears every time infrastructure gets better.
People confuse "the framework remembers what happened" with "the system now knows what should happen."
It does not.
If a workflow charges a customer, activates a subscription, provisions access, sends an email, and updates analytics, the framework can help make that workflow resumable. It can record step completion, retry activities, show where execution stopped, and make the history visible instead of buried inside logs.
But it cannot tell you whether charging the customer should happen before activation or after activation.
It cannot tell you whether a failed activation should trigger a refund, a retry, a manual review, or a grace period. It cannot tell you whether a partner API is actually idempotent just because it accepts an idempotency key. It cannot tell you whether your "step" is one business action or three business actions wearing a trench coat.
That is still design work.
At a fintech in the Gulf I work with now, this comes up in normal product workflows. A scheduled sweep is not interesting because a timer fired. It is interesting because the system needs to know which accounts were considered, which ones were skipped, which external calls succeeded, and what should happen when the process wakes up after a deploy.
Durable execution helps a lot there. A completed step recorded in Postgres is much better than a log line and a prayer. A workflow console is better than SSH-ing into a worker for clues. But the useful conversation shifts upward.
Instead of asking, "how do we build the retry table?", the team asks, "where is the boundary where retry becomes compensation?" Instead of asking, "how do we resume after a crash?", the team asks, "what does resumed mean for this customer?"
That is a better use of engineering time.
idempotency still lives at the edges
One thing I would be careful about in any durable execution pitch is idempotency.
Inside the framework boundary, you may get very strong guarantees. Completed workflow steps can be recorded. Activity results can be remembered. Retries can be controlled. Execution can resume after a crash.
At the external boundary, reality is messier.
Payment providers, banking partners, email systems, identity vendors, and card processors all have their own ideas about duplicate requests. Some support idempotency well. Some support it in documentation. Some support it until a timeout happens between their side effect and your response.
Boundary calls still need stable request identifiers, provider references, and reconciliation. You still need to know whether a timeout means "nothing happened", "something happened but you did not see it", or "check later."
Durable execution reduces the number of places where that logic leaks.
It does not repeal distributed systems.
the wider pattern
The reason I like this category is that it fits a pattern that keeps repeating in software. At first, a reliability property is craft. A strong team builds it internally because they have no choice. Then enough teams hit the same pain, and the property becomes a product.
Observability did this. Feature flags did this. Secrets management did this. CI/CD did this. Policy-as-code did this. Now durable execution is doing it.
The boring control-plane property becomes something you can buy, install, or outsource.
That is good. I do not miss every team building its own mini workflow engine by accident.
But the job does not vanish. It moves.
The value is no longer proving that your worker can survive a restart. The value is deciding which workflows deserve durability, which steps are atomic from the business point of view, which failures should retry, which failures should compensate, and who owns the customer outcome.
That is where durable execution gets interesting.
Not because DBOS or Temporal make failure disappear.
Because they make the failure visible enough that we can stop pretending the hard part was the retry loop.
references
- DBOS Transact
- DBOS Pricing
- Temporal Event History documentation
- Temporal Cloud pricing documentation
To test my projects, I use Railway. If you want $20 USD to get started, use this link.
Top comments (0)