Most LLM demos assume the process stays alive. Kill it midway and the plan, the tool results, and the context all go with it. I've been spending time with a few tools that change that, and this is what I've picked up.
Durable execution changes how you think about failure
Temporal records every step of a workflow in an event history. If a worker dies, another one replays that history and continues from the last completed step. Finished work doesn't run twice.
For LLM workflows this maps well:
- Model and tool calls belong in activities. They're slow, non-deterministic, and can fail, which is what activities are for.
- Workflow code has to stay deterministic. Replay depends on it, so randomness, network calls, and timestamps stay out of it.
- Retries become policy, not code. Backoff and timeouts are configuration instead of nested try/catch blocks.
Local models make failure cheap to test
Running a model through Ollama keeps the loop on your own machine. No rate limits, no API bill, and killing a process mid-run costs nothing. That makes it easy to test the unhappy path often, which is where most of the real bugs are.
The trade-off is that small local models are less predictable than hosted ones. Durable execution helps here because a bad step can be retried or inspected without redoing the whole run.
You can't trust what you can't see
Retries and resumes are invisible by default, and invisible failures are the hardest to debug. Sentry's tracing is a good fit, since a trace per step shows which call was slow, which one retried, and where a run picked back up.
A small React dashboard on top of the workflow history makes this easier still. Seeing a run's attempts as a timeline beats reading logs.
Open questions I'm still working through
- How much state can sit in workflow history before replay gets slow?
- Which failures should resume, and which should stop the run?
- Does durability change how you write prompts and tool definitions?
I'll share more as I find answers.
Top comments (0)