DEV Community

Cover image for How durable execution actually works: checkpoints and replay
Cully for Resonate HQ

Posted on • Originally published at docs.resonatehq.io

How durable execution actually works: checkpoints and replay

If you've ever wondered how durable execution actually works under the hood: Resonate checkpoints your function's progress as it runs and replays it after a crash — so a process can die mid-execution and resume exactly where it left off, no manual state machine required. This is the model in one page.

Checkpoints and replays

"Durable execution" means an application workflow or function can resume from where it left off after the hosting process has disappeared — crashed, restarted, redeployed.

The most practical way to achieve this, while keeping a coherent and sequential flow to the code the developer writes, is checkpointing and replaying.

Resonate checkpoints at steps in your application code — basically whenever you use a Resonate SDK API. At each checkpoint, Resonate saves some data to a datastore.

In Resonate, the checkpoints are called Durable Promises.

A function may execute many times, using the stored data to reach the next step. The Durable Promise ID acts as a unique identifier for the checkpoint and as an idempotency key for the function execution. When the invoked function completes, the result is stored in the Durable Promise.

Checkpointing in Resonate: each SDK step saves progress to a datastore

As the developer, you decide whether to invoke a local function, invoke a remote function, promisify an external system, and when to await the result — all of which is durable thanks to checkpointing and replaying.

Message passing

Each instance of a Resonate Worker (an application node, a microservice) runs in its own process, and each instance sends and receives messages from a Resonate Server.

Resonate system architecture: workers exchanging messages with a Resonate Server

The message-passing protocol is completely agnostic to the transport. Resonate can pass all its messages over HTTP APIs, or over GCP Pub/Sub topics — and some workers might receive on one transport and send on another.

For the formal treatment of these protocols — promise state transitions, task lifecycle, and message-passing semantics — see the Distributed Async Await specification.

The protocol also gives Resonate built-in service discovery and load balancing. When a worker or microservice connects to a Resonate Server, it registers itself both uniquely and as part of a group. The server tracks which workers are available for a given function and load balances requests across them.

Where to go from here

The evaluate track in the docs continues from here, and the resonatehq-examples org on GitHub has worked examples to run.


Adapted from How Resonate works in the Resonate docs.

Top comments (0)