Aturo Phil

Posted on Jul 17

The Sagas Pattern: Coordinating Multi-Step Bitcoin Workflows Without Distributed Transactions

#architecture #bitcoin #distributedsystems #systemdesign

In 1987, Hector Garcia-Molina and Kenneth Salem published a paper called "Sagas" in the ACM SIGMOD Record. The problem they were solving was long-lived database transactions, operations that span seconds or minutes rather than milliseconds, holding locks that block the rest of the system.

Their solution was to break a long transaction into a sequence of smaller, independent local transactions, each with a corresponding compensating transaction that undoes its effect if the sequence needs to abort.

They called this sequence a saga.

Thirty-eight years later, this pattern describes almost exactly what happens when you open a Lightning channel, route a multi-hop payment, or coordinate a submarine swap. Bitcoin and Lightning do not support distributed transactions. They have no rollback mechanism. What they have is time-locked contracts, cooperative protocols, and the ability in failure cases to go on-chain and let the chain settle the state.

A saga is the right mental model for all of this.

The Problem: Multi-Step Operations Without Atomicity

A distributed transaction in a traditional database provides atomicity across multiple systems: either everything commits or everything rolls back. This is achieved through two-phase commit (2PC), which requires a coordinator and a protocol that can hold all participants in a "prepared" state.

Bitcoin has no such mechanism. A Bitcoin transaction is atomic, it either confirms or it does not, but a sequence of Bitcoin-related operations across multiple systems has no coordinator and no rollback. If the third step in a five-step workflow fails, you cannot undo steps one and two by asking the blockchain to forget them.

Consider opening a Lightning channel:

Your node and the peer negotiate parameters (funding amount, fees, CSV delays)
You broadcast a funding transaction to the Bitcoin mempool
The funding transaction confirms (after some number of blocks)
Both nodes exchange funding_locked messages
The channel becomes operational

If step 4 fails, your peer disappears after the funding transaction confirms you have a funded channel that neither party can use cooperatively. You must go on-chain and close it unilaterally. There is no rollback; there is only resolution.

The saga pattern gives you a vocabulary for designing systems that handle this correctly.

Structure of a Saga

A saga consists of:

A sequence of transactions T1, T2, ..., Tn
A corresponding sequence of compensating transactions C1, C2, ..., Cn

If the saga completes T1 through Ti and then Ti+1 fails, it executes compensating transactions Ci, Ci-1, ..., C1 in reverse order to undo what was done.

There are two execution strategies:

Choreography: Each transaction emits events, and subsequent transactions are triggered by those events. There is no central coordinator. This is simpler to implement and has no single point of failure, but it is harder to reason about as the saga grows.

Orchestration: A central coordinator tracks the saga state and issues commands to each participant. Easier to observe and debug, but the coordinator itself becomes a dependency.

For most Bitcoin infrastructure, orchestration is easier to reason about and easier to recover from failure. The coordinator's state can be persisted and replayed.

Modeling a Lightning Channel Open as a Saga

from dataclasses import dataclass
from enum import Enum
from typing import Optional

class ChannelOpenStatus(Enum):
    NEGOTIATING     = "negotiating"
    FUNDING_PENDING = "funding_pending"
    AWAITING_CONF   = "awaiting_confirmation"
    EXCHANGE_LOCKED = "exchange_locked"
    OPEN            = "open"
    # Failure states
    FAILED          = "failed"
    CLOSING         = "closing"

@dataclass
class ChannelOpenSaga:
    saga_id: str
    peer_pubkey: str
    capacity_sat: int
    local_reserve_sat: int
    status: ChannelOpenStatus
    funding_txid: Optional[str] = None
    funding_output_index: Optional[int] = None
    failure_reason: Optional[str] = None

# Transactions (forward path)
async def T1_negotiate_parameters(saga: ChannelOpenSaga, peer_client) -> ChannelOpenSaga:
    """Send open_channel, receive accept_channel."""
    result = await peer_client.open_channel(
        pubkey=saga.peer_pubkey,
        local_funding_amount=saga.capacity_sat,
    )
    return ChannelOpenSaga(**{**saga.__dict__, 'status': ChannelOpenStatus.FUNDING_PENDING})

async def T2_broadcast_funding(saga: ChannelOpenSaga, wallet, broadcast) -> ChannelOpenSaga:
    """Construct and broadcast the funding transaction."""
    tx = await wallet.create_funding_tx(saga.capacity_sat)
    txid = await broadcast(tx)
    return ChannelOpenSaga(**{
        **saga.__dict__,
        'funding_txid': txid,
        'funding_output_index': 0,
        'status': ChannelOpenStatus.AWAITING_CONF
    })

async def T3_await_confirmation(saga: ChannelOpenSaga, chain_watcher) -> ChannelOpenSaga:
    """Wait for the funding transaction to reach required depth."""
    await chain_watcher.wait_for_confirmations(saga.funding_txid, confirmations=3)
    return ChannelOpenSaga(**{**saga.__dict__, 'status': ChannelOpenStatus.EXCHANGE_LOCKED})

async def T4_exchange_funding_locked(saga: ChannelOpenSaga, peer_client) -> ChannelOpenSaga:
    """Exchange funding_locked messages."""
    await peer_client.send_funding_locked(saga.funding_txid)
    await peer_client.receive_funding_locked()
    return ChannelOpenSaga(**{**saga.__dict__, 'status': ChannelOpenStatus.OPEN})

# Compensating transactions (backward path)
async def C1_send_error(saga: ChannelOpenSaga, peer_client) -> None:
    """Compensate T1: tell the peer the negotiation is cancelled."""
    await peer_client.send_error(f"Channel open cancelled: {saga.failure_reason}")

async def C2_attempt_rbf_or_abandon(saga: ChannelOpenSaga, wallet, broadcast) -> None:
    """
    Compensate T2: the funding transaction is in the mempool.
    Options: RBF to zero output (double-spend), or wait for it to be abandoned.
    If the tx already confirmed, C2 cannot undo it — escalate to C3.
    """
    if saga.funding_txid:
        await wallet.attempt_cancel_transaction(saga.funding_txid)

async def C3_force_close(saga: ChannelOpenSaga, channel_manager) -> None:
    """
    Compensate T3: funding confirmed but channel is stuck.
    The only recourse is a unilateral close.
    """
    await channel_manager.force_close(saga.funding_txid, saga.funding_output_index)

Notice that the compensating transactions are not symmetric. C1 (cancel negotiation) is cheap and clean. C2 (cancel a mempool transaction) is possible but probabilistic, it depends on whether the transaction has already confirmed. C3 (force close a confirmed funding transaction) is expensive, it costs an on-chain fee and takes to_self_delay blocks to recover funds.

This asymmetry is a fundamental property of Bitcoin operations. On-chain state is progressively harder to undo. The saga pattern makes this explicit.

The Orchestrator

The orchestrator runs the saga, handles failures, and decides when to compensate:

class ChannelOpenOrchestrator:
    def __init__(self, saga_store, peer_client, wallet, broadcast, chain_watcher, channel_manager):
        self.saga_store = saga_store
        self.peer_client = peer_client
        self.wallet = wallet
        self.broadcast = broadcast
        self.chain_watcher = chain_watcher
        self.channel_manager = channel_manager

    async def run(self, saga: ChannelOpenSaga) -> ChannelOpenSaga:
        steps = [
            (T1_negotiate_parameters, C1_send_error,           [self.peer_client]),
            (T2_broadcast_funding,    C2_attempt_rbf_or_abandon,[self.wallet, self.broadcast]),
            (T3_await_confirmation,   C3_force_close,           [self.chain_watcher]),
            (T4_exchange_funding_locked, None,                  [self.peer_client]),
        ]

        completed = []

        for transaction, compensator, args in steps:
            try:
                saga = await transaction(saga, *args)
                await self.saga_store.save(saga)
                completed.append((compensator, args, saga))
            except Exception as e:
                saga = ChannelOpenSaga(**{
                    **saga.__dict__,
                    'status': ChannelOpenStatus.FAILED,
                    'failure_reason': str(e)
                })
                await self.saga_store.save(saga)
                # Compensate in reverse order
                await self._compensate(completed, saga)
                return saga

        return saga

    async def _compensate(self, completed, saga):
        for compensator, args, saga_at_step in reversed(completed):
            if compensator is None:
                continue
            try:
                await compensator(saga, *args)
            except Exception as comp_error:
                # Log compensation failure — this requires manual intervention
                print(f"COMPENSATION FAILED: {compensator.__name__}: {comp_error}")
                # Do not raise — attempt remaining compensations

The orchestrator saves the saga state after each step. If the process crashes mid-saga, it can be recovered by loading the saved saga and resuming or compensating from where it stopped. This is the idempotency requirement: each step must be safe to re-execute if it was interrupted before the state was saved.

A Payment Routing Saga

A multi-hop Lightning payment is a saga with a different compensation structure. The HTLC mechanism provides built-in compensation: if a payment fails at any hop, HTLCs are failed back upstream. The saga structure is implicit in the protocol.

But the saga framing is still useful for an application that needs to track payment outcomes:

@dataclass
class PaymentSaga:
    saga_id: str
    payment_hash: str
    amount_msat: int
    recipient_pubkey: str
    max_fee_msat: int
    status: str   # 'routing', 'pending', 'succeeded', 'failed'
    failure_code: Optional[str] = None
    preimage: Optional[str] = None

async def run_payment_saga(saga: PaymentSaga, lnd_client, saga_store) -> PaymentSaga:
    """
    Send a payment and track its outcome.
    Lightning's HTLC mechanism handles on-chain compensation automatically;
    the saga here is the application-level tracking.
    """
    try:
        # T1: initiate the payment
        result = await lnd_client.send_payment(
            payment_hash=saga.payment_hash,
            amount_msat=saga.amount_msat,
            dest=saga.recipient_pubkey,
            fee_limit_msat=saga.max_fee_msat,
            timeout_seconds=60
        )

        if result.status == 'SUCCEEDED':
            saga = PaymentSaga(**{
                **saga.__dict__,
                'status': 'succeeded',
                'preimage': result.payment_preimage
            })
        else:
            # Lightning compensated automatically (HTLCs failed back)
            # We record the failure and its reason
            saga = PaymentSaga(**{
                **saga.__dict__,
                'status': 'failed',
                'failure_code': result.failure_reason
            })

    except Exception as e:
        saga = PaymentSaga(**{
            **saga.__dict__,
            'status': 'failed',
            'failure_code': f'exception:{str(e)}'
        })

    await saga_store.save(saga)
    return saga

For Lightning payments, the compensation is handled by the protocol. For your application state balances, order records, external system state, the compensation must be explicit.

Backward vs. Forward Compensation

Garcia-Molina's original paper distinguishes two compensation strategies:

Backward compensation (rollback): Undo completed transactions in reverse order. Return the system to the state before the saga began. This is what most people think of when they hear "compensating transaction."

Forward compensation (retry): Accept that some transactions cannot be undone and instead push forward to a completed state, even if it is not the originally intended state. This is appropriate when compensation is more expensive or more dangerous than completion.

In Bitcoin infrastructure, forward compensation is frequently the right choice. Consider the channel open scenario above: if T3 (await confirmation) succeeds, the funding transaction is confirmed, C3 (force close) is expensive and disruptive. In many cases, the right approach is to keep retrying T4 (exchange funding locked) until the peer reconnects, rather than immediately force-closing.

This is a design decision that must be explicit:

async def T4_exchange_funding_locked_with_retry(
    saga: ChannelOpenSaga,
    peer_client,
    max_attempts: int = 100,
    retry_interval_seconds: int = 60
) -> ChannelOpenSaga:
    """
    Forward compensation: retry T4 rather than force-closing.
    The peer may reconnect within minutes or hours.
    Force close is the last resort, not the first.
    """
    for attempt in range(max_attempts):
        try:
            await peer_client.send_funding_locked(saga.funding_txid)
            await peer_client.receive_funding_locked(timeout=30)
            return ChannelOpenSaga(**{**saga.__dict__, 'status': ChannelOpenStatus.OPEN})
        except TimeoutError:
            if attempt < max_attempts - 1:
                await asyncio.sleep(retry_interval_seconds)
            else:
                raise  # Exhausted retries — propagate to orchestrator

The choice between backward and forward compensation should be documented explicitly for each step, because it has real operational consequences.

Idempotency Is Not Optional

Every transaction in a saga must be idempotent; safe to execute more than once with the same result. The orchestrator may re-execute a step if the process crashes after the step succeeded but before the state was saved.

For Bitcoin operations, idempotency often requires checking current state before acting:

async def T2_broadcast_funding_idempotent(
    saga: ChannelOpenSaga,
    wallet,
    broadcast,
    mempool_client
) -> ChannelOpenSaga:
    """Idempotent version of T2."""
    # If we already have a txid, check if it's still in the mempool or confirmed
    if saga.funding_txid:
        status = await mempool_client.get_tx_status(saga.funding_txid)
        if status in ('mempool', 'confirmed'):
            # Already broadcast — skip, return existing saga
            return ChannelOpenSaga(**{
                **saga.__dict__,
                'status': ChannelOpenStatus.AWAITING_CONF
            })
        # Transaction was dropped — fall through to re-broadcast

    tx = await wallet.create_funding_tx(saga.capacity_sat)
    txid = await broadcast(tx)
    return ChannelOpenSaga(**{
        **saga.__dict__,
        'funding_txid': txid,
        'funding_output_index': 0,
        'status': ChannelOpenStatus.AWAITING_CONF
    })

The pattern is: load current state, check if this step already happened, proceed only if it has not.

Submarine Swaps as a Saga

A submarine swap involves exchanging on-chain bitcoin for Lightning bitcoin, or vice versa. It is one of the cleaner examples of a saga in Lightning infrastructure. It involves two parties, two chains (conceptually), and a hash-locked contract that coordinates the exchange.

Forward swap (on-chain → Lightning):

Step	Transaction	Compensating Transaction
T1	Alice requests swap from LSP; LSP generates invoice and on-chain address	C1: Alice abandons, no funds committed
T2	Alice sends on-chain payment to LSP's script (HTLC locked to invoice hash)	C2: Alice waits for HTLC timeout, reclaims on-chain funds
T3	LSP pays Alice's Lightning invoice (revealing preimage)	C3: Not possible — Lightning payment is final
T4	LSP claims on-chain HTLC using revealed preimage	C4: Not applicable — follows from T3

If T3 fails, the LSP cannot pay the Lightning invoice. T2's on-chain HTLC automatically refunds to Alice after the timeout. The time-lock is the compensating transaction. Garcia-Molina would recognize this immediately: the compensation is built into the contract.

What the Sagas Paper Gets Wrong for Bitcoin

Garcia-Molina assumed compensating transactions are always possible and always effective. Bitcoin proves this is not always true:

Irreversibility: An on-chain transaction that confirms cannot be compensated. The compensation must come from subsequent transactions (force close, timeout claims), not from undoing the original.
Probabilistic finality: "Confirmed" is not binary, it is a confidence level based on depth. A compensating strategy for a transaction with 1 confirmation is different from one with 6.
Time as a resource: Every block that passes changes the compensation options. A 20-block CLTV expiry window gives you 20 blocks to compensate before the on-chain contract resolves automatically in the other party's favor.

These constraints make Bitcoin saga design harder than database saga design, but also more constrained, which means more predictable. The protocol tells you exactly what happens if you do nothing. That is a property most distributed systems lack.

DEV Community