Rollback Chains: When Payment Fails, What Actually Happens

Rollback Chains: When Payment Fails, What Actually Happens

In the previous post, I showed the orchestrator's state transition table. It knows which topic to publish on failure. But what happens on the receiving end? What does "rollback" actually look like in code?

This post walks through three real failure scenarios in my saga system. Each one triggers a different rollback chain, and each service handles compensation differently.

How Compensation Works

Every service implements two operations: the forward action and its compensation.

Service	Forward	Compensation
product-validation	validateExistingProducts()	rollbackEvent()
payment-service	realizePayment()	realizeRefund()
inventory-service	updateInventory()	rollbackInventory()

The forward action does the work and publishes SUCCESS or ROLLBACK to the orchestrator. The compensation undoes the work and publishes FAIL.

There's an important distinction between ROLLBACK and FAIL. ROLLBACK means "I failed, and I need my own compensation first." FAIL means "I already rolled back, now the previous service needs to compensate." The orchestrator uses this to chain rollbacks in the correct order.

Scenario 1: Inventory Fails (Full Rollback Chain)

This is the worst case. Product validation passed. Payment was charged. Then inventory fails because the product is out of stock.

The rollback chain goes: Inventory → Payment → Product Validation → Finish Fail.

Step 1: Inventory Detects the Problem

public void updateInventory(Event event) {
    try {
        checkCurrentValidation(event);
        createOrderInventory(event);
        updateInventory(event.getOrder());
        handleSuccess(event);
    } catch (Exception ex) {
        log.error("Error trying to update inventory: ", ex);
        handleFailCurrentNotExecuted(event, ex.getMessage());
    }
    producer.sendEvent(jsonUtil.toJson(event).orElseThrow());
}

private void checkInventory(int available, int orderQuantity) {
    if (orderQuantity > available) {
        throw new ValidationException("Product is out of stock!");
    }
}

When stock is insufficient, checkInventory throws. The catch block calls handleFailCurrentNotExecuted, which sets status to ROLLBACK. The event goes back to the orchestrator with source=INVENTORY_SERVICE, status=ROLLBACK.

private void handleFailCurrentNotExecuted(Event event, String message) {
    event.setStatus(ROLLBACK);
    event.setSource(CURRENT_SOURCE);
    addHistory(event, "Fail to update inventory: ".concat(message));
}

Step 2: Orchestrator Routes the Rollback

The orchestrator looks up the table: INVENTORY_SERVICE + ROLLBACK → INVENTORY_FAIL. It publishes to inventory-fail.

Step 3: Inventory Compensates Itself

The inventory-service consumes from inventory-fail and restores the previous quantities:

public void rollbackInventory(Event event) {
    event.setStatus(FAIL);
    event.setSource(CURRENT_SOURCE);
    try {
        returnInventoryToPreviousValues(event);
        addHistory(event, "Rollback executed for inventory!");
    } catch (Exception ex) {
        addHistory(event, "Rollback not executed for inventory: ".concat(ex.getMessage()));
    }
    producer.sendEvent(jsonUtil.toJson(event).orElseThrow());
}

private void returnInventoryToPreviousValues(Event event) {
    orderInventoryRepository
        .findByOrderIdAndTransactionId(event.getOrder().getOrderId(), event.getTransactionId())
        .forEach(orderInventory -> {
            var inventory = orderInventory.getInventory();
            inventory.setAvailable(orderInventory.getOldQuantity());
            inventoryRepository.save(inventory);
        });
}

Notice that it uses oldQuantity from the OrderInventory record. When the forward action runs, it saves both old and new quantities. The rollback reads the old value and restores it. No guessing.

After rollback, inventory publishes FAIL back to the orchestrator. Now the orchestrator sees INVENTORY_SERVICE + FAIL → PAYMENT_FAIL and publishes to payment-fail.

Step 4: Payment Refunds

public void realizeRefund(Event event) {
    event.setStatus(FAIL);
    event.setSource(CURRENT_SOURCE);
    try {
        changePaymentStatusToRefund(event);
        addHistory(event, "Rollback executed for payment!");
    } catch (Exception ex) {
        addHistory(event, "Rollback not executed for payment: ".concat(ex.getMessage()));
    }
    producer.sendEvent(jsonUtil.toJson(event).orElseThrow());
}

private void changePaymentStatusToRefund(Event event) {
    var payment = findByOrderIdAndTransactionId(event);
    payment.setStatus(PaymentStatus.REFUND);
    setEventAmountItems(event, payment);
    save(payment);
}

Payment changes the status to REFUND and publishes FAIL. The orchestrator sees PAYMENT_SERVICE + FAIL → PRODUCT_VALIDATION_FAIL and continues the chain.

Step 5: Product Validation Rolls Back

public void rollbackEvent(Event event) {
    changeValidationToFail(event);
    event.setStatus(FAIL);
    event.setSource(CURRENT_SOURCE);
    addHistory(event, "Rollback executed on product validation!");
    producer.sendEvent(jsonUtil.toJson(event).orElseThrow());
}

private void changeValidationToFail(Event event) {
    validationRepository
        .findByOrderIdAndTransactionId(event.getOrderId(), event.getTransactionId())
        .ifPresentOrElse(
            validation -> {
                validation.setSuccess(false);
                validationRepository.save(validation);
            },
            () -> createValidation(event, false));
}

Finally, the orchestrator sees PRODUCT_VALIDATION_SERVICE + FAIL → FINISH_FAIL and the saga ends.

Scenario 2: Payment Fails (Partial Rollback)

Payment fails (card declined, fraud blocked). Inventory was never touched, so there's nothing to roll back there. Only product validation needs compensation.

Payment ROLLBACK → payment-fail → Payment refunds → FAIL
→ product-validation-fail → Validation marks as failed → FAIL
→ finish-fail

Shorter chain. Same mechanism.

Scenario 3: Product Validation Fails (No Rollback Needed)

Product validation is the first step. If the product doesn't exist in the catalog, nothing was charged and nothing was reserved. The orchestrator sees PRODUCT_VALIDATION_SERVICE + FAIL → FINISH_FAIL and skips straight to the end.

The Pattern: Save Before You Change

Every service follows the same pattern for safe rollbacks. Before changing any data, save the original values.

The inventory-service saves oldQuantity and newQuantity in OrderInventory:

private OrderInventory createOrderInventory(Event event,
                                            OrderProducts product,
                                            Inventory inventory) {
    return OrderInventory.builder()
        .inventory(inventory)
        .oldQuantity(inventory.getAvailable())    // save current
        .orderQuantity(product.getQuantity())
        .newQuantity(inventory.getAvailable() - product.getQuantity())  // save new
        .orderId(event.getOrder().getOrderId())
        .transactionId(event.getTransactionId())
        .build();
}

The payment-service saves the payment with status PENDING before processing:

private void createPendingPayment(Event event) {
    var payment = Payment.builder()
        .orderId(event.getOrder().getOrderId())
        .transactionId(event.getTransactionId())
        .totalAmount(totalAmount)
        .totalItems(totalItems)
        .build();
    save(payment);  // status defaults to PENDING via @PrePersist
}

This way, the compensation logic always has the data it needs to undo the change.

Idempotency Guards

Each service checks for duplicate transactions before processing:

private void checkCurrentValidation(Event event) {
    if (orderInventoryRepository.existsByOrderIdAndTransactionId(
            event.getOrder().getOrderId(), event.getTransactionId())) {
        throw new ValidationException("There's another transactionId for this validation.");
    }
}

If Kafka delivers the same message twice (at-least-once delivery), the service detects the duplicate and skips the forward action. Without this check, you'd charge the customer twice or double-deduct inventory.

What's Next

The rollback logic is clean when you read the code, but things get tricky in practice. What if the refund fails? What if Kafka drops a message? In the next post, I'll cover how I test these scenarios with unit tests and real failure injection.

The repo: github.com/pedrop3/saga-orchestration