DEV Community

kanaria007
kanaria007

Posted on • Originally published at zenn.dev

Chapter 4 — RML-3 (History World): Irreversible History, Forward-Only Correction

#distributedsystems #microservices #sre #architecture

The Worlds of Distributed Systems — Chapter 4

“Can you really make it as-if-it-never-happened by reverting the DB?”
“...The outside world may already remember it.”

In Chapter 2 we explored RML-1 (Closed World): a room where failure is safe.
In Chapter 3 we explored RML-2 (Dialog World): recovery as conversation and reconciliation.

Now we step into the third world:

RML-3 — History World

This is the world where you don’t “undo” events.

You don’t erase the past.
You layer corrections on top of what happened.

1) What is the History World?

In one line:

RML-3 is a world with shared history across multiple principals—and accountability for that history.

1.1 Typical RML-3 domains

  • Finance

    • bank transfers, card payments, securities trading

  • Tax / accounting

    • invoices, receipts, journal entries

  • Healthcare / public infrastructure

    • medical records, prescription history, critical public logs

  • B2B / contracts

    • purchase orders, fulfillment records, contractual obligations

What these domains share:

  • Records that retain meaning when a third party looks later
  • Multiple organizations and humans act based on those records

1.2 Three keywords

A practical way to remember RML-3:

  1. Permanence
    Once recorded, facts should be treated as not logically deletable (even if your DB can delete rows).

  2. Sharedness
    Multiple parties rely on the record; it’s not “your system’s internal state” anymore.

  3. Accountability
    You must be able to explain later: what happened, why, and what you did about it.

At this point, “system convenience” stops being the main axis.

2) The hardest part: where does history begin?

In practice, the hardest question is:

Where is the boundary between RML-2 and RML-3?

2.1 Typical signs you’ve crossed into RML-3

You should suspect “History World” when:

  • Money / goods / rights move

    • payments are captured/settled
    • inventory is committed
    • usage rights are granted

  • A promise to third parties is created

    • legally meaningful documents exist (invoices, contracts)
    • obligations arise under ToS / policy

  • The outside world takes action

    • a customer pays, reports taxes, takes medication, etc.

Conversely, things that may still be RML-2:

  • internal memos
  • provisional statuses
  • temporary results that can be recomputed at any time

2.2 Design trick: define a “History Hand-off Point”

A clean way to design the boundary is to name it:

History Hand-off Point — “From here on, this action belongs to the History World.”

Examples:

  • Payments: authorization (RML-2) → capture/settlement (RML-3)
  • Invoices: draft (RML-2) → issued (RML-3)
  • Healthcare: memo (RML-2) → official record (RML-3)

You can (and should) embed this boundary into UI and APIs:

  • UI buttons: “Save (reversible)” vs “Issue / Confirm (enters history)”
  • Endpoints: POST /invoices/{id}/issue, POST /payments/{id}/capture

If the boundary is unnamed, teams will disagree silently—and incidents get much harder.

3) What “rollback” really means in RML-3

In RML-3, rollback in the RML-1/2 sense mostly doesn’t exist.

You don’t delete the past.
You add correction events on top of it.

3.1 The Effect Ledger view

A useful abstraction for History World is an Effect Ledger:

  • Record external effects (payments, invoices, notifications) in an append-only log
  • Chain it (hashes/signatures) to make tampering hard
  • A reconciler (human + system) appends correction events later

A cartoon example:

[TX-1001] 2025-04-01 10:00  "Charge ¥5,000"
[TX-1002] 2025-04-02 09:30  "Refund for TX-1001 (¥5,000)"
[TX-1003] 2025-04-02 10:00  "Apology coupon issued (¥1,000)"
Enter fullscreen mode Exit fullscreen mode

Notice what we did not do:

  • We didn’t delete TX-1001.
  • We added TX-1002 and TX-1003.

That’s the safest mental model:

RML-3 rollback = correction events + reconciliation.

3.2 Rollback becomes “Refund + Correction + Explanation”

From the user’s perspective, “rollback” in RML-3 usually means a bundle of:

  1. Refund (financial correction)
  2. Correction (system state fix)
  3. Explanation (what happened and what you did)

Entering the History World means accepting responsibility to do these three coherently.

4) Design changes required by RML-3

When your system genuinely enters RML-3, your default design instincts need to shift.

4.1 Prefer versions/events over destructive updates

  • In RML-2, “overwrite state” can still be workable.

  • In RML-3, you usually want:

    • versioned records
    • event-like histories
    • append-only “what changed when” trails

Examples:

  • Invoices

    • instead of overwriting status in-place,
    • keep invoice_versions or append “issued/canceled” events

  • Balances

    • instead of writing a single balance column,
    • derive balance from a transaction log

4.2 Record “who decided what, when, and based on what”

In the History World, meta-information matters:

  • Whose decision was it? (system / operator / customer)
  • What did they see at the time of the decision?
  • Which policy/rule/version justified it?

In implementation terms, this often means storing:

  • actor/user ID (or “SYSTEM”)
  • trace ID / request ID
  • policy/rule version
  • a short reason/comment

This connects directly to incident response and “case files” in later chapters.

5) Common anti-patterns (how teams get burned)

5.1 Trying to “rewind” the History World like RML-2

Scenario:

  • You issued an invoice and sent it to the customer.
  • You discover a bug and “delete the invoice record from the DB.”

Results:

  • The email still exists in the customer’s inbox.
  • Other systems may have referenced the invoice.
  • Audits and troubleshooting become extremely hard.

RML interpretation:

You tried to overwrite History World facts using Dialog World rollback.

5.2 A fuzzy History Hand-off Point

If the team disagrees on “what counts as official history”:

  • someone treats drafts as already history
  • someone treats only issued records as history

Then:

  • you either make everything heavy (over-RML-3)
  • or you treat true RML-3 actions too lightly (under-RML-3)

Either way, you lose the advantage of the three-world map.

5.3 Treating audit/transaction logs as “just logs”

Another classic failure:

  • audit logs are deleted/overwritten casually
  • later someone asks: “Who moved this money, when?”
  • you can’t answer with confidence

Important point:

Logs themselves span worlds.
Some logs are RML-1/2 technical artifacts. Some are RML-3 history artifacts.

You must explicitly identify which is which.

6) How to embed History World thinking into real design work

6.1 Make “Is this RML-3?” a normal question

You want these questions to appear naturally in reviews:

  • “At what point does this flow enter RML-3?”
  • “Is this event expected to remain as history?”
  • “Is this log purely technical—or audit-grade history?”

That’s step one.

6.2 Put the boundary into UI / APIs / docs

  • UI: distinguish “Save (reversible)” from “Confirm/Issue (enters history)”
  • API: split endpoints so the hand-off is explicit (/issue, /capture)
  • Docs: keep a list of “History World operations” vs “Dialog World operations”

6.3 Design “correction operations” as first-class features

If you’re in RML-3, build the correction path up front:

  • refunds
  • corrections
  • invalidations
  • append-only comments / notes

So you don’t rely on backdoors like:

“If it breaks, we’ll fix it with SQL.”

7) Mini checklist (RML-3 edition)

7.1 History Hand-off

  • [ ] Is there a named History Hand-off Point in this product area?
  • [ ] Is it visible in UI/API/docs?

7.2 Data & logs

  • [ ] Are history-critical records managed via versions/events (not destructive updates)?
  • [ ] Have you identified which logs are RML-3 audit/history logs?
  • [ ] Do you store “who/when/what/policy-version” for history-grade actions?

7.3 Rollback mindset

  • [ ] Do you treat rollback in RML-3 as refund + correction + explanation?
  • [ ] Are you avoiding “make it disappear” actions for History World facts?

8) Closing — History World reveals itself after you mess up

Most days, you don’t feel RML-3.

Payments settle, invoices go out, and everything looks fine.

But the difference shows up after you mess up:

  • double charges
  • wrong invoices issued
  • incorrect information sent to customers/partners

In that moment, what determines trust is:

  • Can you refund/correct/explain coherently?
  • Can you reconstruct who decided what, and why?
  • Can you show the recovery as history—not as deletion?

RML-3 is the world where distributed systems meet society.

Next, we’ll zoom into the Dialog World mechanics that make History World survivable: structured failures and action hints (Chapter 5), compensations/sagas and idempotency (Chapter 6), and the API/client boundary that carries the worldview (Chapter 7).

Top comments (0)