Immutability - Not a Universal Law but a Trade-off part 2

In the first part, we explored immutability through code, domain modeling, and APIs—starting from immutable functions, moving through state transitions, and ending with intent-driven APIs.

In this part, we zoom out even further.

We’ll look at how immutability affects persistence and system architecture: from relational databases and event stores to streaming systems.

Once immutability reaches storage and distributed systems, it stops being just a coding technique—and becomes an architectural trade-off.

Storage Level

Across all databases, immutability boils down to:

• Don’t UPDATE / DELETE
• Only INSERT (append)

But how natural (or painful) that is depends on the database type.

Some databases fight immutability, some tolerate it, and some are built for it.

Relational Databases (SQL)

• PostgreSQL, MySQL
• by default is mutable (UPDATE/DELETE) so you’re fighting the default model
• you can do immutability but it is pain, here are few options
  • Append-only tables
  • Versioned rows
  • Event store table
• queries become complex (joins, aggregations)
• performance tuning required for large histories

Document Databases (NoSQL)

• MongoDB, Couchbase
• still mutable by default
• they store whole documents
• you can do it by
  • Store versions
  • Embed history

Key-Value Stores

• Redis, DynamoDB
• default is overwrite value
• you can do it by
  • Versioned keys
  • Append logs (lists/streams)
• very easy to create chaos

Wide-Column Stores

• Apache Cassandra
• default is
  • Append-heavy writes
  • Time-series friendly
• closer to immutable by design but Limited querying flexibility

Event Stores

• EventStoreDB, Apache Kafka
• by default mindset Events are first-class and Append-only approach is used
• built for immutability have replay capability and ordering guarantees
• enforce immutability—not just support it.

Time-Series Databases

• InfluxDB, TimescaleDB
• by default Append-only
• but not suited for complex domain logic
• Perfect for tracking, not for modeling behavior.

Object Storage (Blob Storage)

• Amazon S3 Google Cloud Storage
• Objects are not modified New versions are created
• storage is cheap
• mostly used for Snapshots Backups Data lakes

Data Lake / Analytical Storage

• Apache Iceberg Delta Lake
• by default Append-only data files
• Massive scale

Based on Object Storage and Data Lake solution we can see that analytics world embraced immutability before developers did

Immutable DB

• Datomic
• default mindset store facts (Every fact is timestamped and immutable)

Based on the storage level options we see that immutability is not a universal law—it’s a trade-off on system level

System Level

At the system level, immutability stops being just a programming technique.
It becomes an architectural strategy.

This is where immutability starts influencing:

• scalability
• fault tolerance
• messaging
• recovery
• debugging
• data flow between services

And this is also where the trade-offs become much more expensive.

Traditional systems architecture is usually centers around mutable state:
Service → Database → UPDATE state

In immutable architectures system evolve around events and append-only flows:
Command → Event → Stream → Projection
Instead of asking

What is the current state?

We ask

What happened?

Why Immutability Works Well in Distributed Systems

Distributed systems are fundamentally hard because:

• services fail
• networks fail
• retries happen
• messages arrive late
• multiple systems update the same data

Shared mutable state makes those problems even harder.

Immutability reduces coordination problems because data becomes append-only and easier to reason about.

Instead of mutating shared state directly:
balance = 120 systems publish immutable events: BalanceIncreased(+20)

• if a service crashes we can rebuild state replay(events)
• one event can feed many systems OrderPlaced → Billing Shipping Analytics Notifications
• debugging: you can inspect what happened in which order at what time
• systems communicate asynchronously so
  • Consumers scale independently.
  • One failing service does not necessarily stop the whole workflow

Technical Implementation — Command → Event → Stream → Projection

At a high level: Command → Event → Stream → Projection looks elegant and simple. In reality, every step introduces implementation decisions, tooling choices, and trade-offs. Lets investigate it

Command — “What should happen?”

{
  "type": "ShipOrder",
  "orderId": 123
}

A command represents intent. But intent still needs a transport mechanism. Commands are usually delivered through:

• REST API
• gRPC
  • similar to REST conceptually, but optimized for performance
• message queue
  • asynchronous and loosely coupled
  • queues and streams are different (RabbitMQ)

Commands are usually:

• targeted
• owned by one service

Events are usually:

• broadcast
• consumed by many services

Event — “What happened?”

Events are immutable facts.

Flow

Command
    ↓
Validation
    ↓
Business Logic
    ↓
Event Created
    ↓
Persisted
    ↓
Published

;; on command POST /orders/123/ship

(defn ship-order [repository event-publisher order-id]
  (let [order (repository/find repository order-id)
        shipped-order (order/ship order)
        event {:type "OrderShipped"
               :order-id order-id
               :occurred-at (java.time.Instant/now)}]

    ;; persist updated order if needed
    (repository/save repository shipped-order)

    ;; publish immutable event
    (event-publisher/publish event-publisher event)))

Commands are usually delivered as:

• JSON
• Avro
• Protobuf

{
  "type": "OrderShipped",
  "orderId": 123,
  "occurredAt": "2026-05-11T10:00:00Z"
}

Events represent business meaning
Instead of storing: status = SHIPPED
you preserve events:

OrderPlaced
PaymentReceived
OrderShipped

Now history becomes explicit.

Preserve events where ?

• relational database (simplest)
• broker (stream) like Kafka

Stream — “How events are preserved"

The stream is the append-only flow of events.

At this stage, the architecture stops being: request → response
and becomes: continuous flow of facts. This is one of the biggest conceptual shifts.

What Is a Stream?

A stream is not just:

• a queue
• a transport mechanism
• temporary messaging

it becomes:

• system history
• integration layer
• replay source
• recovery mechanism

Flow

Service A
    ↓
publish event
    ↓
stream
    ↓
multiple consumers react

Core Idea Instead of sharing mutable state: systems share immutable facts.

Note: Consumers must tolerate duplicates. Example: event processed twice must not:

• double-charge payment
• double-send email

Projection — “Current queryable state”

Without projections on every request:

• read all events
• compute current state

So consumers of the stream process events and update read model.

When projections become more than simple event handlers use Apache Flink

Storage options

• Relational DB
  • dashboards
  • transactional reads
• Elastic Search
  • full-text search and filtering

Can We Store Commands, Events, and Projections in Same Relational DB?

Yes. And this is actually a very pragmatic architecture to start benefiting from immutability.
You do NOT need:

• Kafka
• Flink
• event sourcing

A relational DB with:

• append-only events
• projections
• explicit domain events

already provides many benefits.

When DB becomes bottleneck start introducing other components like Kafka

The Real Costs (This Part Matters)

Immutability at system level is not “free architecture elegance.”

• It introduces serious complexity.
• Storage Growth. You store: every event every version potentially forever
• Read complexity
  • current state often requires: read + replay + fold/reduce
• Schema Evolution is difficult because old events can not simply disappera
• 0perational Complexity because you have to take care/think about
  • streams
  • retries
  • ordering
  • idempotency
  • projections
  • replay logic This is harder than CRUD apps

When Immutability Works Well

Use immutable/event-driven systems when:

• workflows are complex
• history matters
• auditability is important
• systems are distributed
• multiple services react to the same events
• replay/recovery provides value

Examples:

• payments
• financial systems
• logistics
• ticketing
• analytics pipelines

When NOT to Use It

Avoid it when:

• application is simple CRUD
• state changes have no historical value
• system is small/simple
• team is inexperienced with distributed systems
• operational simplicity matters more than scalability

Conclusion

shared mutable state problems at system level is about multiple services/processes/systems:

• read the same state
• modify the same state
• depend on the latest version being correct

Usually through:

• shared databases
• synchronous APIs
• distributed transactions

Immutability is powerful because in distributed systems we are dealing with time, concurrency, and failure and Immutable systems embrace those realities instead of hiding them behind mutable state.
But the cost is complexity.

The truth: using immutability at system level You reduce problems caused by shared mutable state, but introduce new problems around:

• event design
• replay
• storage
• consistency
• operational complexity

Immutability does NOT remove complexity.
It shifts complexity from:

• synchronization
• locking
• coordination

into:

• event modeling
• eventual consistency
• replay
• projections
• operational tooling

You can chose which problem to fight.

You think this is the end - no

In the next post we are going to take a look into hybrid approaches and real-world examples like Git, Amazon S3, Docker images, and Ticketmaster.