Why making many independent computers behave like one coherent system is where the trouble actually begins.
"Explore how independent machines, networks, concurrency, partial failures, clocks, transparency, and coordination turn “just add more computers” into a fundamentally different engineering problem"
Suppose a system needs more processing power, more storage, better availability, or simply needs to serve more people.
The obvious instinct is: ✨Add machines✨
That sounds almost embarrassingly straightforward. One computer is doing too much work, so give it friends. Divide the workload. Share the resources. Keep going as demand grows.
And there are plenty of reasons to do exactly that. Some problems are inherently distributed. Some become too large for a single machine. Systems need to scale, remain available when components fail, share resources, serve geographically distributed users, and evolve without replacing everything at once.
So we add another computer.
Then another.
Then a hundred.
The trap is subtle: Somewhere during that progression, it is very easy to keep reasoning as though we still have one computer, just bigger.
We do not.
A parallel system may have many processors while remaining tightly coupled around shared resources. A distributed system gives us something fundamentally less comfortable: Independent Computers.
Each has its own processor.
Its own memory.
Its own execution.
Its own problems.
And if one machine needs something held by another, it cannot simply reach into the other machine’s memory.
It has to ask.
Through a message.
Across a network.
That tiny change is where the vibes start getting cursed.
So We Make Them Pretend to Be One
There is a wonderfully ambitious definition of a distributed system: A collection of autonomous or independent computers that appears to its users as a single coherent system.
Read both halves carefully.
Independent computers.
Single coherent system.
Those ideas are almost fighting each other.
The machines are autonomous, yet we want the user to experience coherence. They may run different hardware, operating systems, software, and protocols, yet the interaction should feel consistent. They communicate by passing messages, yet we want applications to behave as though the machinery underneath is one dependable whole.
That gives us one of the central design goals: Single System View.
The user should not have to care which machine stores an object, which server answers a request, which replica is currently active, or which node disappeared five seconds ago.
Ideally, the system handles that mess.
The user sees one thing.
The distributed system quietly negotiates reality underneath.
Beautiful.
Except the machines have not stopped being independent just because we hid them behind a nice interface.
The Network Gets a Vote
On one machine, communication between components can rely on mechanisms built around shared local resources.
Across independent computers, coordination happens through message passing. Coulouris’ characterization makes this constraint explicit, and with it come three consequences that keep resurfacing throughout distributed computing: concurrency, independent failures, and the lack of a global clock.
Now consider what our innocent abstraction has inherited.
The network might be slow.
A message might be delayed.
Communication has latency.
Bandwidth is not infinite.
Topology can change.
Different machines can be running different technologies.
And the network itself can fail.
These facts sound painfully obvious when written down. Yet many distributed designs become fragile precisely because they quietly assume the opposite:
The network is reliable.
Latency is zero.
Bandwidth is infinite.
Transport is free.
The network is secure.
Topology does not change.
There is one administrator.
The infrastructure is homogeneous.
Each assumption makes the system feel a little more like one computer.
Each assumption is also something reality is free to revoke.
The abstraction is not necessarily lying to us.
But it is definitely leaving out some terms and conditions.
There Is No Single “Now”
The machines are also doing things concurrently.
Process A may execute while Process B executes somewhere else. They progress independently, exchanging messages only when coordination requires it.
This sounds manageable until we ask a very ordinary question:
What is the current state of the system?
On a single machine, that question already has nuances.
Across independent computers, it becomes much nastier.
There is typically no process holding perfectly current knowledge of everything happening everywhere. While one node learns that something changed, another node may already have changed again. Messages carrying that information take time to arrive.
There is no convenient global memory containing “the state”.
There is also no perfectly shared global clock quietly settling every argument about which event happened first.
Each computer has a local clock, and clocks do not advance in perfect agreement. Interaction models therefore have to reason explicitly about timing, communication delays, and the difference between synchronous and asynchronous assumptions.
So the system we wanted to behave like one computer cannot casually rely on two things a one-computer mental model makes very seductive:
One authoritative global state
**and
**One authoritative global time
This is where the illusion starts demanding real engineering.
Even Failure Stops Being Simple
Then something breaks.
On a centralized system, a catastrophic machine failure is at least brutally clear: The machine is down.
Distributed systems introduce a stranger condition.
Partial failure.
One component can fail while the rest of the system continues running.
A process crashes.
Another process is fine.
A network link disappears.
Two healthy machines become unable to communicate.
A message is sent but never arrives.
And, wonderfully, the surviving processes may not immediately know what happened.
That last part matters.
A remote machine that has crashed can look suspiciously similar to a remote machine that is simply taking a long time to respond.
The system is no longer choosing between: Working or Broken
It can be Working and Broken at the same time.
That is why distributed systems need explicit failure models: omission failures, communication failures, arbitrary failures, timing failures, and mechanisms for detecting, masking, tolerating, or recovering from them.
Fault tolerance exists because adding machines did not merely add resources.
It added independent failure boundaries.
Fine. Hide All of It.
At this point the natural reaction is almost defensive.
If distribution is causing all this complexity, hide the distribution.
That is transparency.
Make local and remote resources look alike through access transparency.
Let users access resources without knowing their physical location through location transparency.
Hide replication so multiple copies appear to be one resource.
Allow resources to move without breaking applications.
Hide concurrency where possible.
Mask failures so a failed component does not automatically become the user’s problem.
Even allow the system to scale or reconfigure without forcing applications to change.
This is how those independent computers begin to recover the illusion of one coherent machine.
And transparency is incredibly useful.
But there is a trap inside the solution.
More transparency is not automatically better.
Trying to hide every aspect of distribution can hurt performance or conceal behavior that applications genuinely need to understand. Research and classic treatments of distributed systems likewise describe distribution transparency as a major goal while recognizing that locality and distributed-system complexity cannot simply be wished away.
A remote operation still crosses a network even if its API looks local.
A retry still costs time even if failure recovery is invisible.
A replicated resource still needs coordination even if users see only one logical object.
Transparency can hide complexity from the interface.
It cannot repeal the mechanisms underneath it.
The Illusion Gets Surprisingly Expensive
Now the original plan looks different.
We added machines for scalability, but centralized services, centralized data, and centralized algorithms can become bottlenecks as the system grows.
We wanted availability, but partial failures mean the system must detect, mask, tolerate, and recover from faults.
We wanted one Single System View, but concurrency and message passing mean nobody automatically owns the current global state.
We wanted transparency, but excessive transparency can fight performance.
We wanted an extensible system, but growth introduces heterogeneity across hardware, software, middleware, protocols, locations, and administrators.
This does not mean distribution was a mistake.
It means our first mental model was incomplete.
The machines were never pieces of one physical computer.
They were autonomous participants being persuaded to cooperate.
Model the System We Actually Have
Once that is accepted, the design starts looking less mysterious.
Instead of reasoning vaguely about “the system”, we can model different parts of the problem explicitly.
A physical model asks what computers and devices exist and how they are interconnected.
An architectural model asks which entities communicate, how they communicate, what roles they have, and where they are placed. Client-server and peer-to-peer structures become architectural choices rather than incidental diagrams.
A fundamental model goes deeper into assumptions that affect almost every design: How processes interact, How timing behaves, How components can fail, and What security threats exist. Coulouris’ framework explicitly uses architectural and fundamental models to reason about properties such as performance, reliability, scalability, interaction, and failure.
This is the useful shift.
We stop asking:
How do we make these machines behave exactly like one computer?
And start asking:
Which differences must we hide, which must we manage, and which must the design acknowledge?
Now message passing makes sense.
Fault tolerance makes sense.
Replication makes sense.
Caching makes sense.
Different architectural roles make sense.
Even the awkward edge cases make sense.
They are not random complications bolted onto an otherwise simple system.
They are consequences of asking autonomous computers to cooperate while presenting something coherent above them.
One System, Carefully Negotiated
The magic of a distributed system is not that many computers somehow become one computer.
They never do.
They remain independent machines, executing concurrently, communicating through messages, observing only pieces of global reality, running on imperfect clocks, and sometimes failing without permission from the rest of the system.
The achievement is that, despite all of that, The Origami Software Engineer can design enough communication, coordination, replication, fault tolerance, transparency, and structure that a user can often interact with the whole thing as though it were one coherent system.
That distinction matters.
Because once we stop believing the illusion literally, distributed-system design becomes easier to reason about.
The network is not an implementation detail.
Failure is not exceptional.
Global knowledge is not free.
Transparency is not magic.
And coherence is not something multiple machines naturally possess.
It is something the system has to continuously create.
We started by adding computers because one machine was no longer enough.
Then we discovered that every new independent machine also added another piece of reality that had to be coordinated.
That’s not failure.
That’s evolution.
The “I liked this” Starter Pack:
Don’t let your fingers get lazy now.
- Like : It tells me this was worth writing.
- A Comment: Tell me your thoughts, your favorite snack, or a better title for this blog.
- Boost it: Especially with that one developer who definitely needs this.
Thanks for being here. It genuinely helps more than you know!
Find me elsewhere:
- Professional stuff: linkedin.com/in/Aaroophan
- Code stuff: github.com/Aaroophan
- UI stuff: aaroophan.dev/Aaroophan
- Life stuff: instagram.com/Aaroophan
Top comments (0)