Why NFS, AFS, Coda, and GFS Make Different Trade-Offs for Different Failures
A technical comparison of Sun NFS, AFS, Coda, and GFS, examining how transparency, caching, scalability, availability, replication, consistency, reliability, and security constraints shaped each distributed file system.
At first, this looks like a comparison problem.
Put Sun NFS, AFS, Coda, and GFS in four columns.
Add transparency, performance, scalability, availability, reliability, security.
Give each system a score.
Highest total ✨wins✨
Perfectly reasonable.
And almost immediately wrong.
The problem is that these systems are not four contestants solving the same problem with different levels of competence. They entered the room carrying different fears.
NFS is worried about remote files feeling remote.
AFS is worried about thousands of clients exhausting the servers.
Coda is worried about the network simply disappearing.
GFS assumes machines will fail routinely and asks how to keep massive data processing moving anyway.
So the useful question is not: Which distributed file system is best?
It is: Best under which constraint?
Transparency Looks Easy Until the Network Starts Having Opinions
The first instinct is simple: if a file is remote, hide that fact.
Sun NFS takes this intuition seriously. Its protocol is explicitly designed for transparent remote access , with RPC carrying operations across the network and XDR providing a common representation across different machines, operating systems, network architectures, and transport protocols.
NFS is the diplomat here. It wants everybody to speak file system without caring too much about where the bytes live.
Its stateless server follows the same philosophy. The server avoids maintaining protocol state about its clients, which makes recovery after failure dramatically simpler: clients can retry operations instead of requiring the server to reconstruct an elaborate shared history.
That feels beautifully clean.
Then scale arrives.
AFS wants the same kind of location-transparent experience, but it expects thousands of workstations. If every read keeps dragging Vice, the trusted server side, back into the conversation, Vice eventually becomes the colleague who is somehow invited to every meeting.
So AFS gives Venus, the client cache manager, more responsibility.
Venus caches entire files locally. Reads and writes can then happen against the cached copy, while Vice keeps the responsibilities that actually require server authority. The system preserves a homogeneous, location-transparent namespace while deliberately pushing work toward clients.
The simple mental model was:
transparency means hiding the network.
AFS reveals the deeper model:
transparency also depends on deciding who must participate in each operation .
That distinction becomes surprisingly expensive later.
Mobility Exposes the Assumption AFS Was Still Making
AFS caching solves a lot.
But it quietly assumes that the server is still reachable when the client actually needs it.
That assumption feels harmless until a portable workstation leaves the network.
Now the cached copy is sitting there, perfectly usable, while the system technically has nobody authoritative to talk to.
The obvious conservative answer is to stop.
No server, no update.
Safe. Consistent. Miserable.
Coda refuses that bargain.
It inherits much of the AFS model but introduces two complementary mechanisms: server replication and disconnected operation. During disconnected operation, a caching site temporarily assumes the role of a replication site, allowing useful work to continue even when the shared repository is inaccessible. The design explicitly identifies portable workstations as a major use case.
This is where the architecture gets uncomfortable.
If a disconnected client can modify data, another client may modify its own copy elsewhere.
Now both copies are sincere.
Both have history.
Both think they are right.
Coda chooses an optimistic strategy because preventing all such divergence would destroy much of the availability it is trying to create. Conflict detection and later reconciliation become the price of continued operation.
That is the hidden constraint:
Mobility is not just remote access from another machine. It is maintaining useful authority when the normal authority is unreachable.
Once that requirement appears, “just stay consistent” stops being a complete answer.
Performance Stops Being a Number
It is tempting to ask which system is fastest.
Fast at what?
GFS makes this question impossible to ignore.
Its world contains large files, streaming reads, append-heavy writes, commodity machines, many clients, and applications that care more about high sustained bandwidth than low latency.
Under those assumptions, traditional file-system instincts start looking suspicious.
GFS uses a single master for metadata and many chunkservers for data. Clients ask the master where chunks live, then communicate directly with chunkservers for the actual data path. The master knows where everything is, but it very deliberately refuses to carry everybody’s luggage.
Large chunks reduce metadata overhead and repeated master interaction.
Replication assumes failures will happen.
Record append reflects the actual workload instead of pretending random overwrite is equally important.
This is specialization without apology.
AFS reaches performance from the opposite direction.
Its enemy is server interaction.
Whole-file transfer and client caching mean Venus can satisfy much of the workload locally. Callback-based cache validation further reduces the need for clients to repeatedly ask servers whether cached data is still valid. Howard and colleagues identify whole-file transfer and caching as central to AFS’s ability to scale gracefully.
So GFS says:
make the data path bypass central metadata control.
AFS says:
make repeated data access bypass the server entirely.
NFS stays more conservative. Its stateless RPC model gives it simplicity and portability, but that same simplicity keeps the client-server relationship comparatively chatty. Even the protocol specification acknowledges that server simplicity can become awkward when clients want richer file-system semantics.
Nobody is “faster.”
They are removing different bottlenecks.
Scalability Is Usually Someone Becoming Tired
Scale has a habit of revealing whichever component everyone politely assumed would cope.
In AFS, that component is Vice.
So Venus takes work away from it.
In GFS, that component could be the master.
So clients cache metadata and transfer data directly with chunkservers.
In NFS, statelessness keeps servers conceptually simple, but repeated remote interactions remain part of the model.
In Coda, scale is complicated further because availability mechanisms themselves create state: replicas must be tracked, cached versions must be reconciled, and disconnected clients eventually have to return carrying edits made in another reality.
This is why scalability is not just “add more machines.”
Adding machines adds:
- more metadata,
- more coordination,
- more replicas,
- more failure combinations,
- more opportunities for stale state,
- more people asking the same exhausted authority for answers.
The architecture scales when it decides which conversations no longer need to happen.
Availability and Reliability Look Similar Until Something Actually Fails
Coda and GFS both care deeply about failure, but their definitions of surviving it are different.
Coda asks: Can the user keep working?
Server replication protects against some server and network failures. Disconnected operation covers the much uglier case where no appropriate server can be reached at all.
GFS asks: Can the storage service keep functioning while components routinely die?
Its answer includes chunk replication, monitoring, master recovery mechanisms, stale-replica detection, and checksums. Google explicitly designed GFS around the assumption that component failures are normal in clusters built from inexpensive commodity hardware.
Same word: availability.
Different anxiety.
And reliability gets even more specific.
A replica existing somewhere is not enough.
What if it missed an update?
What if its disk corrupted the data?
What if a client reads from something that looks alive but belongs to yesterday?
GFS uses chunk version numbers to identify stale replicas and checksums to detect corrupted data. Replication is therefore not just duplication; replicas must remain trustworthy enough to participate. The verification work around GFS further examines its relaxed and eventual consistency behaviour, showing that consistency here is an explicit system property rather than a vague promise.
The replica does not get authority merely because it still has a pulse.
That is a useful distinction.
Data Integrity Reveals Why Availability Has a Price
The seductive model of replication is: make three copies and relax.
Distributed systems do not relax.
GFS coordinates mutations across replicas and uses versioning and checksumming to identify stale or damaged copies. Its relaxed consistency model is not equivalent to abandoning integrity; instead, the system defines what applications may observe and shapes the interface around workloads that can tolerate those guarantees.
Coda makes a different trade.
It wants updates to remain possible across disconnection and partitions.
That means temporary divergence can be legitimate.
The system therefore needs mechanisms for detecting conflicts and reconciling them later.
This can feel like integrity has been weakened.
A better way to see it is that the definition of integrity changed because the availability requirement changed.
GFS tries hard to keep replicas participating in one coordinated storage reality.
Coda sometimes permits two realities to exist temporarily because refusing the second reality would mean refusing the user.
Neither choice is free.
That is the part the tidy architecture diagrams usually leave out.
Security Is Where Old Trust Assumptions Start Showing
NFS began with portability, transparency, and simple recovery as dominant concerns.
Its classic design consequently places considerable trust in the surrounding authentication environment. The protocol’s early security model relies on mechanisms such as AUTH_UNIX, while the stateless server must repeatedly reason about permissions without maintaining rich client state.
That trade becomes increasingly uncomfortable as the environment gets less friendly.
AFS and Coda operate with a more explicit trusted-server model, and Coda’s design includes authentication and encrypted communication because mobility, caching, and replication make misplaced trust especially expensive.
GFS is different again. Its foundational design discussion spends enormous attention on failure tolerance, replication, throughput, and consistency, but adversarial multi-tenant security is not one of its central design axes.
That does not automatically mean “insecure.”
It means something more important when comparing architectures: absence of emphasis is not evidence of strength.
A system deserves credit for the problem it actually solves, not for requirements we wish it had discussed.
The Leaderboard Was the Wrong Abstraction
Once all these constraints are visible, the rankings become almost embarrassingly contextual.
Sun NFS is compelling when the problem is transparent remote access across heterogeneous systems and the architecture should stay comparatively simple. Its RPC/XDR model and statelessness follow directly from that goal.
AFS becomes compelling when the problem is large shared-workstation populations and servers cannot remain involved in every access. Whole-file caching, Venus, Vice, callbacks, and volumes are consequences of scale.
Coda becomes compelling when connectivity itself cannot be trusted. Server replication and disconnected operation follow from the requirement that users continue working through failures and mobility.
GFS becomes compelling when the environment is large, data-intensive, append-heavy, and built from hardware expected to fail. Large chunks, a metadata master, direct client-to-chunkserver communication, replication, checksums, and relaxed semantics all follow from that workload.
That is the deeper pattern.
Architecture starts looking strange when we stare only at the solution.
Once we reconstruct the constraint that came before it, the weird parts become almost inevitable.
The stateless server was not minimalism for vibes.
Whole-file caching was not an optimization somebody added because caches are fashionable.
Disconnected operation was not a feature bolted onto AFS.
GFS did not relax traditional semantics because correctness became unfashionable.
Each system moved because its original mental model eventually met a requirement it could no longer ignore.
That is what mature architecture means to The Origami Software Engineer:
Not a collection of universally correct choices,
but a record of which assumptions survived contact with the constraints.
That’s not failure.
That’s evolution.
The “I liked this” Starter Pack:
Don’t let your fingers get lazy now.
- Like : It tells me this was worth writing.
- A Comment: Tell me your thoughts, your favorite snack, or a better title for this blog.
- Boost it: Especially with that one developer who definitely needs this.
Thanks for being here. It genuinely helps more than you know!
— Aaroophan Varatharajan
Find me elsewhere:
- Professional stuff: linkedin.com/in/Aaroophan
- Code stuff: github.com/Aaroophan
- UI stuff: aaroophan.dev/Aaroophan
- Life stuff: instagram.com/Aaroophan
Top comments (0)