DEV Community

Zeeshan Ali
Zeeshan Ali

Posted on • Originally published at blog.stackademic.com on

Every Architectural Decision Is a Bet on Which Failure You Can Live With

At some point in every technical interview, someone will say “it depends on the trade-offs” and then look at you like they have just said something profound. They have not. Saying “it depends on the trade-offs” without being able to name the specific failure each choice produces is roughly equivalent to a doctor saying “medicine has side effects” and then leaving the room. Technically accurate, completely useless, and slightly insulting to everyone present.

The reason trade-off thinking is hard to teach is that most resources explain it backwards. They start with the options and describe what each one is good at. Kafka is good for high throughput. PostgreSQL is good for strong consistency. MongoDB is good for flexible schemas. This is the comparison table approach, and it produces engineers who can recite characteristics but cannot reason about failure, which is the only thing that actually matters in production.

The correct starting point is not the options. It is the failure modes.

Every architectural decision is really a question about which failure you are willing to accept, under what conditions it will appear, and how badly it will hurt when it does. The choice between two reasonable options is always a choice between two different ways your system can break. The skill is knowing which breakage you can recover from and which one ends your evening.

A framework that actually works under pressure

Before getting into specific decisions, here is the four-question framework that every trade-off analysis in this series will use. It sounds simple because it is. Simple frameworks survive contact with a whiteboard at 10am when you have not slept well.

What are we optimising for? Not in the abstract, but concretely: latency, throughput, consistency, operational simplicity, cost, developer velocity. Pick the thing that actually matters for this system.

What are we sacrificing? The honest answer to what the first question costs you. Every optimisation trades against something else. Name it explicitly or it will name itself later, in production, at an inconvenient hour.

What breaks first? The specific failure mode, the conditions under which it appears, and roughly at what scale or load it becomes a real problem rather than a theoretical one.

How do we recover? The concrete path back to normal. Not “add more servers” but what specifically you add, what the recovery sequence looks like, and whether recovery requires a maintenance window or can happen live.

The rest of this article uses that framework to walk through the tensions that run through almost every architectural decision. The subsequent pieces in this series apply it to specific choices: Kafka versus RabbitMQ, SQL versus NoSQL, the outbox pattern, saga versus 2PC, sharding, caching, event sourcing. Each one structured around failure, not features.

Continue reading on Stackademic »

Top comments (0)