System Design: Building Scalable Systems
A practical, capstone guide to system design fundamentals — database scaling, caching, load balancing, queues, and the general principles of scalability — synthesizing the database, infrastructure, and messaging guides across this series into the design process actually used to reason about a system's capacity, bottlenecks, and growth.
Table of Contents
- Introduction
- The System Design Process
- Database Scaling
- Caching
- Load Balancing
- Queues and Asynchronous Processing
- Scalability: General Principles
- Putting It Together: Designing a URL Shortener
- Putting It Together: Designing a News Feed
- Estimation and Capacity Planning
- Consistency, Availability, and the CAP Theorem in Practice
- Common Pitfalls
- Quick Reference Table
- Conclusion
Introduction
System design is the discipline of reasoning about how a software system will actually behave at scale — where it will bottleneck, how it fails, and how each of its major components (database, cache, load balancer, queue) is chosen and configured to meet real capacity and reliability requirements. This guide is deliberately a synthesis piece: nearly every building block covered here has its own deep-dive guide elsewhere in this series — SQL Server, PostgreSQL, Cosmos DB/MongoDB, and Redis for data; RabbitMQ, Kafka, and Azure Service Bus for queues; Azure Compute, AWS Compute, and Kubernetes/Helm for the infrastructure that runs and load-balances everything. What this guide adds is the process — how to reason from a set of requirements to a coherent architecture using those building blocks, which is precisely the skill system design interviews and real greenfield architecture decisions actually test.
Requirements → Estimate scale (Section 9) → Identify the bottleneck →
Choose: database strategy (Section 2) + caching (Section 3) +
load balancing (Section 4) + queues (Section 5) →
Validate against consistency/availability trade-offs (Section 10)
1. The System Design Process
Start with requirements, not technology
Functional requirements: what must the system DO (post a message, shorten a URL, serve a feed)
Non-functional requirements: how well must it do it (how many requests/second, what latency,
what consistency guarantee, what availability target)
The single most common mistake in system design — both in real architecture work and in interview settings — is jumping straight to naming technologies ("we'll use Kafka and Redis and Kubernetes") before establishing what the system actually needs to do and at what scale. Every technology choice in this guide should be a direct, traceable answer to a specific requirement, not a default reached for out of familiarity.
Estimate before you design
As covered in depth in Section 9, rough capacity estimates (requests per second, data volume, read/write ratio) determine almost everything downstream — a system serving 100 requests/second and one serving 100,000 requests/second are not the same design problem scaled up; they often require fundamentally different architectural choices at nearly every layer.
Identify the actual bottleneck before reaching for a solution
Is the bottleneck: CPU? Memory? Disk I/O? Network bandwidth? Database write throughput?
Database read throughput? A single point of contention (a lock, a queue)?
Every technique in this guide — caching, read replicas, sharding, load balancing, queuing — solves a specific bottleneck, and applying the wrong technique to a bottleneck it doesn't actually address wastes real engineering effort. A read-heavy system with a database CPU bottleneck benefits enormously from caching; a write-heavy system with the same database CPU bottleneck often needs sharding (Section 2) instead, since caching doesn't help writes.
2. Database Scaling
Vertical scaling: the first, simplest lever
A bigger machine: more CPU, more RAM, faster disks (NVMe SSDs)
As covered in this series' SQL Server and PostgreSQL guides, the simplest response to a database bottleneck is often just a bigger machine — more RAM means more of the working data set fits in the buffer pool/cache (reducing disk I/O), faster disks reduce write latency directly. Vertical scaling has a real ceiling (the largest available machine, and cost that grows non-linearly near that ceiling), but it's frequently the right first move precisely because it requires zero application changes.
Read replicas: scaling read throughput specifically
Primary (handles all writes) → replicates to → Replica 1, Replica 2, Replica 3 (handle reads)
As covered in this series' SQL Server, PostgreSQL, and Azure Compute guides, adding read replicas lets read-heavy workloads scale horizontally by distributing read queries across multiple replicas, while writes still funnel through a single primary — this is the standard first horizontal-scaling move for a database, and it directly addresses read-throughput bottlenecks specifically, doing nothing for write-throughput bottlenecks.
The replication lag trade-off
t=0ms: Write commits on the primary
t=15ms: A read against a replica might still return the OLD value — replication hasn't caught up yet
Read replicas introduce eventual consistency for reads served from them — a read immediately following a write, if routed to a replica, can return stale data for a brief window. This is the same eventual-consistency trade-off covered in this series' Event-Driven Architecture and Microservices guides, showing up here at the database layer specifically; systems that need strict read-your-own-writes consistency either route the specific read back to the primary immediately after a related write, or accept and design around the lag.
Sharding (horizontal partitioning): scaling write throughput and total data volume
Shard by customer_id % 4:
Shard 0: customers 0, 4, 8, 12, ...
Shard 1: customers 1, 5, 9, 13, ...
Shard 2: customers 2, 6, 10, 14, ...
Shard 3: customers 3, 7, 11, 15, ...
Unlike replicas (which each hold a complete copy of the data), sharding splits the data itself across multiple independent database instances, each holding only a subset — this is the technique that scales both write throughput (each shard handles only a fraction of total writes) and total storage capacity (no single machine needs to hold the entire dataset) beyond what any single machine, however large, could handle. This is exactly the partitioning concept covered in depth in this series' Cosmos DB/MongoDB guide, and it applies to relational databases too, just with more manual operational complexity than a document database's built-in partitioning typically requires.
Choosing a shard key: the same high-stakes decision covered in this series' Cosmos DB/MongoDB guide
Good shard key: evenly distributes load, matches the most common query pattern
Bad shard key: creates a "hot shard" (uneven distribution), or forces most queries to fan out across every shard
As emphasized in this series' Cosmos DB/MongoDB guide, the shard key decision is usually the single most consequential, hardest-to-change decision in a sharded system — it determines both whether load distributes evenly and whether common queries can be routed to a single shard (fast) or must scatter-gather across every shard (slow, expensive).
Denormalization: trading redundancy for read speed
Normalized: Order references CustomerId; getting a customer's name requires a JOIN
Denormalized: Order stores a local copy of CustomerName directly, avoiding the JOIN entirely
As covered in this series' Microservices and Cosmos DB/MongoDB guides, deliberately duplicating data to avoid an expensive join or cross-service call is a standard, well-established technique once read performance genuinely matters more than storage efficiency or write-side simplicity — the cost is keeping the duplicated copies eventually consistent (via the domain events covered in this series' Event-Driven Architecture and DDD guides) rather than relying on a single, always-current, normalized source.
Choosing between SQL and NoSQL for a specific component
This is directly the decision framework covered in this series' Cosmos DB/MongoDB guide's comparison section — strong relational consistency and complex ad-hoc querying point toward SQL Server/PostgreSQL; flexible schema, massive horizontal scale, and simpler access patterns point toward a document database; and most real systems of any real size use both, choosing per-component rather than standardizing on one database technology for an entire system.
3. Caching
Why caching is often the single highest-leverage system design lever
Database read: 5-50ms (disk I/O, query planning, network round trip)
Cache read (Redis): under 1ms (in-memory, purpose-built for exactly this)
As covered in this series' Redis guide, moving frequently-read, expensive-to-compute, or expensive-to-fetch data into an in-memory cache can reduce read latency by one to two orders of magnitude, while simultaneously reducing load on the underlying database — for a genuinely read-heavy system, caching is frequently the single change with the best ratio of implementation effort to scalability improvement.
Cache-aside as the standard pattern
// The cache-aside pattern, from this series' Redis guide
var cached = await _cache.GetStringAsync(key);
if (cached is not null) return Deserialize(cached);
var value = await _database.GetAsync(id); // cache miss — fetch from the source of truth
await _cache.SetStringAsync(key, Serialize(value), expiry);
return value;
As covered in this series' Redis guide, cache-aside (check the cache, fall back to the database on a miss, populate the cache for next time) remains the default pattern for the majority of read-heavy caching needs — simple, self-healing, and it degrades gracefully (a cache outage just means every request falls through to the database, slower but still functionally correct).
Where to cache: multiple layers, each solving a different problem
Client-side cache (browser) → eliminates the network round trip entirely for repeat requests
CDN cache (edge) → serves static/cacheable content from a location near the user
Application-level cache (Redis) → avoids expensive database queries/computation
Database query cache → avoids re-planning/re-executing an identical query
A well-designed system typically layers caching at multiple points, each addressing a different part of the request path's latency — this connects to this series' REST guide's discussion of HTTP caching headers (ETag, Cache-Control) for the client/CDN layers, and the Redis guide for the application layer specifically.
Cache invalidation: the genuinely hard part
"There are only two hard things in Computer Science: cache invalidation and naming things."
As covered in this series' Redis guide, choosing a sensible TTL is the simplest invalidation strategy (data becomes stale for at most the TTL duration, then naturally refreshes) — for data that needs to be fresher than a TTL alone can guarantee, explicit invalidation (deleting or updating the cache entry the moment the underlying data changes, via the write-through pattern or an event-driven cache invalidation handler per this series' Event-Driven Architecture guide) is necessary, at the cost of real additional complexity in keeping every code path that mutates the underlying data also correctly invalidating every affected cache entry.
Cache stampede and the thundering herd problem
A popular cache key expires → thousands of concurrent requests all miss simultaneously →
all of them hit the database at once, for the SAME data, right when the database is least prepared for it
As covered in this series' Redis guide, a cache stampede occurs when a hot key's expiration causes a sudden spike of concurrent cache misses, all independently re-computing and re-fetching the same underlying data simultaneously — request coalescing (via HybridCache's built-in stampede protection, or a simple in-flight-request lock) ensures only one request actually goes to the database while the rest wait for and share that single result, rather than all of them redundantly hammering the database at the same moment.
4. Load Balancing
Distributing requests across multiple instances of a service
Client → Load Balancer → [Instance 1, Instance 2, Instance 3, ...]
As covered throughout this series' Azure Compute, AWS Compute, and Kubernetes/Helm guides, a load balancer sits in front of multiple instances of a service, distributing incoming requests across them — this is the fundamental mechanism that makes horizontal scaling of stateless application servers possible at all: without it, adding more instances wouldn't actually distribute load, since clients would have no way to spread their requests across the available instances.
Load balancing algorithms
Round robin: requests distributed evenly, in sequence, across instances
Least connections: routes to whichever instance currently has the fewest active connections
Weighted: some instances receive proportionally more traffic (useful for canary deployments,
per this series' CI/CD Pipelines guide)
Consistent hashing: the same client/key consistently routes to the same instance — important for
session affinity or cache locality
Different algorithms solve different problems — round robin is the simplest, reasonable default for uniform, stateless instances; least-connections better handles instances with genuinely varying per-request processing times; weighted routing is directly how the canary deployment strategies covered in this series' CI/CD Pipelines and Kubernetes/Helm guides gradually shift traffic to a new version.
Layer 4 vs. Layer 7 load balancing
Layer 4 (transport): routes based on IP/port only — fast, protocol-agnostic, no visibility into HTTP content
Layer 7 (application): routes based on HTTP-level detail — URL path, headers, cookies — enabling
content-based routing, but with more per-request overhead
As referenced in this series' Kubernetes/Helm guide's Ingress discussion, a Layer 4 load balancer (a basic TCP/UDP load balancer) is faster and simpler but can't make routing decisions based on HTTP content; a Layer 7 load balancer (an Application Load Balancer, an Ingress controller) can route based on URL path or hostname — directly enabling the API Gateway pattern covered in this series' Microservices guide, where a single entry point routes different paths to entirely different backend services.
Health checks as the mechanism that keeps load balancing correct
As covered in this series' Health Checks guide, a load balancer only routes traffic to instances currently passing their configured health check — this is what prevents a load balancer from continuing to send traffic to an instance that's crashed, is overloaded, or has lost connectivity to a critical dependency, and it's the same underlying mechanism whether the load balancer is a cloud provider's managed service, a Kubernetes Service, or an Ingress controller.
Sticky sessions and their cost
Sticky session: the SAME client always routes to the SAME instance (via a cookie or consistent hashing)
Sticky sessions are sometimes necessary for stateful protocols (a raw WebSocket connection, or an application holding session state in local memory rather than a shared store) — but as covered in this series' SignalR guide's discussion of scaling out via a Redis backplane, the better long-term solution is usually making application instances genuinely stateless (storing session state in Redis rather than in-process) so that any instance can serve any request, which is both simpler to reason about and considerably easier to scale and rebalance than a sticky-session-dependent architecture.
5. Queues and Asynchronous Processing
Decoupling request handling from slower, non-immediate work
Synchronous: client waits for the ENTIRE operation (including slow, non-essential parts) to complete
Queue-based: client gets a fast response once the essential part is done; slower work happens asynchronously
As covered in this series' Background Services, RabbitMQ, Kafka, and Event-Driven Architecture guides, introducing a queue between a fast-path request handler and slower downstream work (sending an email, generating a report, processing an uploaded file) is one of the most effective ways to keep a system's user-facing latency low, independent of how long the full end-to-end business process actually takes.
Smoothing traffic spikes
Without a queue: a traffic spike directly overwhelms downstream processing capacity
With a queue: a traffic spike fills the queue; downstream workers process it at a sustainable,
steady rate, catching up over time rather than falling over immediately
As covered in this series' RabbitMQ and Kafka guides, a queue acts as a buffer, absorbing a burst of incoming work and letting it be processed at whatever steady rate the downstream system can actually sustain — this is a distinct, valuable benefit from decoupling alone, directly addressing the "sudden spike overwhelms the system" failure mode that a purely synchronous architecture is especially vulnerable to.
Choosing a queue/messaging technology for the actual need
Simple task distribution, flexible routing: RabbitMQ
High-volume event streaming, replay needed: Kafka
Managed service within Azure, enterprise features: Azure Service Bus
This is the exact decision framework covered in depth across this series' RabbitMQ, Kafka, and Azure Service Bus guides — worth restating here specifically that the choice should follow from the actual requirement (throughput, replay need, routing complexity, operational preference), the same principle underlying every technology choice throughout this guide.
Backpressure: what happens when producers outpace consumers
Queue depth growing continuously, not draining → consumers can't keep up with producers
A queue that's growing without bound is a symptom, not a solution — it means the downstream processing capacity is genuinely insufficient for the actual incoming rate, and the queue is only delaying, not solving, an eventual failure (running out of memory/storage, or accumulating unacceptable processing latency). The real fixes are scaling out consumers (per this series' Background Services and Kubernetes/Helm guides' autoscaling coverage), or — if the incoming rate is genuinely unsustainable long-term — applying rate limiting or load shedding at the point of ingestion rather than letting the queue absorb an ever-growing backlog indefinitely.
6. Scalability: General Principles
Horizontal vs. vertical scaling, revisited at the whole-system level
Vertical: bigger machines — simpler, but has a real ceiling and doesn't improve fault tolerance
Horizontal: more machines — requires statelessness/partitioning, but scales further and improves
fault tolerance (losing one of many instances is less catastrophic than losing the only one)
This is the same distinction covered per-component throughout this series (databases in Section 2, application servers via load balancing in Section 4) — the general system design principle is that horizontal scaling requires deliberate architectural support (statelessness, partitioning, load balancing) that vertical scaling doesn't, but horizontal scaling is what actually removes the hard ceiling vertical scaling eventually hits.
Statelessness as the property that makes horizontal scaling of application servers possible
// ❌ Stateful — session data held in this specific instance's memory
private static Dictionary<string, UserSession> _sessions = new();
// ✅ Stateless — session data lives in a shared store (Redis, per this series' Redis/SignalR guides);
// ANY instance can handle ANY request
As covered in this series' Redis, SignalR, and Background Services guides, an application server that holds no meaningful state in its own local memory (storing session data, cache, and coordination state in shared, external stores instead) can be freely, arbitrarily scaled out or replaced — this single architectural property is what makes load balancing (Section 4) and autoscaling (per this series' Kubernetes/Helm guide) actually work correctly, rather than routing a specific client to a specific "sticky" instance that happens to hold state nothing else has access to.
Single points of failure
Load balancer with only ONE instance → the load balancer itself becomes a single point of failure
Database with no replica → the database itself becomes a single point of failure
Every component in a system design should be examined for whether it's a genuine single point of failure — this recursive question applies at every layer, including components (like the load balancer itself) that exist specifically to provide redundancy for something else; a system is only as resilient as its least-redundant critical component, which is why this series' Azure Compute, AWS Compute, and Kubernetes/Helm guides all cover multi-instance, multi-replica configurations even for infrastructure that might initially seem like it should just be "one thing."
CDNs for reducing load and latency at the edge
Static/cacheable content (images, JS bundles, API responses with long-lived Cache-Control headers)
served from a CDN edge location NEAR the user, never reaching the origin servers at all
As referenced in this series' Cloud Cost Optimization guide, a CDN both improves latency (content served from a location physically near the user, rather than from a potentially distant origin data center) and reduces load on origin infrastructure — genuinely useful for any content that's cacheable and doesn't need to be freshly computed per request.
Rate limiting as protection against both abuse and legitimate overload
// per this series' ASP.NET Core guide
builder.Services.AddRateLimiter(options => { /* fixed window, sliding window, token bucket, concurrency */ });
As covered in this series' ASP.NET Core guide, rate limiting protects a system from being overwhelmed regardless of whether the excess traffic is malicious abuse or simply legitimate demand exceeding current capacity — it's a genuine scalability tool, not purely a security one, since it's what keeps a system degrading gracefully (rejecting excess requests cleanly) rather than falling over entirely under load it wasn't provisioned to handle.
7. Putting It Together: Designing a URL Shortener
Requirements
Functional: given a long URL, generate a short code; given a short code, redirect to the original URL
Non-functional: 100M new URLs/month, 10:1 read:write ratio (redirects far more common than creations),
redirects must be very low latency
The design, built layer by layer
1. Database: a simple key-value mapping (short_code → long_url) — no complex relational queries needed,
so a document/key-value store (per this series' Cosmos DB/MongoDB guide) or even a simple relational
table (per this series' SQL Server/PostgreSQL guides) both work; given the read-heavy ratio,
read replicas (Section 2) help scale the redirect path specifically.
2. Caching: redirects are the hot path (10:1 read:write) — cache short_code → long_url mappings in
Redis (per this series' Redis guide), since a redirect lookup is exactly the kind of high-frequency,
simple key lookup Redis excels at. This likely handles the vast majority of read traffic without
ever touching the database.
3. Load balancing: the redirect service is entirely stateless (Section 6) — any instance can serve any
redirect — so a straightforward Layer 7 load balancer (Section 4) in front of many instances scales
horizontally with no special coordination needed.
4. Queue: not obviously necessary for the core path — URL creation and redirection are both fast,
synchronous operations with no genuinely slow, deferrable work involved.
Why this system doesn't need a queue, and that's a legitimate design decision
Not every system design needs every building block — this series' consistent theme (echoed in the Microservices and Event-Driven Architecture guides) of matching architecture to genuine need applies directly here: a URL shortener's operations are fast and synchronous by nature, so introducing a queue would add complexity without solving an actual problem this specific system has.
8. Putting It Together: Designing a News Feed
Requirements
Functional: users post updates; users see a feed of updates from people they follow
Non-functional: 10M users, some users ("celebrities") followed by millions of others — a genuinely
different scale problem than the URL shortener above
Why this is a meaningfully harder design problem: the "celebrity problem"
Regular user posts an update → needs to reach, say, 200 followers' feeds
Celebrity posts an update → needs to reach 10 MILLION followers' feeds
This asymmetry — the same "post an update" operation having wildly different fan-out costs depending on who's posting — is a classic system design challenge, and it's exactly why a single, uniform approach doesn't work well here.
Fan-out on write vs. fan-out on read
Fan-out on write: when a user posts, immediately push the update into every follower's pre-computed feed
(fast reads, since the feed is pre-built — but a celebrity's post means millions of writes)
Fan-out on read: a user's feed is computed on-demand at read time, by querying everyone they follow
(fast, cheap writes — but reading a feed means querying potentially many people's posts live)
This is where the queue-based, asynchronous processing covered in Section 5 (and this series' RabbitMQ, Kafka, and Event-Driven Architecture guides) becomes genuinely necessary rather than optional: fan-out on write is implemented as an asynchronous background process (per this series' Background Services guide) triggered by a published "post created" event, specifically because synchronously updating millions of followers' feeds within the original post-creation request would make that request unacceptably slow.
The hybrid approach real systems typically use
Regular users: fan-out on write (fast reads for the common case, manageable write cost)
Celebrities: fan-out on read (avoid the catastrophic write amplification of pushing to millions
of feeds); their posts are merged into a follower's feed at read time instead
This hybrid — choosing the fan-out strategy per-author based on follower count, rather than one uniform strategy for everyone — is a genuinely common real-world pattern, directly illustrating this guide's core message: system design is about identifying where a single, uniform approach breaks down under genuine scale, and applying a more nuanced, targeted solution specifically where it's needed, rather than uniformly over-engineering every part of the system to handle its most extreme case.
The rest of the design, using this guide's other building blocks
Database: posts stored relationally or in a document store (per this series' database guides);
pre-computed feeds stored in a fast key-value store (Redis, per this series' Redis guide)
Caching: a user's feed itself is effectively a cache — the fan-out-on-write process IS a form of
pre-computed caching
Queue: Kafka (per this series' Kafka guide) is a particularly natural fit here specifically because
multiple independent consumers (the feed fan-out service, a notifications service, an
analytics service) all need to react to the same "post created" event independently
Load balancing: the read-serving API layer is stateless and horizontally scaled, per Sections 4 and 6
9. Estimation and Capacity Planning
Back-of-envelope estimation as a genuine, standard practice
100M users, 10% daily active → 10M daily active users
Each posts 2 times/day on average → 20M posts/day → ~230 posts/second average
(peak traffic is typically 3-5x average — design for roughly 700-1,000 posts/second at peak)
Rough, order-of-magnitude estimates — not precise numbers — are what actually drive early architectural decisions: whether a single database can handle the write load, whether caching is worth the complexity, whether sharding is genuinely necessary yet. Getting the estimate approximately right (within an order of magnitude) is what matters; false precision at this stage is generally not worth the effort.
Storage estimation
20M posts/day × 500 bytes/post average × 365 days × 5 years = ~18 TB over 5 years
(before accounting for images/media, which typically dominates actual storage far more than text)
Estimating data growth over a multi-year horizon informs whether a single database instance's storage ceiling will realistically be reached, and on what timeline — directly informing when sharding (Section 2) becomes a genuine near-term necessity versus a premature optimization for a problem that won't materialize for years.
Read/write ratio as a primary driver of architecture choice
Read-heavy (10:1 or higher): caching (Section 3) and read replicas (Section 2) are high-leverage
Write-heavy (closer to 1:1, or write-dominant): sharding (Section 2) and queue-based write buffering
(Section 5) become more directly relevant
This single ratio, established early via rough estimation, meaningfully shapes which of this guide's techniques deserve the most design attention — a system that's overwhelmingly read-heavy (the URL shortener from Section 7) benefits enormously from caching; a system with substantial write volume (the news feed's celebrity fan-out problem from Section 8) needs write-path techniques that caching alone doesn't address.
10. Consistency, Availability, and the CAP Theorem in Practice
The CAP theorem, briefly
In the presence of a network partition (a genuine, unavoidable possibility in any distributed system), a system must choose between consistency (every node sees the same data at the same time) and availability (every request receives a response, even if it might not reflect the most recent write) — you cannot have perfect versions of both simultaneously during a partition.
Why this is a genuinely practical concern, not just theory
As covered in this series' Cosmos DB/MongoDB guide's consistency-level discussion, this isn't an abstract academic point — Cosmos DB's five explicit, tunable consistency levels (Strong through Eventual) are a direct, practical expression of exactly this trade-off, letting a system choose its position on the consistency/availability spectrum per operation, rather than being forced into one fixed choice for the entire system.
Where this trade-off shows up throughout this guide's other techniques
Read replicas (Section 2): choosing availability/read-scale over strict consistency (replication lag)
Caching (Section 3): choosing availability/speed over strict freshness (a TTL-bound staleness window)
Eventual consistency generally: the practical, default choice throughout event-driven and microservices
architectures, per this series' Event-Driven Architecture and Microservices guides
Nearly every scalability technique covered in this guide — replication, caching, sharding, asynchronous queue-based processing — trades some degree of strict, immediate consistency for availability, latency, or throughput; recognizing this explicitly, rather than discovering it as a surprising side effect later, is what separates a deliberate, well-reasoned system design from one that accidentally introduces subtle correctness bugs because a consistency trade-off was never consciously made.
Choosing where strict consistency genuinely matters
A bank account balance: strict consistency matters enormously — eventual consistency here is
genuinely dangerous, not just an inconvenience
A social media "like" count: eventual consistency is entirely acceptable — nobody is meaningfully
harmed by a brief, momentary undercount
The practical skill in applying the CAP theorem isn't memorizing it — it's correctly identifying, for each specific piece of data in a system, whether strict consistency is a genuine business requirement or whether eventual consistency is an entirely acceptable, even preferable, trade-off for the availability and performance it buys — a judgment call that should be made deliberately and explicitly for each component, not applied uniformly across an entire system by default in either direction.
11. Common Pitfalls
| Pitfall | Why it hurts | Better approach |
|---|---|---|
| Naming technologies before establishing requirements | Solutions get chosen that don't actually match the real bottleneck | Estimate scale and identify the actual bottleneck first, per Sections 1 and 9 |
| Applying caching to a write-heavy bottleneck | Caching helps reads; it does nothing for write throughput | Match the technique to the actual bottleneck (sharding/queuing for writes, caching/replicas for reads) |
| Choosing a shard key without considering query patterns | Creates hot shards or forces expensive scatter-gather queries | Choose a shard key matching the most common, most performance-critical query pattern |
| No cache invalidation strategy beyond "it'll expire eventually" | Data can be stale for longer than acceptable for some use cases | Choose TTL vs. explicit invalidation deliberately, per the actual freshness requirement |
| Stateful application servers | Breaks horizontal scaling and complicates load balancing | Keep session/coordination state in a shared store (Redis), not local memory |
| Uniform fan-out/processing strategy applied regardless of scale (the celebrity problem) | A single approach that works for the common case catastrophically fails for the extreme case | Identify where uniform approaches break down at genuine scale; apply targeted, hybrid solutions there specifically |
| Treating every piece of data as needing strict consistency by default | Unnecessarily sacrifices availability/performance where eventual consistency would be entirely acceptable | Make the consistency-vs-availability trade-off deliberately, per data type, per Section 10 |
| Designing for a scale the system will never actually reach | Wastes engineering effort and adds complexity with no real payoff | Estimate real, near-term scale; add sharding/complex distributed techniques only once genuinely justified |
Quick Reference Table
| Building block | Solves | Covered in depth in |
|---|---|---|
| Vertical scaling | Simple capacity increase, no application change needed | SQL Server, PostgreSQL guides |
| Read replicas | Read throughput scaling, at the cost of replication lag | SQL Server, PostgreSQL, Azure Compute guides |
| Sharding | Write throughput and total storage scaling | Cosmos DB/MongoDB guide |
| Caching (cache-aside) | Read latency and database load reduction | Redis guide |
| Load balancing | Distributing requests across horizontally-scaled instances | Azure Compute, AWS Compute, Kubernetes/Helm guides |
| Health checks | Keeping load balancing routing only to genuinely healthy instances | Health Checks guide |
| Queues | Decoupling, smoothing traffic spikes, asynchronous processing | RabbitMQ, Kafka, Azure Service Bus, Background Services guides |
| Statelessness | The property enabling horizontal scaling and load balancing to work at all | Redis, SignalR guides |
| CDN | Edge caching for latency and origin load reduction | Cloud Cost Optimization guide |
| Rate limiting | Graceful degradation under abuse or excess legitimate load | ASP.NET Core guide |
| CAP theorem trade-offs | The consistency/availability trade-off underlying most scaling techniques | Cosmos DB/MongoDB, Event-Driven Architecture, Microservices guides |
Conclusion
System design is fundamentally about matching a small set of well-understood techniques — vertical and horizontal database scaling, caching, load balancing, and asynchronous queuing — to a specific system's actual, estimated requirements, rather than reaching for the most sophisticated available tool by default. Every building block covered in this guide has its own deep, dedicated treatment elsewhere in this series, and the genuine skill this guide has focused on is the reasoning process that connects them: estimate scale, identify the real bottleneck, and apply the specific technique that addresses that bottleneck — caching and replicas for read-heavy problems, sharding and queues for write-heavy ones, load balancing and statelessness as the foundation that makes horizontal scaling of anything possible at all.
The two worked examples in this guide — a URL shortener needing almost none of this guide's heavier techniques, and a news feed whose celebrity-follower asymmetry demands a genuinely hybrid, non-uniform architecture — are meant to illustrate the same lesson from opposite directions: good system design isn't about applying every available technique everywhere, it's about correctly identifying which techniques a specific system's specific bottlenecks actually require, and having the judgment (echoed throughout this series' Microservices, DDD, and Vertical Slices guides) to leave out everything else.
Found this useful? Feel free to star the repo, open an issue with corrections, or share the bottleneck estimation that correctly predicted which piece of your system would actually need to scale first.
Top comments (1)
The level of detail here is impressive. ~
I especially like that you’re not just showing the implementation, but walking through the reasoning behind the design choices and tradeoffs.
That makes this much more useful than a typical “here’s how I built it” post.