System design interviews ask a very small set of recurring questions dressed up in different scenarios - design a URL shortener, design a chat app, design a rate limiter. Underneath nearly all of them is the same handful of building blocks, combined differently depending on the specific problem. This post covers those building blocks individually - what each one is, an analogy, an example, and how they combine - so the interview stops being about memorizing famous architectures and starts being about reasoning from a shared set of first principles.
Most system design content jumps straight to "here's how to design Twitter," which teaches a specific answer without teaching the reasoning that produced it. Understanding each component on its own - what problem it solves, what it costs, when it's the wrong tool - is what actually transfers to a new, unfamiliar problem in an interview or in production work. This post deliberately stays at that component level.
Component 1: Load Balancer
A load balancer sits in front of multiple servers and distributes incoming requests across them, so no single server gets overwhelmed while others sit idle.
The Analogy: Think of a host at a busy restaurant. Customers don't walk in and pick their own table - the host looks at which servers have capacity and seats accordingly, so one waiter isn't drowning in tables while three others stand around. A load balancer does the same thing for incoming requests and backend servers.
Without a load balancer, a client connects directly to a single server - that server becomes a single point of failure, and its individual capacity becomes the entire system's capacity ceiling. With a load balancer in front of several servers, traffic spreads across all of them, and if one server goes down, the load balancer simply stops routing to it while traffic continues across the remaining servers.
Common distribution strategies include round robin, where requests rotate evenly across servers in order - simple, and it works well when servers are similar and requests are roughly similar in cost. Least connections routes new requests to whichever server currently has the fewest active connections, which works better when request processing time varies significantly. Health checks have the load balancer periodically ping each server, automatically removing any server that stops responding from rotation until it recovers.
How it stands alone vs. as part of a whole system: On its own, a load balancer solves exactly one problem - distributing traffic across multiple identical or similar servers. It does not solve data consistency, does not make individual requests faster, and provides no benefit at all with only one server behind it. Its real value only appears once there are multiple servers that need traffic spread across them - which is precisely why it's usually one of the first components added when a single server can no longer handle load alone.
Component 2: Cache
A cache is a smaller, faster storage layer that keeps a copy of frequently accessed data, so repeat requests can be served without going back to the slower, authoritative source every time.
The Analogy: Think of keeping frequently used spices on the counter instead of in a pantry down the hall. Walking to the pantry every single time is correct but slow; keeping a copy of the things reached for constantly, right at hand, trades a small amount of staleness risk - the pantry might get restocked with a fresher jar you don't know about yet - for a large amount of speed.
Without a cache, every single request pays the full cost of a database round trip, even for data that hasn't changed since the last request five seconds ago. With a cache, the application checks the cache first: a cache hit means the data was found and gets returned immediately without touching the database at all, while a cache miss means the data wasn't cached, gets fetched from the database, and then gets stored in the cache for next time.
The real cost of caching is staleness. If the underlying data changes but the cache still holds the old value, the cache now serves incorrect data until it expires or is explicitly invalidated. A few common strategies manage this: TTL, or time to live, lets cached data automatically expire after a set duration - simple, but data can be stale for up to that entire duration. Write-through updates the cache and the database at the same time on every write - the cache stays consistent, but every write becomes slightly more expensive. Explicit invalidation has application code actively remove or update the specific cache entry the moment underlying data changes - the most accurate approach, but it requires the application to remember to do this correctly everywhere that data can possibly change.
How it stands alone vs. as part of a whole system: A cache alone speeds up reads specifically - it does nothing for writes, and it introduces a genuine new failure mode, serving stale data, that didn't exist before. As part of a whole system, caching is usually layered at multiple levels simultaneously - a CDN caching static assets, an in-memory cache caching frequent database query results, and a browser cache on the client side - each solving the same underlying problem at a different point in the request path.
Component 3: Message Queue
A message queue sits between a producer, something creating work, and a consumer, something processing that work, holding messages until the consumer is ready, so the two sides don't need to move at the same pace or even be running at the same time.
The Analogy: This is the same analogy covered in more depth in an earlier post on this blog about Azure Service Bus - a restaurant order slip. The waiter doesn't wait next to the stove for a dish to finish cooking; the order becomes a slip that queues behind other orders, and whichever cook is free picks it up next. The waiter and the cook move at their own pace, decoupled by the slip sitting between them.
Without a queue, a producer calling a consumer directly means that if the consumer is slow or temporarily down, the producer either fails outright or blocks waiting - the two are tightly coupled in time. With a queue between them, the producer drops a message and moves on immediately, and the consumer picks up messages whenever it's ready, at its own pace. Neither side needs the other to be available or fast at the same moment.
Azure Service Bus, covered in an earlier post on this blog, is a real production implementation of exactly this pattern - queues for a single producer and single consumer, topics for one producer reaching many independent consumers, and a dead-letter queue specifically for messages that fail processing repeatedly, so a bad message doesn't block every healthy message behind it.
How it stands alone vs. as part of a whole system: A queue alone doesn't process anything - it only holds messages until something consumes them, meaning at least one consumer must exist and actually be running for the queue to be useful at all. As part of a whole system, queues are the standard way to connect a fast, user-facing action like placing an order to slower background work like sending a confirmation email, updating inventory, or logging analytics, without making the user wait for all of that slower work to finish first.
Component 4: Database - SQL vs NoSQL
The database is the persistent storage layer where a system's actual data lives, queried and updated over the system's lifetime. The SQL versus NoSQL choice is really a question of which storage shape matches the actual shape of the data and its access patterns.
The Analogy: This is the same analogy covered in more depth in an earlier post on this blog - a filing cabinet, representing SQL and its fixed-schema rows, versus a box of index cards, representing NoSQL and its flexible-shape documents. The filing cabinet enforces that every folder has the same fields, in the same order - useful when relationships between data matter and consistency is critical. The index card box allows each card to carry different fields entirely - useful when the shape of the data varies or changes frequently, and rigid structure would slow things down rather than help.
Choose SQL, the relational option, when data has a consistent, well-defined shape, when relationships between entities matter through foreign keys and joins across tables, and when strong consistency and transactions genuinely matter - a bank transfer either fully succeeds or fully fails, never half-completes. Choose NoSQL - document, key-value, or wide-column stores - when data shape varies significantly between records, when horizontal scale across many servers matters more than strict relational consistency, and when the access pattern is simple, mostly key-based lookups rather than complex joins.
An earlier post on this blog compared Azure SQL, Cosmos DB, and Azure Table Storage directly as a real example of this decision - Azure SQL for this blog's own Posts table, since it has a consistent, relational shape, Cosmos DB for operational and log data with genuinely variable shape, and Table Storage for high-volume, simple key-based lookups where cost matters more than query power.
How it stands alone vs. as part of a whole system: A database alone is just storage - it doesn't solve traffic distribution, doesn't make repeat reads fast, and doesn't decouple slow processing from fast requests. As part of a whole system, the database usually sits behind a cache for read speed, is fed by queues for write-heavy background processing, and is the one component every other piece in the system ultimately exists to protect from excessive direct load.
Component 5: CDN (Content Delivery Network)
A CDN is a globally distributed network of servers that cache and serve content from a location physically closer to the person requesting it, rather than every request traveling all the way to one central server.
The Analogy: Think of a chain of regional warehouses instead of one central warehouse. Shipping a product from a warehouse in the same region as the customer is faster than shipping everything from one central location on the other side of the country - the product itself is identical, only the distance it travels changes.
Without a CDN, a request from a user in Australia to a server in the United States travels the full physical distance every time, adding real, unavoidable latency. With a CDN, that same user's request is served by a nearby edge server holding a cached copy, and the response comes from a server much closer physically, cutting latency substantially for content that doesn't change per request.
How it stands alone vs. as part of a whole system: A CDN is only useful for content that can actually be cached - static assets like images, CSS, JavaScript, or video, not personalized or frequently-changing data specific to one user. As part of a whole system, a CDN typically handles the static-asset portion of a request while the load balancer, cache, database, and queue handle the dynamic, personalized portion - splitting the request path so each component only handles the kind of traffic it's actually good at handling.
Putting It All Together: One Request Path
A realistic combined request path looks something like this.
- A user requests a page.
- Static assets - images, CSS, JavaScript - get served directly from the CDN, fast and close to the user.
- The dynamic portion of the request hits the load balancer, which routes it to one of several application servers.
- That application server checks the cache first: on a cache hit, it returns the result immediately; on a cache miss, it continues to the database.
- The application server queries the database for the actual data, then stores the result in the cache for next time.
- Any slow, non-urgent follow-up work - sending an email, logging analytics, updating a search index - gets published to a message queue instead of being done inline, so the user gets their response immediately while a separate consumer processes the queue at its own pace.
Every component in that path solves a different problem. None of them substitute for each other, and a real system rarely needs only one.
Key Lessons
Each component solves one specific problem - load balancers distribute traffic, caches speed up reads, queues decouple timing between producer and consumer, databases persist data, and CDNs shorten physical distance.
None of these components is free - a load balancer adds a hop, a cache adds a staleness risk, a queue adds a processing delay, and a CDN adds cache-invalidation complexity for content that does change.
The right question for any of these components is never "should I use this" in the abstract, but "what specific problem in this system does this solve, and is that problem actually present."
Real systems combine several of these components along a single request path, each handling the part it's actually good at, rather than any single component trying to do everything.
This same reasoning already appears across this blog's own Azure posts - Service Bus as a message queue, Cosmos DB and Table Storage as NoSQL choices, Azure SQL as the relational choice - system design vocabulary and specific cloud service choices are the same underlying ideas expressed at different levels of abstraction.
What's Next
Future System Design posts on this blog will apply these same components to specific, realistic scenarios, including a walkthrough of designing a resilient integration platform, drawing on real experience connecting Salesforce and ServiceNow at Blue Yonder.
Summary
System design interviews, and production architecture, both draw from the same small set of building blocks: load balancers to distribute traffic, caches to speed up repeat reads, message queues to decouple producers from consumers, databases to persist data in a shape that matches its actual structure, and CDNs to shorten the physical distance between content and the people requesting it. None of these is a default choice - each earns its place by solving a specific problem that's genuinely present in a given system. Understanding what problem each one solves, individually, is what makes it possible to reason through an unfamiliar system design question rather than just recalling a memorized architecture.
Originally published at TechStack Blog: https://www.techstackblog.com/post.html?slug=system-design-fundamentals-components
Related reading on TechStack Blog:
Azure Service Bus: https://www.techstackblog.com/post.html?slug=azure-service-bus-deep-dive
Cosmos DB and Blob Storage: https://www.techstackblog.com/post.html?slug=azure-cosmos-db-blob-storage
More from TechStack Blog: CS Fundamentals: https://www.techstackblog.com/category.html?cat=cs-fundamentals
Azure: https://www.techstackblog.com/category.html?cat=azure
Top comments (0)