amananandrai

Posted on May 3, 2024 Edited on Oct 30

System Design Fundamentals, Explained Simply

#computerscience #architecture #beginners #systemdesign

System design is how you decide which pieces a big app needs and how they talk to each other, so it stays fast, keeps working when parts fail, and grows with its users. This guide covers the concepts that come up most in interviews and on the job, each explained twice: once like you're 5, once like you're an engineer.

How to read this guide

Every topic follows the same pattern:

ELI5 box: a one-line, everyday analogy so the idea clicks first.
The real explanation: what it is and how it works, kept short.
A table or diagram for anything that is an "X vs Y" or a flow.
Real-world example: where you've already used it.
Interview tip: what to actually say when it comes up.

Networking: TCP/IP, IPv4 vs IPv6, TCP vs UDP, HTTP versions and WebSocket, DNS, PKI and TLS, encryption, proxies, API gateway, CDNs
Databases: SQL vs NoSQL, NoSQL types, ACID vs BASE, partitioning, consistent hashing, replication, indexes, consistency models
Distributed systems: scaling, CAP and PACELC, leader election, consensus (Paxos and Raft), microservices, messaging, distributed storage, batch vs stream processing
Caching and probabilistic data structures: caching strategies, eviction, Bloom filter, Count-Min Sketch
Concurrency: locking, threads, race conditions, deadlocks, CAS, barriers, semaphores, monitors
Infrastructure: data centers, VMs vs containers, disk access, load balancers
Reliability essentials (new): rate limiting, idempotency and retries, circuit breakers, unique IDs, observability, availability nines
Putting it together (new): back-of-the-envelope estimation and a worked URL shortener design
Bonus, low-level design: design patterns and object-oriented design
Interview cheat sheet and further reading

Part 1: Networking

Everything in a system talks over a network, so this is where every design starts.

TCP/IP model

ELI5: Sending a letter. You write it, put it in a tracked envelope, the post office adds the address and picks a route, and a truck drives it there.

The TCP/IP model splits "getting data from A to B" into four layers. Each layer only worries about its own job.

Layer	Job	Examples
Application	What the data means	HTTP, DNS, SMTP
Transport	Deliver it to the right app, reliably or fast	TCP, UDP, QUIC
Internet	Find a route between machines	IP
Network access	Move bits over a cable or radio	Ethernet, Wi-Fi

The older OSI model has seven layers. It is useful for teaching, but TCP/IP is what the internet actually runs on.

Interview tip: "Layer 4" and "layer 7" come up with load balancers. Layer 4 means transport (IP and port); layer 7 means application (URL, headers).

IPv4 vs IPv6

ELI5: House numbers. The old street had only about 4 billion numbers, and now every phone and fridge wants one. The new system has so many we'll never run out.

	IPv4	IPv6
Address size	32 bits	128 bits
Number of addresses	About 4.3 billion	About 3.4 × 10^38
Example	142.250.181.238	2001:db8::1
How we cope with shortage	NAT (many devices share one public IP)	Not needed
IPsec support	Optional add-on	Originally required to be supported, now optional

Two common myths to avoid: IPv6 does not encrypt traffic by default, and IPv4 already supports traffic prioritization (DSCP). The real win of IPv6 is the huge address space, which removes the need for NAT. The two are not directly compatible, so networks run both side by side ("dual stack") during the transition.

TCP vs UDP

ELI5: TCP is a phone call where you keep asking "did you hear that?" and repeat anything missed. UDP is shouting across the playground: faster, but some words get lost.

	TCP	UDP
Connection	Handshake before sending	None, just send
Delivery	Guaranteed, in order	Best effort, may arrive out of order or not at all
Speed	Slower (acknowledgments, retransmits)	Faster, lower latency
Flow and congestion control	Yes	No
Used for	Web pages, APIs, email, file transfer	Video calls, online games, DNS, live streams

For a video call, a late packet is useless anyway, so UDP's speed wins. For a bank transfer, every byte matters, so TCP wins.

HTTP/1.1 vs HTTP/2 vs HTTP/3 vs WebSocket

ELI5: HTTP/1.1 is asking the teacher one question and waiting for the answer before the next. HTTP/2 is asking many questions in one trip. WebSocket is a walkie-talkie left on, so either side can talk anytime.

	HTTP/1.1	HTTP/2	HTTP/3	WebSocket
Runs on	TCP	TCP	QUIC (over UDP)	TCP (starts as HTTP, then upgrades)
Many requests on one connection at once	No	Yes (multiplexing)	Yes	Not applicable
Headers	Plain text, repeated	Compressed	Compressed	Tiny frame headers
Who can start talking	Client only	Client only	Client only	Both sides, anytime
Best for	Simple APIs	Most websites	Mobile and lossy networks	Chat, live scores, multiplayer, collaborative editing

HTTP/2 still suffers when one TCP packet is lost, because everything behind it waits (head-of-line blocking). HTTP/3 fixes this by running on QUIC. HTTP/2 "server push" is effectively dead: major browsers removed it, so don't lean on it.

Real-world example: WhatsApp Web and Slack keep a WebSocket open so new messages appear instantly.

DNS lookup

ELI5: A phonebook. You know your friend's name, not their number. You ask Mom; she checks the country book, then the ".com" book, then the family that owns the name, who gives her the number. Mom remembers it for next time.

DNS turns a name like www.google.com into an IP address. Your device asks a recursive resolver (from your ISP, or a public one like 1.1.1.1), which does the legwork:

The root, TLD and authoritative servers form the hierarchy that holds the answers. The resolver is a helper that walks that hierarchy and caches results for each record's TTL (time to live).

Interview tip: DNS is also a simple load-balancing and failover tool. One name can return several IPs, or different IPs by region (GeoDNS).

Public key infrastructure, certificate authorities and TLS

ELI5: A school ID card. Anyone can say they're your bank. The website shows an ID card signed by a principal your browser trusts (the certificate authority), so you know it's real.

Certificate: a file that says "this public key belongs to example.com," signed by a certificate authority (CA).
CA: an organization (such as Let's Encrypt or DigiCert) your browser or operating system already trusts.
PKI: the whole system of keys, certificates, CAs and the chain of trust between them.

HTTPS is HTTP inside a TLS connection. (SSL is TLS's deprecated predecessor; only TLS 1.2 and 1.3 should be used today.) Here is the modern TLS 1.3 handshake:

Two things worth noticing. First, the certificate's job is to prove who the server is; the session key comes from a Diffie-Hellman exchange. Second, because a fresh key is made every session, stealing the server's private key later can't decrypt old recorded traffic. This is called forward secrecy. The older TLS 1.2 method, where the browser encrypted the session key with the server's RSA public key, lacked this and was removed in TLS 1.3.

Symmetric vs asymmetric encryption

ELI5: Symmetric is one key that locks and unlocks a diary, so both friends need a copy. Asymmetric is handing out open padlocks to everyone while only you keep the key.

	Symmetric	Asymmetric
Keys	One shared secret key	A public key and a private key
Speed	Very fast	Much slower
Hard part	Getting the key to the other side safely	Proving a public key really belongs to someone (that's what CAs are for)
Algorithms	AES, ChaCha20	RSA, Elliptic-curve (ECDH for key exchange, ECDSA and Ed25519 for signatures)
Used for	Encrypting the actual data	Key exchange and digital signatures

Asymmetric cryptography does two different jobs, and they're easy to mix up:

Encrypting with someone's public key keeps a message secret, because only their private key can open it. It does not prove who sent it.
Signing with your own private key proves you sent it, because anyone can check it with your public key. Signatures give you authentication and non-repudiation.

Real systems use both. TLS uses asymmetric crypto to agree on a key and verify the server, then switches to fast symmetric encryption (AES or ChaCha20) for the data.

Forward proxy vs reverse proxy

ELI5: A forward proxy is your big brother going to the shop for you, so the shop only sees him. A reverse proxy is a restaurant waiter, so you never see which cook in the kitchen made your food.

	Forward proxy	Reverse proxy
Sits in front of	Clients	Servers
Hides	Who the client is	Which server answered
Used for	Company web filtering, privacy, caching for an office	Load balancing, TLS termination, caching, shielding servers
Examples	Squid, corporate proxies	Nginx, HAProxy, Envoy, Cloudflare

API gateway

ELI5: The hotel front desk. It checks who you are, makes sure you're not visiting too often, and sends you to the right room.

An API gateway is a specialized reverse proxy that is the single entry point to many backend services. It handles the jobs every service would otherwise repeat: routing, authentication, rate limiting, request and response transformation, and logging.

Real-world example: a mobile app calls one URL, api.shop.com, and the gateway sends /orders to the orders service and /payments to the payments service.

Interview tip: Mention that the gateway itself must be scaled and replicated, or it becomes a single point of failure.

CDNs and edges

ELI5: An ice-cream van parked in every neighborhood, so nobody has to travel to the faraway factory.

A content delivery network (CDN) is a set of servers spread around the world that keep copies of your content close to users. Each of these servers is an edge (or point of presence). On a cache miss, the edge fetches from your origin server and keeps a copy.

Key techniques:

Anycast or GeoDNS: route each user to the nearest edge.
TTL and cache invalidation (purging): decide how long a copy stays fresh, and remove stale copies when content changes.
Tiered caching: edges ask a regional cache before going all the way to the origin.
Edge compute: run small bits of code at the edge (Cloudflare Workers, Lambda@Edge).

Real-world example: Netflix places its own caching servers inside internet providers' networks so most video never crosses the wider internet.

Part 2: Databases

The database is usually the hardest part of a system to scale, so most design discussions end up here.

Relational (SQL) vs NoSQL

ELI5: A relational database is a strict form where everyone fills in the same boxes. NoSQL is a set of toy boxes, and each box can hold different stuff.

	Relational (SQL)	NoSQL
Data shape	Tables with a fixed schema	Key-value, documents, wide columns or graphs; flexible schema
Relationships	Joins across tables	Usually you store data the way you read it (denormalized)
Transactions	Strong ACID transactions	Varies; many now offer ACID too (MongoDB, DynamoDB)
Scaling	Traditionally vertical; sharding takes extra work	Built to spread across many machines
Pick it when	Data is relational and correctness matters (orders, payments)	Massive scale, simple access patterns, or changing data shapes
Examples	PostgreSQL, MySQL	Redis, Cassandra, MongoDB, DynamoDB

The old rule "SQL means ACID, NoSQL means BASE" is outdated. Many NoSQL databases support ACID transactions, and "distributed SQL" databases like Google Spanner and CockroachDB scale horizontally.

Interview tip: Don't pick NoSQL just for "scale." Say what your access pattern is ("we always look up by user ID") and pick the database that fits it.

Types of NoSQL

ELI5: Key-value is numbered lockers. Wide-column is a giant notebook where each row can have its own columns. Document is a folder of differently filled-in forms. Graph is a friendship map with strings between people.

Type	How data looks	Great for	Examples
Key-value	`key → value`	Caches, sessions, shopping carts	Redis, DynamoDB
Wide-column	Rows with flexible column families	Huge write-heavy workloads, time series	Cassandra, HBase, Bigtable
Document	JSON-like documents	Product catalogs, user profiles, content	MongoDB, Couchbase, Firestore
Graph	Nodes and edges	Social networks, recommendations, fraud detection	Neo4j, Amazon Neptune

ACID vs BASE

ELI5: ACID is buying candy: you give the money and get the candy, or neither happens. BASE is class gossip: everyone hears the news eventually, but for a moment some kids don't know yet.

ACID describes reliable transactions:

Atomicity: all steps happen or none do. Moving ₹500 debits one account and credits another, never just one.
Consistency: a transaction never breaks the database's rules (for example, a balance can't go negative).
Isolation: transactions running at the same time don't see each other's half-finished work.
Durability: once committed, the data survives a crash.

BASE describes systems that trade some of that for availability and scale:

Basically available: the system keeps answering, even during failures.
Soft state: data may be changing in the background as copies sync.
Eventually consistent: if writes stop, all copies converge to the same value.

	ACID	BASE
Priority	Correctness	Availability and scale
Cost	Coordination makes it slower to scale	Users may briefly see stale data
Good for	Payments, inventory, bookings	Likes, view counts, feeds, analytics

Partitioning and sharding

ELI5: The library is too big for one room, so books A–M go in room 1 and N–Z in room 2. That's sharding.

Horizontal partitioning (sharding): split rows across machines. Users 1–1M on shard 1, 1M–2M on shard 2.
Vertical partitioning: split columns into separate tables. Keep a user's name and email in one table and their large profile photo data in another.

How you pick the shard key decides everything:

Strategy	How it works	Watch out for
Range	Shard by ranges of the key (A–M, N–Z)	Hot spots, such as all new users landing on the last shard
Hash	Shard by hash(key) mod N	Range queries get hard; resharding moves lots of data (consistent hashing helps)
Directory	A lookup table maps key to shard	The lookup service must be fast and highly available

Interview tip: Always name your shard key and say how you'll handle hot keys (for example, a celebrity user) and queries that need data from several shards.

Consistent hashing

ELI5: Kids sit around a round table, and each toy goes to the next kid clockwise. When a new kid sits down, only a few toys move to them. Nobody else's toys get shuffled.

With plain hash(key) mod N, changing N (adding a server) moves almost every key. Consistent hashing places servers and keys on a circle (a "hash ring"). Each key belongs to the first server clockwise from it. Adding or removing a server only moves the keys next to it, roughly 1/N of the data.

In practice each server gets many positions on the ring (virtual nodes), so load spreads evenly and a failed server's keys scatter across all the others.

Real-world example: Cassandra, DynamoDB and many distributed caches use consistent hashing.

Database replication

ELI5: Photocopies of your homework in several bags. Lose one bag and you still have it.

Replication keeps copies of the same data on several machines, for safety and to serve more reads.

flowchart LR
    App[App] -->|writes| L[(Leader)]
    App -->|reads| F1[(Follower 1)]
    App -->|reads| F2[(Follower 2)]
    L -->|replicates| F1
    L -->|replicates| F2

Setup	How it works	Trade-off
Leader-follower (primary-replica)	One node takes writes; followers copy it and serve reads	Easy and common; writes are limited to one node
Multi-leader	Several nodes accept writes and sync with each other	More write capacity and multi-region; you must resolve conflicting writes
Leaderless	Clients write to and read from several replicas (quorums)	Very available; more complex consistency rules (Cassandra, Dynamo)

Timing	Behavior	Trade-off
Synchronous	The write waits until replicas confirm	No data loss on failover, but slower writes
Asynchronous	The write returns immediately; replicas catch up	Fast, but replicas lag and recent writes can be lost if the leader dies

Interview tip: Mention replication lag. A user who updates their profile and immediately reads from a lagging follower sees old data. A common fix is "read your own writes" from the leader for a few seconds.

Database index

ELI5: The index at the back of a book. It's how you find "dinosaurs" without reading every page.

An index is an extra data structure that lets the database find rows without scanning the whole table. Reads get much faster; writes get a bit slower and storage grows, because every index must be updated on each write.

Index type	Good for
B-tree	The default. Exact matches, ranges, sorting
Hash	Exact-match lookups only
LSM-tree (log-structured)	Write-heavy stores (Cassandra, RocksDB)
Full-text (inverted)	Searching words inside text (Elasticsearch)
Geospatial	"Find restaurants near me"

Interview tip: Index the columns you filter or sort by most. A composite index on (user_id, created_at) makes "this user's latest posts" fast.

Strong vs eventual consistency

ELI5: Strong consistency is one whiteboard everyone looks at. Eventual consistency is a note passed around the class: everyone gets it, just not at the same moment.

Strong consistency: once a write succeeds, every read after it returns that value. Simple to reason about, but needs coordination, which adds latency.
Eventual consistency: reads may return old values for a while, but all copies converge if writes stop. Fast and highly available.

There are useful levels in between, such as read-your-own-writes (you always see your own changes) and causal consistency (a reply never appears before the message it replies to).

Use strong consistency for	Eventual is fine for
Bank balances, inventory counts, seat bookings	Like counts, view counts, follower lists, news feeds

Part 3: Distributed systems and scalability

Once one machine isn't enough, you have many machines that must cooperate over an unreliable network. This part covers how they do it.

Vertical vs horizontal scaling

ELI5: Buy a bigger horse (vertical), or get more horses (horizontal).

	Vertical (scale up)	Horizontal (scale out)
How	Bigger machine: more CPU, RAM, disk	More machines behind a load balancer
Pros	Simple, no code changes	Near-unlimited growth, survives single machine failures
Cons	Hard ceiling, expensive at the top, single point of failure	Needs stateless services, data partitioning, coordination

Interview tip: Start simple ("one server and one database handle the first 10,000 users"), then explain what breaks first as load grows and how you'd scale that part out. To scale out easily, keep app servers stateless: put sessions in Redis or a database, not in server memory.

CAP theorem (and PACELC)

ELI5: Two friends each have a copy of the same notebook, and the phone line between them breaks. Someone asks a question. Each friend can answer with what they have, which might be old (availability), or say "wait, I can't check" (consistency). Phone lines will break sometimes, so you decide in advance which you prefer.

CAP names three properties:

Consistency (C): every read sees the latest write (or gets an error).
Availability (A): every request to a working node gets a non-error answer.
Partition tolerance (P): the system keeps running even when the network splits nodes into groups that can't talk.

The common phrase "pick two of three" is misleading. In any real distributed system, network partitions will happen, so P is not optional. The real rule is: when a partition happens, you must choose between consistency and availability.

Choice during a partition	Behavior	Examples
CP	Refuse or delay some requests rather than return stale data	ZooKeeper, etcd, HBase, Spanner
AP	Keep answering, possibly with stale data; reconcile later	Cassandra, DynamoDB (default settings), DNS

PACELC fills in the rest of the story: if there is a Partition, choose A or C; Else, when everything is healthy, choose between Latency and Consistency. Even with no failures, waiting for replicas to agree costs time.

Interview tip: Pick per feature, not per system. "The payment ledger is CP; the 'who's online' indicator is AP."

Leader election

ELI5: Kids picking a team captain. If the captain goes home, they quickly pick a new one using a rule everyone agrees on, like "highest jersey number wins."

Many systems need exactly one node in charge: to accept writes, to run a scheduled job, or to hand out locks. Leader election picks that node and picks a new one if it dies. The hard part is avoiding split brain, where two nodes both think they're the leader.

Method	How it works	Notes
Consensus-based (most common today)	Nodes vote; a candidate needs a majority. Built into Raft	Majority voting prevents two leaders in the same term
Lease via a coordination service	Nodes race to create a lock or key in ZooKeeper or etcd; the winner holds it while it keeps renewing	The easy, practical choice for most applications
Bully algorithm	A node that notices the leader is gone messages all higher-ID nodes. If none answer, it declares itself leader	Classic textbook algorithm; lots of messages
Ring algorithm	An election message travels around a ring; each node swaps in its own ID if it's higher. When the message returns, the highest ID wins	Classic textbook algorithm; slow on large rings

Interview tip: Say "I'd use etcd or ZooKeeper for leader election" rather than inventing your own. Also mention fencing tokens: each new leader gets a higher number, so storage can reject writes from an old leader that hasn't realized it was replaced.

Consensus: Paxos and Raft

ELI5: A class vote where an idea only wins if more than half the class agrees. Two different answers can never both get more than half, so the class never ends up with two answers.

Consensus lets a group of machines agree on a value (or an ordered log of values) even if some crash or messages are delayed. The core idea is the majority quorum: with 5 nodes, 3 must agree. Any two majorities overlap in at least one node, so the group can't decide two different things. A 5-node cluster survives 2 failures; a 3-node cluster survives 1.

Paxos (1989–1998) was the first proven algorithm. Proposers suggest values, acceptors vote, learners learn the result, in prepare and accept phases. It is correct but famously hard to understand and implement.
Raft (2014) was designed to be understandable. One elected leader receives all writes, appends them to a log, and copies the log to followers. An entry is committed once a majority has it. If the leader dies, a new election starts.

Real-world example: etcd (Kubernetes' brain), Consul, CockroachDB and Kafka's KRaft mode use Raft. Google's Chubby and Spanner use Paxos.

Microservices

ELI5: Instead of one giant robot, you have a team of small robots that each do one job. If one breaks, the others keep working, but they have to talk to each other a lot.

A microservice architecture splits an app into small services, each owning one business capability and its own data, deployed independently. The alternative is a monolith: one codebase, deployed as one unit.

	Monolith	Microservices
Deploys	One big deploy	Each service ships on its own schedule
Scaling	Scale everything together	Scale only the busy services
Failure	A bug can take down everything	Failures can be isolated (with care)
Complexity	Low at first	High: networking, monitoring, distributed data
Best for	Small teams, early products	Many teams, large systems

Services talk through synchronous calls (REST, gRPC) or asynchronous messages (Kafka, queues). Common supporting pieces are an API gateway, service discovery, distributed tracing and circuit breakers.

Interview tip: Each service owning its own database means no cross-service joins and no simple transactions. Multi-service workflows use the saga pattern: a series of local steps, each with an "undo" action if a later step fails.

Distributed messaging systems

ELI5: A mailbox. You drop a letter in, and the other person reads it when they're ready. Nobody has to wait at the door.

A message system sits between producers and consumers so they don't have to be online at the same time or run at the same speed. This smooths traffic spikes, lets you add new consumers without changing producers, and keeps work from being lost if a consumer crashes.

Model	How it works	Examples
Queue	Each message goes to one consumer; good for spreading work	RabbitMQ, Amazon SQS
Publish-subscribe log	Every subscriber group gets every message; messages are kept and can be replayed	Apache Kafka, Pulsar, Kinesis

Delivery guarantees (a favorite interview question):

Guarantee	Meaning	Risk
At-most-once	Sent once, never retried	Messages can be lost
At-least-once	Retried until acknowledged	Duplicates; consumers must be idempotent
Exactly-once	Each message takes effect once	Only within specific systems (like Kafka transactions); in practice it's at-least-once plus deduplication

Distributed file systems and object storage

ELI5: A huge toy collection spread across many friends' houses, with extra copies of each toy. A master list says where everything is.

A distributed file system splits big files into chunks, stores each chunk on several machines, and keeps a metadata service that knows where every chunk lives. Users see one big file system.

Distributed file systems: HDFS, Google File System (GFS), CephFS. Built for big-data jobs that read huge files sequentially.
Object storage: Amazon S3, Google Cloud Storage, Azure Blob Storage. Stores whole objects (images, videos, backups) by key over HTTP. Extremely durable and cheap, and the default answer for "where do we store user uploads?"

Interview tip: Store large files in object storage and keep only their URL and metadata in your database. Serve them through a CDN.

Batch vs stream processing (MapReduce and beyond)

ELI5: Counting candies in a giant jar. Each friend counts one handful by color (map). Then one friend per color adds up all the totals (reduce).

MapReduce (Google, 2004) made it easy to process huge data sets across thousands of machines:

Map: each machine processes its own chunk and outputs key-value pairs, such as ("red", 12).
Shuffle: all pairs with the same key are sent to the same machine.
Reduce: each machine combines the values for its keys, such as "red" = 4,310.

The framework handles splitting, scheduling and retrying failed machines, so you only write the map and reduce functions. Hadoop MapReduce has largely been replaced by faster tools, but the idea lives on.

	Batch	Stream
Processes	A large, finished data set	Events continuously as they arrive
Latency	Minutes to hours	Milliseconds to seconds
Examples	Apache Spark, Hadoop, BigQuery	Apache Flink, Kafka Streams, Spark Structured Streaming
Use cases	Daily reports, training ML models, search indexing	Fraud detection, live dashboards, trending topics

Part 4: Caching and probabilistic data structures

Caching is the cheapest big speed-up in most systems. Probabilistic structures trade a tiny bit of accuracy for huge memory savings.

Caching

ELI5: Keep your favorite toy on your desk instead of in the attic box. You grab it faster, but you have to remember to swap it when it changes.

A cache keeps a copy of frequently used data in fast storage (usually memory) so you skip slow work like a database query or a network call. Caches live at many layers: the browser, the CDN, the app server's memory, a shared cache like Redis or Memcached, and the database's own buffer.

How to read and write with a cache:

Strategy	How it works	Trade-off
Cache-aside (lazy loading)	App checks the cache; on a miss, reads the database and fills the cache	Most common; first request is slow; data can go stale
Read-through	The cache itself loads from the database on a miss	Simpler app code; needs a cache that supports it
Write-through	Every write goes to cache and database together	Cache always fresh; writes are slower
Write-back (write-behind)	Write to cache now, flush to database later	Very fast writes; data loss if the cache dies first
Write-around	Writes skip the cache and go to the database	Avoids filling the cache with data nobody reads

What to throw out when the cache is full (eviction):

Policy	Evicts	Good when
LRU (least recently used)	What hasn't been used for the longest	The usual default
LFU (least frequently used)	What is used least often	Popularity is stable over time
FIFO	The oldest entry	Simple cases
TTL (time to live)	Anything past its expiry time	Data that must not be too stale

The hard parts:

Invalidation: when the data changes, delete or update the cached copy. A short TTL is a simple safety net.
Cache stampede: a popular key expires and thousands of requests hit the database at once. Fix it by letting only one request refill the key, or by refreshing it slightly before it expires.
Cold start: an empty cache after a restart sends all traffic to the database. Warm it up before taking full traffic.

Interview tip: Whenever you add a cache, say three things: what you cache and under which key, how long (TTL), and how it gets invalidated when data changes.

Bloom filter

ELI5: A bouncer with a quick checklist. It can say "definitely not on the list" or "maybe on the list, let me check properly." It's never wrong about "definitely not."

A Bloom filter answers "have I seen this item before?" using very little memory.

Start with an array of m bits, all 0, and k different hash functions.
To add an item, hash it k times and set those k bits to 1.
To check an item, hash it k times. If any of those bits is 0, the item was definitely never added. If all are 1, it was probably added: other items may have set those bits by chance.

So it can have false positives but never false negatives. More bits mean fewer false positives. A standard Bloom filter can't delete items (a counting Bloom filter can).

Real-world example: Cassandra and other LSM-tree databases check a Bloom filter before reading a file from disk, skipping files that definitely don't contain the key. Web crawlers use them to skip URLs they've already visited.

Count-Min Sketch

ELI5: Tally marks on a few small boards to estimate how many red cars you've seen. It might guess a little too high, but never too low.

A Bloom filter answers "is it there?"; a Count-Min Sketch answers "how many times have I seen it?" for a huge stream, in fixed, small memory.

Keep a small grid of counters: d rows, each with its own hash function, and w columns.
To count an item, hash it once per row and add 1 to that counter in each row.
To estimate an item's count, look up its counter in every row and take the minimum.

Other items sometimes land in the same counters, which can only push counts up, never down. Taking the minimum across rows picks the least-polluted counter. Its estimates are never too low and only sometimes too high.

Real-world example: finding trending hashtags or the most-requested URLs in a never-ending stream, where keeping an exact counter for every item would use too much memory.

Interview tip: Reach for these when the question says "billions of items" and an approximate answer is acceptable. A related tool is HyperLogLog, which counts how many unique items you've seen (such as daily unique visitors) in a few kilobytes.

Part 5: Concurrency and synchronization

When many things touch the same data at the same time, results can quietly go wrong. These tools keep them correct.

Optimistic vs pessimistic locking

ELI5: Pessimistic locking is locking the bathroom door before you go in. Optimistic locking is coloring in a shared coloring book without asking, but checking before you save that nobody colored the same spot. If they did, you redo it.

	Pessimistic	Optimistic
Assumes	Conflicts are likely	Conflicts are rare
How	Lock the row first (`SELECT … FOR UPDATE`), then change it	Read a version number; update only if the version is unchanged, otherwise retry
Cost	Others wait; risk of deadlocks	Wasted work and retries when conflicts do happen
Use when	Many writers fight over the same rows (the last concert seats)	Mostly reads, or writers rarely touch the same row (editing your own profile)

Optimistic locking in one SQL line:

UPDATE products SET stock = stock - 1, version = version + 1
WHERE id = 42 AND version = 7;  -- 0 rows updated means someone else got there first: retry

The deciding factor is contention (how often writers collide), not simply whether reads outnumber writes.

Threads, race conditions, locks and deadlocks

ELI5: Many kids building one Lego castle together. A race condition is two kids grabbing the same brick at once. A lock means only the kid holding the "talking stick" can touch that part. A deadlock is Kid A holding the red brick and wanting the blue one, while Kid B holds the blue brick and wants the red one. They wait forever.

Thread: one line of work inside a program. Multithreading runs several at once to use all CPU cores, or to keep working while waiting on disk or network.
Race condition: the result depends on timing. Two threads both read a counter of 5, both add 1, both write 6, and one increment is lost.
Lock (mutex): only one thread may enter a critical section at a time.
Deadlock: threads wait on each other in a cycle and none can continue. Prevent it by always taking locks in the same order, or by using timeouts.
Livelock: threads keep reacting to each other and changing state, but none makes progress, like two people stepping aside in a hallway in the same direction again and again.

Atomic operations and CAS (compare-and-swap)

ELI5: "I'll change the score to 6, but only if it's still 5." If someone else changed it first, you look again and retry.

Compare-and-swap (also called compare-and-set) is a single CPU instruction that updates a value only if it still equals what you expect, all in one indivisible step. Loop on it until it succeeds and you get thread-safe updates without locks. These are called lock-free data structures, and Java's AtomicInteger and C++'s std::atomic are built on CAS.

It is the same idea as optimistic locking, just inside one machine's memory instead of a database.

Barriers, semaphores and monitors

Tool	ELI5	What it does	Typical use
Barrier	The field-trip bus doesn't leave until every kid is back on board	Threads wait at a point until all of them arrive, then all continue	Parallel computations done in rounds
Semaphore	The bouncy castle holds 5 kids, tracked with 5 tickets	A counter: take a ticket to enter (wait), return it to leave (signal); blocks when none are left	Limiting concurrent access, such as at most 10 database connections
Monitor	A room with one door and a guard who lets one kid in at a time and keeps a waiting bench	Bundles a lock with "wait until something changes" signals (condition variables)	Producer-consumer queues; Java's `synchronized` with `wait()` and `notify()`

A semaphore with one ticket (a binary semaphore) behaves much like a mutex. Monitors are higher level and harder to misuse than raw semaphores.

Interview tip: In system design interviews, concurrency usually shows up as "two users buy the last item at the same time." Answer with a database-level fix (a row lock, optimistic versioning, or an atomic UPDATE … WHERE stock > 0), or a distributed lock if the data spans services.

Part 6: Infrastructure and resource management

Your software runs on real machines in real buildings. Knowing their shape explains why systems fail the way they do.

Data centers, racks and hosts

ELI5: The data center is a giant library building. Racks are the bookshelves, and hosts are the books (individual computers).

Host: one physical server.
Rack: a cabinet of 20–40 servers sharing power and a top-of-rack network switch.
Data center: a building with thousands of racks, plus power, cooling and network links.
Availability zone (AZ): one or more data centers with independent power and networking, a few kilometers from the others in the same region.
Region: a group of AZs in one geographic area, such as Mumbai or Frankfurt.

Failures happen at every level: a disk, a host, a whole rack (if its switch dies), a data center, even a region. Spread copies across these failure domains, not just across machines.

Interview tip: "Replicas in three availability zones" is the standard answer for surviving a data center outage. Add a second region only when the requirements call for surviving a regional disaster.

Virtual machines vs containers

ELI5: VMs are separate houses, each with its own plumbing: very private, but slow to build. Containers are apartments sharing one building's plumbing: quick to move into, but the walls are thinner.

	Virtual machine	Container
What's inside	A full operating system plus the app	Just the app and its libraries
Shares	Only the hardware (via a hypervisor)	The host's operating system kernel
Startup	Tens of seconds to minutes	Under a second to a few seconds
Size	Gigabytes	Megabytes
Isolation	Strong	Weaker, since all containers share one kernel
OS flexibility	Can run any OS (Windows on a Linux host)	Must match the host kernel (Linux containers need a Linux kernel)
Examples	VMware, KVM, AWS EC2	Docker, containerd, managed by Kubernetes

In the cloud you usually get both: containers running inside VMs. For stronger isolation with container speed, there are lightweight micro-VMs such as AWS Firecracker.

Random vs sequential disk access

ELI5: Reading a book page by page is fast. Jumping to a random page every time is slow.

Reading or writing data in one continuous run is much faster than jumping around:

Hard disks (HDDs) must physically move a head to each spot, so random access can be 100 times slower or worse than sequential.
SSDs have no moving parts and handle random access far better, but sequential access is still faster and wears the drive less.

This one fact shapes many systems. Databases write changes to an append-only log first (a write-ahead log). Kafka is fast because it only ever appends to and reads files in order. LSM-tree databases like Cassandra turn random writes into sequential ones.

Load balancer

ELI5: The lunch lady sending each kid to the shortest line.

A load balancer spreads incoming requests across many servers, removes servers that fail health checks, and lets you add or remove servers without users noticing.

	Layer 4 (transport)	Layer 7 (application)
Decides using	IP address and port	URL path, headers, cookies
Speed	Very fast, simple	Slightly slower, much smarter
Can do	Spread TCP/UDP connections	Route `/api` and `/images` to different services, TLS termination, A/B tests
Examples	AWS Network Load Balancer	Nginx, HAProxy, Envoy, AWS Application Load Balancer

Algorithm	How it picks a server	Good when
Round robin	Next in line	Servers and requests are similar
Weighted round robin	Bigger servers get more turns	Servers differ in size
Least connections	Fewest active connections	Requests vary in length
Least response time	Fastest recent responses	Performance varies between servers
IP or key hashing	Same client or key always goes to the same server	You need stickiness or cache locality

Interview tip: The load balancer must not be a single point of failure. Run a pair in active-passive mode, or use a managed cloud load balancer, which is already redundant.

Part 7: Reliability essentials

These topics weren't in the original post, but interviewers ask about them constantly. They are what keep a system up when things go wrong.

Rate limiting

ELI5: The candy jar refills one candy per minute. You can take candy only if there's some in the jar, so nobody can grab them all at once.

Rate limiting caps how many requests a user, IP address or API key can make in a time window. It protects you from abuse, bugs in clients and sudden traffic spikes. Rejected requests get HTTP 429 Too Many Requests.

Algorithm	How it works	Notes
Token bucket	Tokens drip into a bucket at a fixed rate; each request spends one	Allows short bursts; the most common choice
Leaky bucket	Requests queue and drain at a steady rate	Smooth output, no bursts
Fixed window	Count requests per calendar minute	Simple, but allows double traffic at window edges
Sliding window	Count requests over the last 60 seconds	Accurate and fair; a bit more work

Interview tip: With many servers, keep the counters in a shared store like Redis so all servers enforce one limit together.

Idempotency and retries

ELI5: Pressing the elevator button five times still brings one elevator.

An operation is idempotent if doing it twice has the same effect as doing it once. This matters because networks fail in a confusing way: the request may have worked even though the reply was lost. The only safe move is to retry, and retries are only safe if the operation is idempotent.

GET, PUT and DELETE are meant to be idempotent. POST (such as "create payment") is not, by default.
Fix it with an idempotency key: the client sends a unique ID with the request, and the server remembers which IDs it has already processed. Stripe's payments API works this way.
Retry with exponential backoff and jitter: wait 1s, 2s, 4s, plus a random amount, so thousands of clients don't retry at the same instant.

Circuit breakers and timeouts

ELI5: The fuse box in your house. When something shorts out, the fuse trips so the whole house doesn't catch fire.

When a service you depend on is slow or down, calling it anyway ties up your threads and spreads the failure upstream (a cascading failure). A circuit breaker watches the error rate:

stateDiagram-v2
    [*] --> Closed
    Closed --> Open: too many failures
    Open --> HalfOpen: after a cool-down period
    HalfOpen --> Closed: trial request succeeds
    HalfOpen --> Open: trial request fails

Closed: requests flow normally.
Open: fail fast immediately, or return a fallback (cached data, a default) without calling the broken service.
Half-open: let a few trial requests through to see if it has recovered.

Always pair this with timeouts on every network call. A call without a timeout can wait forever.

Unique ID generation

ELI5: Giving every kid in a huge school a name tag, when kids are signing up at many desks at once, without two kids ever getting the same tag.

One database's auto-increment counter becomes a bottleneck and a single point of failure at scale. Options:

Approach	How it works	Trade-off
UUID (v4)	128 random bits	No coordination; large, and random IDs don't sort by time and make database indexes slower
UUID v7 / ULID	Timestamp first, then random bits	No coordination, and sorts by creation time
Snowflake (Twitter)	64 bits: timestamp in milliseconds + machine ID + per-millisecond sequence	Compact, time-sortable, about 4,096 IDs per millisecond per machine; machines need unique IDs
Ticket server / ID ranges	A central service hands out blocks of numbers	Simple; the central service must be highly available

Observability

ELI5: A car's dashboard. Speed, fuel and warning lights tell you something is wrong before the engine dies.

You can't fix what you can't see. Observability rests on three kinds of data:

Signal	What it is	Answers	Tools
Metrics	Numbers over time	"Is it healthy? Is latency rising?"	Prometheus, Grafana, Datadog
Logs	Timestamped event records	"What exactly happened?"	ELK / OpenSearch, Loki
Traces	One request's path through every service	"Which service made this request slow?"	OpenTelemetry, Jaeger

A useful starting set of metrics is the four golden signals: latency, traffic, errors and saturation. Track latency as percentiles (p50, p99), not averages, because averages hide your slowest users.

Availability and the "nines"

ELI5: If the shop promises to be open "99.9% of the time," it's still allowed to be closed for a few hours each year.

Availability	Downtime per year	Downtime per month
99% ("two nines")	3.65 days	7.3 hours
99.9% ("three nines")	8.76 hours	43.8 minutes
99.99% ("four nines")	52.6 minutes	4.4 minutes
99.999% ("five nines")	5.3 minutes	26 seconds

SLI (indicator): what you measure, such as the share of requests that succeed.
SLO (objective): your internal target, such as 99.9% over 30 days.
SLA (agreement): the promise to customers, with refunds or penalties if you miss it.

Components in a chain multiply: two services at 99.9% that both must work give about 99.8%. Redundant copies in parallel push availability up, which is why replication matters.

Interview tip: Each extra nine costs far more than the last. Ask what the business actually needs before designing for five nines.

Part 8: Putting it all together

Concepts click when you see them used together. This part shows the quick math interviewers expect, then a full design that uses most of this guide.

Back-of-the-envelope estimation

ELI5: Before a party, you guess how many pizzas to order. You don't need the exact number, just enough to not run out.

Rough numbers decide your design: one database or fifty, a cache or not. Round aggressively; the order of magnitude is what matters.

Handy conversions:

1 day ≈ 86,400 seconds ≈ 100,000 (10^5) seconds.
1 million requests per day ≈ 12 requests per second.
Peak traffic ≈ 2–3 times the average.

Approximate latencies every engineer should know:

Operation	Approximate time
Read from main memory	100 nanoseconds
Random read from an SSD	100 microseconds
Network round trip inside one data center	0.5 milliseconds
Hard disk seek	10 milliseconds
Network round trip between continents	150 milliseconds

The takeaway: memory is about 1,000 times faster than SSD, and SSD is about 100 times faster than a cross-continent round trip. That's why we cache, and why CDNs exist.

Worked example: design a URL shortener

ELI5: A coat check. You hand over a long coat (the URL), get a tiny ticket (the short code), and later the ticket gets you the coat back.

1. Requirements

Functional: turn a long URL into a short one; visiting the short URL redirects to the long one; optional custom aliases and expiry.
Non-functional: redirects must be very fast and highly available; short codes must never collide.

2. Estimates (assumed numbers)

100 million new URLs per month ≈ 40 writes per second.
Reads outnumber writes about 100 to 1 ≈ 4,000 redirects per second, maybe 10,000 at peak.
About 500 bytes per record × 100 million a month × 12 months × 5 years ≈ 3 TB of storage.
Code length: Base62 (a–z, A–Z, 0–9) with 7 characters gives 62^7 ≈ 3.5 trillion codes. That's plenty.

3. High-level design

4. Key decisions, and which concepts they use

Decision	Choice	Concept from this guide
Short code generation	Unique numeric ID (Snowflake-style or ID ranges), encoded in Base62	Unique ID generation
Database	Key-value store (DynamoDB or Cassandra): one lookup by code, no joins	SQL vs NoSQL
Scaling the database	Shard by short code with consistent hashing; 3 replicas across availability zones	Sharding, consistent hashing, replication
Fast redirects	Cache-aside in Redis with LRU eviction; a small share of links gets most clicks	Caching
App servers	Stateless, behind a load balancer, scaled horizontally	Horizontal scaling, load balancer
Abuse protection	Rate limit link creation per user and IP	Rate limiting
Click analytics	Publish click events to Kafka; process asynchronously	Messaging, stream processing
Consistency	A new link must work right away (read your own write); click counts can be eventually consistent	Consistency models, CAP

5. Trade-offs to discuss

301 vs 302 redirect: a 301 (permanent) is cached by browsers, which cuts load but hides repeat clicks from analytics. A 302 (temporary) sends every click through your servers.
Hashing the URL instead of using IDs: taking the first 7 characters of a hash avoids an ID service, but you must detect and handle collisions.
Hot links: a viral link can overload one cache shard. Keep a small in-memory cache on each app server for the hottest keys.

Part 9: Bonus, low-level design

Everything above is high-level design (HLD): boxes and arrows between services. Many companies also run a separate low-level design (LLD) round, where you design the classes inside one service (a parking lot, an elevator, a library system). Design patterns and object-oriented design belong there.

Object-oriented design

ELI5: Modeling things as "objects" that know stuff and do stuff. A Dog object knows its name and knows how to bark.

Principle	Meaning	ELI5
Encapsulation	Hide internal data; expose methods	You press the TV remote's buttons, you don't rewire the TV
Abstraction	Show what something does, not how	You drive a car without knowing how the engine works
Inheritance	A class reuses and extends another	A puppy is a dog, so it can do everything a dog can
Polymorphism	One interface, many implementations	"Speak!" makes a dog bark and a cat meow

The SOLID principles (single responsibility, open-closed, Liskov substitution, interface segregation, dependency inversion) are the usual checklist for good class design.

Design patterns

ELI5: Lego instruction booklets for shapes people build all the time.

Design patterns are proven, reusable solutions to common design problems. They come in three families:

Family	Pattern	What it does	ELI5
Creational (making objects)	Singleton	Exactly one instance exists	There's only one class president
	Factory	Create objects without naming their exact class	Ask the toy factory for "a car"; you don't care how it's built
	Builder	Build a complex object step by step	Ordering a sandwich: bread, then fillings, then sauce
Structural (combining objects)	Adapter	Make incompatible interfaces work together	A travel plug adapter
	Decorator	Add behavior to an object without changing it	Adding sprinkles and syrup to plain ice cream
	Facade	One simple interface over a complex system	One "movie night" button that dims lights, starts the TV and closes blinds
Behavioral (how objects talk)	Observer	Subscribers are notified when something changes	Hitting the subscribe bell on YouTube
	Strategy	Swap algorithms at runtime	Choosing to walk, bike or bus to school
	Command	Wrap a request as an object, so it can be queued or undone	A restaurant order ticket that waits in line for the cook

Interview tip: Several patterns appear in system design at a larger scale. Pub-sub messaging is the observer pattern across services; an API gateway is a facade; a circuit breaker is a proxy with a strategy for failure.

Interview cheat sheet

A simple order to follow in any system design interview:

Clarify requirements. What must it do? How many users? What matters most: speed, correctness or availability?
Estimate. Requests per second, storage, read-to-write ratio.
Sketch the high-level design. Clients, load balancer, stateless services, database, cache.
Design the data. Schema, SQL or NoSQL, shard key, indexes.
Go deep on one or two parts. Usually the hottest path or the hardest consistency problem.
Find what breaks. Single points of failure, hot keys, replication lag, cascading failures, and how you'd fix each.
State your trade-offs out loud. Every choice costs something; say what you gave up and why.

If you hear…	Reach for…
"Read-heavy" or "slow responses"	Caching, read replicas, CDN
"Write-heavy" or "huge data"	Sharding, LSM-tree databases, message queues
"Real-time updates"	WebSockets, pub-sub
"Must not lose or double-charge"	ACID transactions, idempotency keys
"Global users"	CDN, multi-region replication, GeoDNS
"Spikes and bursts"	Queues, autoscaling, rate limiting
"Billions of items, approximate is fine"	Bloom filter, Count-Min Sketch, HyperLogLog
"One service keeps failing"	Timeouts, retries with backoff, circuit breakers

Conclusion

System design is mostly about trade-offs: speed against correctness, simplicity against scale, cost against availability. There's rarely one right answer, but there are clearly reasoned ones. If you can explain each concept here simply, like the ELI5 boxes, and say when you'd use it and what it costs, you're ready for most interviews and most real design discussions.