A few weeks ago I fell into one of those late-night research holes. It started because a friend preparing for a Staff Engineer loop sent me a link to a Medium post titled something like “60 Scenario-Based System Design Interview Questions Every Engineer Should Know, Part 2.” She wanted to know if she should spend her weekend memorizing it.
I opened it expecting a checklist I could skim in ten minutes. What I got instead was a wall of questions, each followed by three or four bullet points that read like they were assembled from a template: “use a load balancer,” “add caching,” “shard the database,” “use message queues for async processing.” Not wrong, exactly. Just hollow. None of it explained why you’d choose one approach over another, what breaks first when you’re wrong, or what an interviewer is actually listening for when you say the word “cache” out loud.
That bothered me enough that I spent the better part of two weeks doing something closer to an actual investigation: rereading my own interview notes from the last few loops I’ve sat on both sides of the table for, digging through what the better prep resources are saying about how these interviews changed going into 2026, and rebuilding a handful of the classic scenario questions from scratch with real trade-offs and real code instead of bullet-point mush. This is the writeup of that investigation. It’s long, on purpose. If you’re prepping for a system design round this quarter, I’d rather give you nine questions you actually understand than sixty you can recite.
I’ll say the opinionated part up front: lists of 60 questions are a symptom of a broken prep strategy, not a cure for it. Interviewers don’t have a bank of sixty scenarios they cycle through. They have four or five they know cold, and they spend the entire forty-five minutes pulling on whatever thread you hand them. What you need is a repeatable way of thinking, not a longer list to memorize. Everything below is built around that belief, so if you came here for a quick-reference cheat sheet, this probably isn’t it. If you want to understand why the cheat sheets keep failing people, keep reading.
What Actually Changed by 2026
Before getting into scenarios, it’s worth being honest about how much the bar moved in the last couple of years, because a lot of the “classic” prep material hasn’t caught up.
Three shifts stood out to me while researching this, and I’ve now seen all three show up in real loops, not just blog posts predicting them.
First, AI-aware design stopped being a specialty topic. A few years ago, “design a RAG pipeline” was the kind of question you’d only get if you were interviewing for an ML platform team. Now it shows up as a follow-up inside completely unrelated prompts. You’re designing a customer support ticketing system and the interviewer asks how you’d add an AI-generated response suggestion without leaking one customer’s data into another customer’s retrieval context. You don’t get to opt out of knowing what a vector index is anymore, even as a backend generalist.
Second, cost stopped being implicit. It used to be enough to say “we’ll scale horizontally” and move on. I’ve noticed interviewers now actually stop you and ask what a request costs, whether you’re over-provisioning, and whether your caching strategy is saving money or just saving latency. Vague scaling answers that don’t reason about cost per request read as junior now, even when the architecture itself is fine.
Third, the boring operational stuff got graded explicitly. Observability, deployment strategy, rollback plans, and what happens during a partial outage used to be things you’d mention if you had time left. Now they’re often an explicit part of the rubric. An elegant architecture that has no story for “how do you know it’s broken at 3 a.m.” is an incomplete answer.
None of this means the fundamentals changed. Consistent hashing, the CAP theorem, database sharding, and message queues are exactly as relevant as they were five years ago. What changed is the ceiling. The floor is the same, but “good” now requires you to reason about AI components, cost, and operations as first-class parts of the design instead of afterthoughts.
The Framework I Actually Use
I know frameworks are their own cliché in this space, but I want to write mine down honestly because it’s less about acronyms and more about order of operations. I’ve watched candidates (myself included, years ago) lose a perfectly good answer because they jumped straight to “we’ll use Kafka” before anyone agreed on what the system actually needed to do.
Here’s the shape I try to follow, roughly in this order, though real conversations never stay this linear:
+---+------------------------+---------------------------------------------+
| # | Step | What I'm actually trying to get out of it |
+---+------------------------+---------------------------------------------+
| 1 | Clarify the scope | What's in, what's out, who are the users, |
| | | read-heavy or write-heavy, real-time or not |
+---+------------------------+---------------------------------------------+
| 2 | Estimate the numbers | Rough QPS, data volume, growth rate. Doesn't |
| | | need to be precise, needs to shape decisions |
+---+------------------------+---------------------------------------------+
| 3 | Draw the naive version | The simplest thing that could work, even if |
| | | it obviously won't scale, as a baseline |
+---+------------------------+---------------------------------------------+
| 4 | Find the bottleneck | Where does the naive version actually break |
| | | first, given the numbers from step 2 |
+---+------------------------+---------------------------------------------+
| 5 | Fix it, one layer at a | Add caching, sharding, queues, replicas, one |
| | time | at a time, and say what each one trades away |
+---+------------------------+---------------------------------------------+
| 6 | Talk about failure | What happens when a node dies, a region goes |
| | | down, or a dependency times out under load |
+---+------------------------+---------------------------------------------+
| 7 | Talk about running it | Metrics, alerts, deploys, rollback, cost |
+---+------------------------+---------------------------------------------+
The part people skip is step 4. They jump from “here’s the naive design” straight to “here’s the fully scaled design” without ever explaining what forced the change. That’s the single biggest tell that separates someone reciting a memorized architecture from someone actually reasoning through a problem live. If you can’t say “this breaks because a single Postgres instance tops out around X writes per second on commodity hardware,” you’re pattern-matching, not designing.
With that out of the way, here’s the investigation itself: nine scenarios I rebuilt properly, followed by a faster round of ones I didn’t have room to go deep on but think you should still know exist.
1. Design a Rate Limiter for a Public API
This is the question I’d bet money shows up in some form in almost every backend loop, and it’s also the one where shallow answers fall apart the fastest, because “just use a counter” invites an immediate follow-up about what happens with two servers instead of one.
The naive version puts a counter in memory on each API server. It works until you have more than one server, at which point a client can get double their limit just by hitting different instances behind the load balancer. That’s the bottleneck from step 4 above, and it’s the whole reason this question exists.
The fix is to centralize the counter state, usually in Redis, and to pick an algorithm that doesn’t have obvious edge cases. Fixed windows are the trap here. If your window resets on the minute, a client can send their full quota at 11:59:59 and again at 12:00:01, getting double the intended rate in two seconds. Token bucket avoids this because it tracks a continuously refilling budget instead of a hard reset boundary, and it naturally allows short bursts without allowing sustained abuse.
+--------------------+-------------+----------------+------------------------+
| Algorithm | Burst-safe? | Memory cost | Main weakness |
+--------------------+-------------+----------------+------------------------+
| Fixed window | No | Very low | Boundary burst problem |
| Sliding window log | Yes | High (per req) | Storage grows with QPS |
| Sliding window ctr | Mostly | Low | Slight approximation |
| Token bucket | Yes | Low | Slightly more logic |
| Leaky bucket | No (smooths)| Low | Delays bursty clients |
+--------------------+-------------+----------------+------------------------+
The part candidates almost always skip is the race condition. If two requests from the same user hit two different API servers at the same instant, and both read the current token count before either writes back the decrement, you can let both through even though only one token was left. You fix this with an atomic operation, not application-level logic. A Lua script executed inside Redis does the read-check-decrement as a single atomic step, since Redis runs Lua scripts without interleaving other commands.
-- token_bucket.lua
-- KEYS[1] = bucket key, ARGV[1] = capacity, ARGV[2] = refill rate/sec,
-- ARGV[3] = now (unix seconds), ARGV[4] = requested tokens
local bucket = redis.call("HMGET", KEYS[1], "tokens", "ts")
local tokens = tonumber(bucket[1])
local ts = tonumber(bucket[2])
local capacity = tonumber(ARGV[1])
local rate = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local requested = tonumber(ARGV[4])
if tokens == nil then
tokens = capacity
ts = now
end
local elapsed = math.max(0, now - ts)
tokens = math.min(capacity, tokens + elapsed * rate)
local allowed = 0
if tokens >= requested then
tokens = tokens - requested
allowed = 1
end
redis.call("HMSET", KEYS[1], "tokens", tokens, "ts", now)
redis.call("EXPIRE", KEYS[1], 3600)
return allowed
If you want to actually test this instead of just talking about it, spinning up a local Redis is a two-minute job with Docker and doesn’t require any paid service:
# docker-compose.yml
services:
redis:
image: redis:7-alpine
ports:
- "6379:6379"
The follow-up question I’d ask if I were interviewing you is what happens when Redis itself is unreachable. Fail open and you have no rate limiting during an outage, right when a bad actor is most likely to be hammering you. Fail closed and you’ve turned a Redis blip into a total API outage for everyone. There’s no universally correct answer, and saying so out loud, then picking one based on the specific system’s risk profile, is a better answer than pretending the dilemma doesn’t exist.
2. Design a URL Shortener That Doesn’t Get Abused
Everyone knows the base62 encoding trick and the read-heavy caching story for this one by now, so I don’t think it’s actually testing what people assume it’s testing anymore. What it’s really testing, at least in the loops I’ve seen recently, is whether you think about abuse and correctness under concurrency.
The generation strategy matters more than people give it credit for. Hashing the long URL and truncating is tempting because it’s stateless, but collisions are inevitable at scale and you end up needing a collision-resolution path anyway, which erases the simplicity you were going for. A counter-based approach, where you reserve blocks of IDs per server ahead of time and convert them to base62, avoids collisions entirely and scales horizontally as long as each server owns a distinct block. The trade-off is that you now have a piece of shared, coordinated state (the counter) that becomes its own small distributed systems problem.
The part the sixty-question lists never mention is abuse. A public shortener is a magnet for phishing links, and if you don’t rate limit creation per account or IP, and don’t scan destination URLs against a threat-intelligence blocklist before activating a new short link, you will get abused within days of shipping. I’d also want a way to disable a link after the fact without deleting the row, since you’ll want the audit trail for whichever security or legal team eventually asks about it.
3. Design an Idempotent Payment Processing Flow
This is the scenario I actually think is underrated in most prep material, and it’s the one I’d genuinely worry about if you couldn’t answer it, because get it wrong in production and you double-charge real customers.
The core problem: a client sends a payment request, the server processes it successfully, but the response is lost to a network blip before the client sees it. The client, following retry logic, sends the same request again. Without protection, you now have two charges for one purchase.
The fix is an idempotency key, generated client-side and sent as a header on the request. The server checks whether it has already seen that key before doing anything else. If it has, it returns the stored result of the original request instead of processing again.
def handle_payment(request):
key = request.headers.get("Idempotency-Key")
if not key:
raise BadRequest("Idempotency-Key header required")
existing = redis.get(f"idem:{key}")
if existing == "processing":
# another request with the same key is mid-flight right now
raise Conflict("Request already in progress")
if existing:
return json.loads(existing) # replay the original result, no new charge
locked = redis.set(f"idem:{key}", "processing", nx=True, ex=30)
if not locked:
raise Conflict("Request already in progress")
try:
result = charge_card(request.body)
redis.set(f"idem:{key}", json.dumps(result), ex=86400)
return result
except Exception:
redis.delete(f"idem:{key}")
raise
The subtlety most answers miss is the middle state. Between “we’ve seen this key” and “we have a stored result,” there’s a window where a second identical request can arrive while the first is still mid-flight. If you don’t lock on that in-progress state too, you can still double-charge, just less often, which honestly makes it a worse bug because it’ll pass every simple test and then bite you rarely and unpredictably in production. The SET NX in the snippet above is doing exactly that locking, and it's the line I'd want a candidate to explain unprompted.
I’ll admit I went back and forth on whether to store the idempotency key at the application layer or push this down to the database with a unique constraint on a request ID column instead. Both are legitimate, and honestly the database constraint is probably the more bulletproof of the two since it survives even if your Redis logic has a bug. I lean toward doing both in a real system: the database constraint as the actual safety net, and the Redis check as a fast path that avoids hitting the payment processor a second time at all.
4. Design a Notification Fan-Out System
Picture a social app where a popular account posts something and needs to notify a few million followers. The naive approach, looping through followers and writing a notification row for each one synchronously inside the request that created the post, will time out the request before it finishes for anyone with a large enough following. That’s your step-4 bottleneck.
The standard fix is to decouple creation from delivery with a queue. The post-creation request publishes a single event, a worker fleet consumes it, and the actual fan-out (writing to each follower’s notification feed, or pushing to a message broker per user) happens asynchronously in the background.
Where this gets genuinely interesting is the fan-out strategy itself, and I think this is the part interviewers actually care about:
- Fan-out on write : when the post is created, immediately write a copy into every follower’s feed. Reads are then trivially fast, just fetch the pre-built feed. The cost shows up on write, and for an account with tens of millions of followers, a single post can trigger tens of millions of writes.
- Fan-out on read : don’t precompute anything. When a user opens their feed, merge posts from everyone they follow at read time. Writes stay cheap no matter how popular an account is, but reads get expensive, especially for users who follow a lot of accounts.
- Hybrid : fan-out on write for almost everyone, but flip to fan-out on read specifically for accounts above some follower threshold. This is what most large platforms actually do, because it avoids the “celebrity problem” without sacrificing read latency for the common case.
I like this question because the “right” answer genuinely depends on the numbers from step 2, not on memorized best practice. If you’re designing for a small internal tool with a thousand users, none of this matters and precomputing everything is fine. The skill being tested is recognizing when the standard-advice threshold applies to your specific scenario, not reciting the hybrid approach because it sounds sophisticated.
5. Design Leader Election for a Cluster of Workers
You’ve got a fleet of worker processes and exactly one task that must only run on one of them at a time, something like a scheduled cleanup job, and if two workers run it simultaneously you get duplicate or corrupted output. How do you guarantee only one worker acts as the leader at any moment, especially when workers crash without warning?
The building block here is a distributed coordination service, typically etcd or ZooKeeper (Consul works too), because they solve consensus for you and you really don’t want to hand-roll this yourself. The mechanism is usually a lease: a worker attempts to create a specific key with a short time-to-live, and whichever worker’s write lands first becomes the leader. That worker then has to keep renewing the lease before it expires, essentially proving it’s still alive, or another worker’s write will succeed and take over.
The trap I’ve seen candidates fall into is treating the leader’s authority as absolute once granted. In a real distributed system, a leader can be paused (a long garbage collection pause, a network partition that cuts it off without killing it) for longer than the lease TTL, lose leadership because its lease expired, and then resume running, not realizing a new leader has already taken over. If both the old and new leader think they’re in charge simultaneously, you have a split-brain problem. The fix is a fencing token, a monotonically increasing number handed out with each new leadership grant, which downstream systems can use to reject writes from a stale leader even if that stale leader still thinks it’s in charge. This is the detail that, in my experience, separates someone who’s used ZooKeeper as a black box from someone who understands why it’s built the way it is.
6. Design a Real-Time Chat System with Presence
This one tests something different from the others: connection state at scale, not just data at scale. A user’s “online” status and their open WebSocket connection have to live somewhere, and that somewhere has to survive individual server restarts without kicking everyone offline.
The naive version keeps each user’s connection tied to a single server’s in-memory state. It works fine until you have more than one chat server, which you will, because a single machine can only hold so many concurrent open sockets (tens of thousands, depending on hardware, well before you hit typical scale). Once you have multiple chat servers, you need a way for a message sent by a user connected to server A to reach a recipient connected to server B, and you need a shared source of truth for who’s currently online at all, since no single server has the full picture anymore.
The common pattern is a pub/sub layer (Redis pub/sub, or Kafka for higher durability needs) sitting between the chat servers. When server A gets a message, it doesn’t try to deliver it directly; it publishes to a channel keyed by the recipient’s user ID, and whichever server currently holds that user’s connection is subscribed and delivers it. Presence itself usually lives in Redis as a simple key with a TTL that each connected server refreshes on a heartbeat; if the heartbeat stops (server crash, network drop), the key naturally expires and the user shows as offline without anyone needing to explicitly clean it up.
The honest doubt I have about this one, and I’d respect a candidate who raised it unprompted: exactly-once delivery in a system like this is genuinely hard, and most real chat systems quietly accept at-least-once delivery with client-side deduplication by message ID instead of fighting for a guarantee that’s extremely expensive to actually enforce end to end. If an interviewer pushes you toward promising exactly-once, I think the stronger answer is to explain why that promise is expensive and suggest the deduplication approach instead of just agreeing to build something you’d struggle to actually deliver.
7. Design a Retrieval-Augmented Chatbot Without Leaking Data Between Customers
This is the question that didn’t really exist in this form three years ago, and it’s the clearest example of the AI-aware shift I mentioned earlier. The scenario: you’re building a support chatbot for a B2B SaaS product, where each customer has their own private knowledge base, and the bot needs to answer questions grounded in that customer’s documents specifically, never another customer’s.
The architecture itself is standard RAG: documents get chunked, embedded, and stored in a vector index; a user’s question gets embedded the same way; you retrieve the nearest chunks by similarity; you stuff those chunks into a prompt alongside the question and let the LLM generate a grounded answer instead of hallucinating from its training data alone.
The part that actually gets tested is isolation. If your vector index is one shared collection across all customers, a similarity search has no inherent concept of “customer boundary,” and a badly filtered query can retrieve, and then leak, another customer’s private data straight into the response. The fix is to make the tenant boundary a hard filter applied before or during the similarity search, not something you try to clean up after the fact by post-processing the LLM’s output. Most vector databases support metadata filtering for exactly this reason, and I’d treat “filter by tenant ID as part of the retrieval query itself” as the answer I actually want to hear, not “we’ll ask the model not to mention other customers.”
Since the project I write these articles for cares about not assuming everyone has an OpenAI and Pinecone budget, here’s what the same architecture looks like running entirely locally, which is also genuinely useful for interview prep since you can actually run it and see the failure modes yourself instead of taking my word for them:
# docker-compose.yml: fully local RAG stack, no external API calls
services:
ollama:
image: ollama/ollama:latest
ports:
- "11434:11434"
volumes:
- ollama_data:/root/.ollama
qdrant:
image: qdrant/qdrant:latest
ports:
- "6333:6333"
volumes:
- qdrant_data:/qdrant/storage
volumes:
ollama_data:
qdrant_data:
# ingest.py: embed and store one customer's docs with a tenant filter
import requests
OLLAMA = "http://localhost:11434/api/embeddings"
QDRANT = "http://localhost:6333"
def embed(text: str) -> list[float]:
r = requests.post(OLLAMA, json={"model": "nomic-embed-text", "prompt": text})
return r.json()["embedding"]
def upsert_chunk(collection: str, chunk_id: str, text: str, tenant_id: str):
vector = embed(text)
requests.put(
f"{QDRANT}/collections/{collection}/points",
json={
"points": [{
"id": chunk_id,
"vector": vector,
"payload": {"tenant_id": tenant_id, "text": text},
}]
},
)
def search(collection: str, query: str, tenant_id: str, top_k: int = 5):
vector = embed(query)
resp = requests.post(
f"{QDRANT}/collections/{collection}/points/search",
json={
"vector": vector,
"limit": top_k,
"filter": {
"must": [{"key": "tenant_id", "match": {"value": tenant_id}}]
},
},
)
return resp.json()["result"]
Pull a small embedding and chat model into Ollama first (ollama pull nomic-embed-text and ollama pull llama3.1), and you have a working, tenant-isolated RAG loop running on your own laptop with zero API keys and zero per-token billing. I built this exact stack while researching this section specifically to make sure the "filter" argument in the search call was doing what I claimed it does, and watching a query with the wrong tenant ID come back empty, instead of leaking another tenant's chunk, is a good sanity check if you want to actually convince yourself before an interview instead of just repeating what a blog post said.
8. Design an LLM Inference Gateway Under Load
Different flavor of AI question, and one I think is more likely to show up for backend and infra roles specifically, since it’s really a queueing and backpressure problem wearing an AI costume. You have a single (or small number of) GPU-backed LLM inference service behind your API, it can only handle a limited number of concurrent generations before latency degrades for everyone, and you’re getting far more requests than it can serve at once.
Unlike a typical stateless web request, an LLM generation call is expensive and slow (seconds, sometimes tens of seconds), so simply adding more application servers in front of it doesn’t help, since the bottleneck is the GPU capacity itself, not request routing. This is where I think a lot of candidates default to “just add more servers” out of habit, without noticing that the actual constrained resource here isn’t compute-in-general, it’s a specific, expensive, hard-to-scale-instantly piece of hardware.
The real answer involves a request queue in front of the inference workers, with the gateway accepting a request, enqueueing it, and returning a job ID immediately rather than holding the HTTP connection open for the full generation time. The client polls or holds a WebSocket for the result. Behind the queue, you run a fixed pool of workers matched to actual GPU capacity, and you apply backpressure, rejecting new requests with a clear “try again shortly” once the queue passes some depth, rather than accepting unbounded work and letting latency degrade silently for everyone already waiting. I’d also want priority tiers in the queue (paying customers ahead of free-tier ones, for instance) since undifferentiated first-come-first-served queueing is rarely what the actual business wants once you ask.
The cost-awareness point from earlier applies directly here too: GPU time is the most expensive resource in this whole system by a wide margin, so idle capacity is expensive and you’d want autoscaling tied to queue depth rather than static provisioning, while accepting that scaling GPU workers up takes real minutes, not seconds, which has to shape your backpressure thresholds.
9. Design Multi-Region Failover for a Critical Service
Last deep dive, and it’s the one where I think the CAP theorem stops being a whiteboard abstraction and becomes a genuinely uncomfortable business conversation. You’re running a service across two or more regions for availability, and one region goes down entirely. What happens?
If your data layer is synchronously replicated across regions, you get strong consistency (every region agrees on the current state) at the cost of write latency, since every write has to round-trip to the other region before it’s acknowledged, and if a region is unreachable, you either block writes entirely or you have to make an explicit decision to keep serving from the surviving region and reconcile later. If your replication is asynchronous, writes are fast and the surviving region keeps serving traffic immediately during an outage, but you accept that some recently written data might not have replicated yet and could be lost or need reconciling once the failed region comes back.
There isn’t a version of this where you get instant failover, zero data loss, and no write latency penalty, all three at once, and I think the honest, correct answer in an interview is to say that plainly and then argue for the trade-off that fits the actual system. A payments ledger and a “user’s last-seen timestamp” have wildly different tolerance for stale or lost writes, and I’d be far more impressed by a candidate who asked which one we were building before committing to an answer than one who confidently picked synchronous replication for everything because it sounds more rigorous.
The operational half of this question, which I mentioned earlier is graded much more explicitly now, is how you actually detect the region is down and how you fail over. Health checks need to be robust against false positives (a network blip between regions isn’t the same as a region actually being down, and flapping back and forth between regions on a flaky link is worse than just staying degraded), and DNS-based failover has propagation delay that a lot of candidates forget to account for, meaning “failover” isn’t instant even once you’ve decided to trigger it.
The Rapid-Fire Round
I promised nine deep dives, not sixty shallow ones, but it’s worth naming a few more scenarios you should at least be able to sketch the shape of, even if I’m not walking through each one in full here. Consider these the ones I’d want you to recognize on sight, not the ones I think deserve rote answers:
Designing a distributed cache invalidation strategy (write-through versus write-behind, and the classic thundering-herd problem when a hot key expires and every request tries to recompute it at once). Designing a job scheduler that survives the scheduler process itself crashing mid-run. Designing search-box autocomplete with typo tolerance under tight latency budgets (this is a trie-plus-ranking problem more than a database problem). Designing a feature flag system where flag evaluation has to be fast enough to run on every single request without adding a database round trip. Designing an audit log that’s genuinely tamper-evident, not just “we wrote it to a table.” Designing a webhook delivery system with retries that doesn’t accidentally deliver the same event twice to a customer who didn’t ask for at-least-once semantics.
Every one of these follows the same seven-step shape from earlier. If you can walk through the framework on a scenario you’ve never seen before, you don’t need to have pre-memorized it.
Where I Landed
Going back to my friend’s original question: no, I don’t think memorizing a sixty-question list is a good use of a weekend, and having actually rebuilt nine of these from scratch, I’m more convinced of that than when I started. The value isn’t in having seen a specific scenario before. It’s in having a repeatable process for turning an ambiguous prompt into constraints, a naive design, a bottleneck, and a fix, and being able to narrate your own trade-offs honestly instead of defending a memorized answer as though it were the only correct one.
If there’s one thing I’d want you to take from this, it’s that the best answers I’ve heard in real loops, and the best ones I tried to write above, all share the same texture: a moment where the person says “this depends on X, and here’s what I’d need to know to decide,” instead of confidently picking a side because it’s the version they rehearsed. Interviewers can tell the difference, and increasingly, in 2026, that’s the actual thing being graded.
Tags: SystemDesign, SoftwareEngineering, TechInterviews, DistributedSystems, BackendDevelopment, AIEngineering, CareerAdvice
Top comments (0)