I Won’t Start My Next System Design Interview With a Load Balancer

First 10 minutes are a chance to make the problem smaller before making the diagram bigger.

Your blank whiteboard can make a load balancer look strangely comforting. You know where to put it, you know what connects to it, and within thirty seconds your answer looks like a system design answer.

Then someone asks what happens when the user is offline, and suddenly the boxes are less helpful than they looked.

I love a tidy diagram, but I don’t want it making decisions on my behalf. Before reaching for familiar components, I want a smaller thing settled. What does this system owe the person using it?

Try a quick exercise with me. Imagine the prompt is to design a notification system. Write down what should happen when someone replies to your post while your phone is offline. When you return an hour later, should that notification still exist?

If your answer is yes, you have already identified a requirement that a live connection alone cannot satisfy. You need a durable way to recover the missed notification. We have a design decision before drawing our first infrastructure box, which is a lovely place to start.

Pick one moment worth designing properly

Notification system can mean an unread bell, an emergency pager, a marketing email service or a mobile push platform. Starting with all four gives you an impressive amount of work and very little agreement about what matters.

For this practice example, let’s choose an in-app inbox. A reply creates a notification for one recipient. That person can see recent notifications and mark them as read. An offline user can return and retrieve what they missed. Email, SMS and scheduled campaigns stay outside our first pass.

Those are proposed boundaries, so in an interview I’ll confirm them before treating them as facts. If the interviewer wants emergency alerts, the discussion needs to change immediately.

Write one user action and one visible outcome before naming a technology. For our example, a person returns to the app and can find the reply notification even though they were disconnected when it arrived.

This is much more useful than asking whether the system should be scalable. Almost any interviewer can say yes to that, and neither of you learns very much.

Agree on the promise hiding inside success

Now we need to be a little picky about the word delivered. Does it mean the service accepted an event, the inbox contains a notification, an online device received an update, or the person actually opened it?

Those are different events. Treating them as one vague success flag makes the design sound simpler while leaving the important part unresolved.

For our first version, propose that an accepted notification is durably recorded before we acknowledge success. An online client should see a new item soon after it becomes available, while a reconnecting client fetches missed items from durable storage. Read state changes when the user takes the relevant action, not merely because the server sent an update.

We also need an answer for repeated input. If a producer retries the same reply event, should the user see two identical notifications? Assume no, and define an identity such as the event ID together with the recipient ID. Creating the same logical notification twice must resolve to the same stored item.

Notice that we haven’t promised that every network delivery happens exactly once. We have defined the user-visible outcome and left room to choose the mechanism that protects it.

My notes at this point can be wonderfully plain. Keep accepted work, recover after disconnection, suppress duplicate inbox items, and keep read state meaningful. Each of those statements will give a later box a job.

Ask for a number only when it changes a decision

Capacity estimates can become a small performance of their own. You multiply a few large numbers, say a storage unit with confidence, and then choose the architecture you were going to choose anyway.

I prefer asking what the estimate will help us decide before doing the arithmetic.

Suppose our illustrative workload has 100,000 active users, each receiving ten notifications a day. That gives us one million inbox writes daily, or roughly twelve per second on average. This is a practice assumption, not traffic data from a real company.

Twelve writes per second tells us surprisingly little about the hardest moment. If one announcement targets all 100,000 users, materializing a separate inbox item for each recipient creates 100,000 writes from one event. Now we have a reason to discuss bursts, background work and whether broadcast messages need a different representation.

Reads deserve their own estimate too. A badge fetched on every page view can have a different workload from an inbox opened twice a day. Ask about those access patterns rather than deriving every request from the same daily-user number.

Retention changes the storage discussion. At one million notifications per day, thirty days means thirty million notification records before accounting for indexes, replication or metadata. That does not tell us which database to choose, but it makes the retention assumption worth confirming.

An estimate earns its place when it changes what you build, what you defer, or what you need to ask next. Otherwise it is arithmetic wearing an interview outfit.

Draw one complete path before adding the clever parts

We can finally draw, and honestly, the diagram is easier now because the requirements have started choosing its responsibilities.

Begin with a producer submitting an event. An ingestion path validates it and records the accepted work durably. A worker creates the recipient’s inbox item using the notification identity to handle retries. An inbox API serves stored notifications and updates read state. A live-update path lets connected clients learn that something changed, while reconnecting clients fetch missed items.

That is a starting design, not a finished architecture. It gives us a concrete path to challenge rather than a collection of products to admire.

If the producer times out after the service commits the event, its retry must not create another logical notification. If a worker stops halfway through, pending work must remain discoverable. If the live connection drops, the inbox should still contain the item. Each failure sends us back to one of the promises we already wrote down.

Be careful around the handoff between a database and a queue. Recording work in one place and separately publishing a message can leave a gap if the process stops between those actions. A transactional outbox is one possible way to make committed pending work discoverable; a durable queue can also be the acceptance boundary in a design built around that contract. The choice needs an explanation, not just another arrow.

Load balancing may absolutely belong in this system. So may caching, partitioning and a particular messaging technology. I just want to introduce them when there is a responsibility or bottleneck to explain, instead of letting them become the opening sentence.

Give your opening a shape you can practise

For a ten-minute rehearsal, use four loose stages. These timings are my practice proposal, not a rule for every interview. Follow the interviewer’s direction when they want to move faster or explore a different area.

1. Use the first two minutes to choose the user moment. Describe the action, the expected outcome and the boundaries you are proposing. Confirm them rather than quietly shrinking the assignment.
2. Use the next two to define success and failure. Establish what accepted means, which work must survive, and what duplicates or delays the user can tolerate.
3. Spend two minutes estimating the work. Separate average traffic from bursts, and connect each number to a decision. Keep unanswered assumptions visible.
4. Use the remaining four to draw one complete path. Trace an event through it, identify where durable state lives, and choose a failure case to explore next.

If you want a concrete starting point, use this in-app notification-center prompt. Stop after the opening discussion and first path instead of rushing to reproduce a full solution. Our workload numbers and proposed boundaries above are practice choices to test against the prompt, not facts supplied by it.

Then close the notes and record a two minute explanation. Listen for places where a component appears without a reason, where an assumption quietly becomes a fact, or where you say reliable without explaining what survives a failure.

You can repair one of those gaps and repeat the opening without solving an entirely new question. That makes the session small enough to finish and specific enough to learn from.

Let the diagram show decisions you actually made

An opening full of questions can still go nowhere if the answers never affect the design. After each clarification, say what it changes. Offline access creates a recovery requirement. Repeated events require a stable identity. Large fan-out raises a workload question that an average request rate hides.

That is the thread I want running through my answer. A reader should be able to point at a box and explain which requirement put it there.

For the next practice session, leave the infrastructure names off the page for a few minutes. Pick the user moment, make the promise concrete, and follow the work.

When the load balancer finally arrives, it can have a job instead of being there to make the whiteboard feel less empty.

~ Emily