Vahid Aghajani

Posted on Sep 14 Originally published at software-engineer-blog.com

Design a notification system — a full mock system design interview (Ep 4)

#systemdesignmockinterview #programming #softwareengineering

📺 Prefer to watch? 90-second YouTube Short · 💬 Telegram

Originally published on software-engineer-blog.com.

You walk into a system design interview and hear "Design a notification system." Your instinct is to draw a box labeled "Notification API," connect it to a push provider and an email provider, and call it done. That instinct will cost you the round.

The trap is that "notifications" is not one system—it is two systems running at completely different speeds, and you cannot see that difference until you ask before you draw.

Mental model: A notification system is not a delivery system; it is a speed-matcher. Services produce notifications at one rate (unpredictable spikes), providers consume them at wildly different rates (20,000/s push, 200/s SMS), and your job is to decouple those two curves so that a ten-million-message campaign does not leave a password reset waiting fourteen hours.

Four questions before the first box

A real candidate does not guess. They ask:

What notification types do we send? Push, email, SMS, in-app, Slack—each has its own delivery latency and throughput. Push is fast and fire-and-forget. SMS is slow and expensive. Email is somewhere in the middle. You cannot design one queue if the drain rates differ by a factor of a hundred.
What is the expected volume? The answer tells you whether to build or to buy. A hundred million notifications a day sounds large until you do the arithmetic: that is roughly 1,160 per second on average. But the average is useless. Peak hour for a social network might be ten times the average. Peak for a campaign is all ten million notifications arriving within minutes.
What is the latency requirement? A password reset should arrive in seconds. A marketing email can arrive in hours. A campaign notification can arrive whenever the queue drains it. Latency is not a number; it is a contract that shapes whether you use a queue at all.
What happens if delivery fails? Do we retry? How many times? Do we log it and move on? Do we notify the service that sent the notification? This answer determines whether your system is fire-and-forget or transactional, and it changes whether you need a database row per notification.

The number that decides the design

Once you have answers, you size. Assume:

100 million notifications per day
50% push, 30% email, 20% SMS (rough split)
24-hour delivery window

That gives you:

Average per second:
  Push:  100M * 0.5 / 86400 = 578/s
  Email: 100M * 0.3 / 86400 = 347/s
  SMS:   100M * 0.2 / 86400 = 231/s

Peak hour (assume 10x average):
  Push:  5,780/s
  Email: 3,470/s
  SMS:   2,310/s

Provider throughput (from SLAs, not your code):
  Push:  ~20,000/s (Apple, Google, Firebase handle this)
  Email: ~5,000/s (SendGrid, AWS SES have rate limits)
  SMS:   ~200/s (Twilio, carrier agreements are slow)

The SMS number is the bottleneck. It is not because your code is slow; it is because the provider is slow. Your system must accept notifications in milliseconds (202 Accepted, queue them, and drain to SMS at 200 per second without blocking services or letting a ten-million-message campaign starve a password reset.

Why one queue does not work

You cannot put all three notification types into a single FIFO queue. Here is why:

Imagine a campaign sends 10 million SMS notifications. They arrive and sit in one queue. A user requests a password reset and gets a notification pushed behind those 10 million campaign messages. At 200 SMS per second, the password reset waits:

10,000,000 / 200 = 50,000 seconds = 13.8 hours

The user forgets what they clicked for.

The fix: separate queues per notification type, each with its own sender that respects the provider's throughput.

Services → Notify API (202 Accepted)
           ↓
        [Notify Store]
           ↓
        ┌──────────────────┐
        │ Push Queue       │ → Push Sender → Push Provider (20,000/s)
        │ Email Queue      │ → Email Sender → Email Provider (5,000/s)
        │ SMS Queue        │ → SMS Sender → SMS Provider (200/s)
        └──────────────────┘

Now each notification type drains at the speed of its provider, and a password reset SMS no longer waits behind a campaign.

The API contract

Your /notify endpoint does not send. It accepts and queues:

POST /notify
Content-Type: application/json

{
  "user_id": "12345",
  "notification_type": "sms",
  "provider": "twilio",
  "template": "password_reset",
  "params": {
    "reset_link": "https://..."
  },
  "retry_policy": "exponential",
  "max_retries": 3
}

→ 202 Accepted
  Location: /notify/event-id-xyz

{
  "event_id": "event-id-xyz",
  "status": "queued",
  "created_at": "2025-01-15T10:23:45Z"
}

The service gets a 202 in milliseconds. It is decoupled. It does not care whether SMS is slow or push is fast. It submitted the notification and moved on.

Why you cannot send it inside the request

The interviewer will push back: "A password reset is one API call. Why not just send it inside the POST /login request? It is simpler."

This is the moment that separates a rehearsed answer from a real one. Do not say "because decoupling is good" or "because queues are scalable." Say this:

On latency: If you send SMS inline, the login request blocks until Twilio responds. Twilio is a network hop to their servers, possibly in another region, and SMS is slow—they might batch for throughput. The user waits 2–5 seconds for a success response. The password reset request blocking on SMS delivery is a blast radius violation: a third-party service outage brings down login.

On reliability: If the SMS provider times out, the request times out. The client retries. Twilio may have received the message even though the response did not come back (the connection dropped). You now send the same password reset twice. The user gets two SMS.

Queuing solves both: send a 202 in 5 milliseconds, store the request durably, and retry independently. The provider timeout is a sending problem, not a login problem.

The retention and idempotency problem

Once you queue, you need to store the notification somewhere. A row per notification:

CREATE TABLE notifications (
  id UUID PRIMARY KEY,
  user_id BIGINT NOT NULL,
  notification_type VARCHAR(50),
  provider VARCHAR(50),
  template_id VARCHAR(100),
  params JSONB,
  status VARCHAR(20),
  created_at TIMESTAMP,
  sent_at TIMESTAMP,
  retry_count INT DEFAULT 0,
  last_error TEXT,
  provider_message_id VARCHAR(200),
  expires_at TIMESTAMP,
  INDEX (user_id, created_at),
  INDEX (status, created_at),
  INDEX (provider, status)
);

Each row is roughly 200–400 bytes. A hundred million notifications a day is about 20–40 GB. Retention is not free, but it buys you:

Deduplication: If the sender crashes and restarts, it re-reads the queue from the database. It checks provider_message_id. If the SMS was already sent, it updates the row and moves on. Without this, a crash turns into duplicate notifications.
Audit trail: A customer can ask "did you already send my password reset?" You check the database. You answer yes, the sent_at was at 10:23 UTC, and the SMS went through to Twilio (we have the Twilio message ID).
Preferences and do-not-contact: You can query the user's notification preferences at send time. If the user unsubscribed from marketing emails between the time the notification was queued and the time you try to send it, you do not send it.

The provider timeout trap

Here is where candidates break: the SMS provider sends the message, the connection times out, and your code sees an error.

sender.py:

response = twilio.send_sms(to, message)
# Twilio sends the SMS to the carrier
# Connection drops
# Your code sees: TimeoutError
# Your code retries
# Twilio receives the request again
# Carrier receives the SMS twice

The fix is idempotency:

request_id = notification["id"]  # UUID, generated once at /notify
response = twilio.send_sms(
    to=to,
    message=message,
    idempotency_key=request_id  # Twilio deduplicates on this
)

if response.status == "success" or response.status == "already_sent":
    notification["status"] = "sent"
    notification["provider_message_id"] = response.id
    db.update(notification)
else:
    notification["retry_count"] += 1
    if notification["retry_count"] >= max_retries:
        notification["status"] = "failed"
        log(notification)
    else:
        queue.requeue(notification, delay=backoff(retry_count))

The idempotency key is the notification UUID, generated once at /notify. Twilio stores it, deduplicates on retry, and returns "already sent" instead of sending twice.

Putting it together: the data flow

With all pieces:

Service calls POST /notify with a notification request.
API generates a UUID, writes the row to the notifications table with status queued, and returns 202 Accepted in <5ms.
An async worker reads the queued rows, groups by notification type, and pushes each batch into its type-specific queue (Redis streams, Kafka, SQS, or a database-backed queue).
Type-specific sender workers poll their queues, check user preferences, and call the provider API with an idempotency key.
Provider responds with success or error.
Sender updates the row: status becomes sent (or failed after max retries).
If a timeout occurs, retry logic uses exponential backoff and the same idempotency key; the provider deduplicates.
A cleanup job deletes notifications older than the retention window (e.g., 30 days).

Comparison: when to queue vs. when to send inline

Scenario	Send Inline (Don't Queue)	Queue (Decouple)
Latency SLA	User must wait for delivery (rare; login, payment confirmation)	Delivery is best-effort or has a loose SLA (marketing, campaign, most push)
Volume per second	< 100 per second; provider handles sync calls	> 100 per second; one provider slower than the others
Failure mode	Client retries the whole request; acceptable	Client gets 202; retry is independent; provider timeout is not a blast radius
Idempotency	Client-side retry is acceptable; user may get duplicates	Server-side idempotency key prevents duplicates on timeout
User expectation	Synchronous: "did it work?" in the response	Asynchronous: "submitted" in the response; delivery status checked separately

LLM inference angle: batch vs. stream

If you are serving notifications to a large language model for personalization (e.g., generating a subject line or body from a template), the same throughput mismatch applies:

Your notification queue drains to an LLM API at 5,000–10,000 tokens per second (your rate limit or batch size).
The LLM API handles 50,000+ tokens per second (their scale).
You are not bottlenecked by the model; you are bottlenecked by your queue or your network.
A ten-million-message campaign with personalization will fan out to N parallel LLM requests to avoid the batching latency.
Caching the LLM output per template + user segment reduces the number of API calls; one password reset should not regenerate a subject line if a thousand identical resets came before it.

The architecture does not change: separate queues, separate senders, idempotency keys, and timeout handling. But the bottleneck moves from the SMS provider to your LLM API, and caching becomes a first-class concern.

Verdict

Reach for queuing and separate per-type senders when notification types have throughput that differs by more than 10× or when services produce notifications in spikes that exceed any single provider's SLA. Reach for inline sending when latency is critical (the client must see success in the response), volume is predictable and low (<100/s), and the provider is fast. In practice, almost all real systems queue; the inline case is rare.

Watch the 90-second reel for the full walkthrough: https://youtube.com/shorts/--GXhdPi1zA

DEV Community