DEV Community

Sibasish Mohanty
Sibasish Mohanty

Posted on

Event-Driven Systems: Building for Scale and Decoupling

#systemdesign #interview #database #javascript

In part 7 of our System Design series, we’re focusing on asynchronous communication patterns that help systems scale, remain resilient, and avoid tight coupling.

We’ll cover:

  1. Queues & Streams – Kafka, RabbitMQ; smoothing spikes
  2. Backpressure – Managing slow consumers and load shedding
  3. Delivery Semantics – At-most-once, at-least-once, exactly-once

1. Queues & Streams

TL;DR: Decouple producers from consumers to handle traffic spikes and processing delays.

  • Message Queues: RabbitMQ, SQS – good for task-based workloads.
  • Event Streams: Kafka, Kinesis – append-only logs, useful for event sourcing and replay.

👉 Example: Order placement publishes events to a queue; different consumers handle billing, inventory update, and notification separately.

👉 Interview tie-in: "Why use a queue instead of direct HTTP calls?" — Decoupling improves resilience and scales better under spikes.

2. Backpressure

TL;DR: Don’t let slow consumers cause system meltdown.

  • Load Shedding: Drop low-priority messages when overwhelmed.
  • Buffer Limits: Fixed queue sizes prevent uncontrolled memory growth.
  • Consumer Throttling: Consumers signal readiness to receive data.

👉 Example: Kafka pauses sending data to a consumer when it’s overloaded until the consumer catches up.

👉 Interview tie-in: "How do you handle a slow consumer in a messaging system?" — Implement backpressure strategies.

3. Delivery Semantics

TL;DR: Pick the right guarantee based on business needs.

  • At-most-once: Risk of message loss, but no duplicates (good for non-critical notifications).
  • At-least-once: Duplicates possible; consumers must be idempotent (e.g., order processing).
  • Exactly-once: Hard to implement; requires complex coordination (Kafka Streams offers it in some cases).

👉 Example: Payment processing must be exactly-once; use unique transaction IDs to avoid duplication.

👉 Interview tie-in: "Which delivery guarantee would you pick for sending email notifications?" — At-most-once is acceptable since duplicates are harmless.

✅ Takeaways

  • Use queues and streams to decouple and scale
  • Implement backpressure to prevent system overload
  • Choose the right delivery semantics based on business criticality

💡 Practice Question:

"Design a video processing pipeline where users upload videos and get them transcoded. How do you handle spikes and ensure each video is processed exactly once?"

Top comments (0)