I struggled with system design and event-driven architecture for years.
Then I learned that queues, streams, and event buses are three completely different things, even though they look identical on a whiteboard.
You draw the same boxes every time:
- Producers
- a broker
- And something subscribing on the other end.
That picture is what kept tripping me up, because it hides the only part that actually matters: what's going on inside the broker, and the way each one pushes your system to scale in a different direction.
Here's the mental model that finally got it to stick.
A queue is a to-do list. A message lands in line, one worker picks it up, finishes it, and it's gone. Consumed once, then deleted. Want to go faster? Add more workers pulling from the same line, and that's pretty much the whole trick. SQS, RabbitMQ, Celery all work this way: one job, one consumer, done a single time. You reach for it when the work genuinely can't run twice, like charging a card or firing off a welcome email.
A stream is a logbook you can rewind. Reading a message doesn't delete it. Each consumer just keeps a little bookmark, an offset, marking where it left off, so two teams can read the same data at once at completely different speeds. One of them can jump back to last Tuesday while the other stays on the live edge.
An event bus is a switchboard. An event shows up, and the bus works out who should hear about it from rules you write, instead of you wiring every connection by hand. "order.paid" fires once, and shipping, analytics, and the fraud check all react to it without any of them knowing the others exist.
So same picture, three problems that barely overlap.
I went through all three properly in my latest video, including how each one works under the hood and how it changes the way your system scales.
Top comments (0)