Building a Scalable Notification System with RabbitMQ and Microservices

Ever wondered how apps like Slack, WhatsApp, or Gmail send millions of notifications without breaking a sweat? The secret lies in asynchronous message queues.

In this article, I'll write about my learning and insight through building a production-ready notification system using RabbitMQ, Node.js, Fastify, Docker and SendGrid as part of my HNG Internship project.

*But, why we need message queue.? *
This below is call without queue message, I mean it is Synchronous in nature.

app.post('/send-notification', async (req, res) => {
await sendEmail(req.body);
res.send('Email sent');
});

But this can cause problems if service(email) is down, the the request will fail immediately. Also, API response is slow (waits for email to send), No retry logic in place and Can't scale email sending independently.

Below is with queue message i.e Asynchronous = don’t wait, continue working with another task.

`// Publish to queue - returns immediately
app.post('/send-notification', async (req, res) => {
await publishToQueue(req.body);
res.send('Notification queued'); // Instant response
});

// Separate consumer processes the queue
consumeQueue(async (message) => {
await sendEmail(message);
});`

Just like I stated above the problems in Synchronous approach. The asynchronous is just the opposite; Fast API responses, retry logic, persistence of message and etc

Key Learnings

1. Asynchronous is Better Message queues allow services to work independently without blocking each other.
2. Always Retry Transient failures are common. Automatic retries with exponential backoff are essential.
3. Monitor Everything Track queue length, processing time, and error rates to catch issues early.
4. Design for Failure Services will fail. Build systems that gracefully handle and recover from failures

Learning progress
I had ZERO experience with:

• RabbitMQ (message queues)
• Fastify (Node.js framework)
• Microservices architecture
• Asynchronous messaging patterns

So I decided to get hands on:

• I completed a 35-minute Fastify crash course from Traversy Media.
• I Studied RabbitMQ fundamentals through YouTube tutorials and documentations
• I built, broke, and rebuilt the system several times. > I strongly recommend reading documentations with YouTube tutorials.

I also recommend using an email address that belongs to your own domain for smooth delivery on SendGrid. I used my Hostinger domain email, and it worked well. SendGrid won’t accept Gmail or other public email providers because I don’t own those domains. "So, I’d advise using an email address from a domain you own."

WHAT I BUILT:

• Email Service consuming from RabbitMQ queues
• Retry logic with exponential backoff (2s, 4s, 8s)
• Dead letter queue for failed messages
• Integration with SendGrid for email delivery
• Docker containerization for deployment

The end result ? here you go;

• Handles 1,000+ notifications/minute
• 99.5% delivery success rate
• Automatic retries prevent message loss
• Services scale independently

The magic of RabbitMQ: If my Email Service crashes and restarts, messages wait patiently in the queue. Nothing is lost. This is how apps like Slack, WhatsApp, and Gmail handle millions of messages daily!

The best part was seeing messages flow through RabbitMQ, get processed, and arrive in my inbox just seconds later. That first successful test felt like pure magic!

Huge thanks to HNG Internship (@HNG Internship) for providing hands-on, real-world projects that push you to learn, adapt, and deliver!

Let's Connect!
If you found this helpful or have questions, drop a comment below! I'd love to hear about your experiences with message queues and microservices
X
linkedin