Microservices and What is a Messaging Queue?

Well as a backend developer, we know that maximum of the projects out there are almost the same either a clone of a website like YouTube, Facebook, Instagram, etc. No doubt they teach you a lot of things from database design to writing the API logic. But to make your resume stand out we need to add some jazz to it and this can be done by using a messaging queue like RabbitMQ and breaking down our project into a microservice architecture.

So I know I have thrown a lot of terms out there and as a beginner, you might not be able to understand some or all of them. So let's tackle them one by one.

Microservice architecture

Now what is a microservice putting it in simple terms as of now all the YouTube tutorials you see and everything you develop are contained in a single Project. Like, let us take an e-commerce website we have all things like authentication, payments, delivery, reviews everything under one big project. Now let us say if our payment service goes into fault due to some reason now this will eventually affect our application and might bring down the whole system.

Now as an engineer Do we really need it Nooo. If payment fails at Amazon, or Flipkart they don't break down they still work well try-catch block helps sometimes but with severe errors it will break. So let's break our main project into multiple services based on functionality like

• Authentication
• Payments
• Products

Create a separate service for each of them and eventually have communication between them moreover even if one goes down others are still working so your whole website won't go down. This will help you a lot in the longer run creating a robust architecture for your website and also it's really scalable at the same time. So in brief breaking down and modularising your project based on your business logic is more of a microservice architecture. Well, do it up to some level breaking it to a granule level will cause a lot of problems and then you will spend a whole lot of time just understanding it rather than coding it out. To read more about microservices follow this article here

Messaging Queue

Imagine a messaging queue as a kind of digital post office for computer programs. It's a system that helps different parts of a computer program or different computer programs talk to each other in a safe and organized way.

Here's how it works:

• Sending Messages: Think of one part of a program as sending a message (like an email) to another part. This message can contain information, instructions, or data.

• Message Storage: Instead of sending the message directly, it's placed in a queue, like putting your letter in a mailbox. The message waits there until the other part of the program is ready to read it.

• Receiving Messages: The other part of the program checks the queue and takes out the message when it's ready. It then does what the message says or uses the information in it.

The cool thing about messaging queues is that they allow parts of a program to talk to each other without waiting for an immediate response. This is like sending an email - you don't need an instant reply, and it allows different parts of a program to work at their own speed.

Messaging queues also help when you have lots of messages coming in or going out. They can keep messages organized, make sure they're not lost, and send them to the right place.

So, messaging queues are like the post office for computer programs, helping them communicate in a smooth, organized, and efficient way.

Now that we know the basics of what is microservice architecture and what is messaging queue we have the basic blocks to create something in our own project. Even Now you didn't get it Don't worry I will help you develop it as well.

Now let's say you have a project that has an emailing functionality so write now it's integrated into it. So now Let's create a separate microservice that does the email-sending functionality for you. This can be easily done using two ways. Let's explore them both ...

Using the normal API call

Normally we know that the frontend works by pinging the backend and sending out a response to it we can use that as well because we have two different services the frontend and backend and they are communicating with each other via API call. So why can't we have to backend sending API calls to each other and exchanging information between them?

Now to make it more interesting I have used two different languages here we are creating our email and everything in nodejs/javascript and sending it out via a Django service here is the code below

const express = require('express');
const axios = require('axios');
const bodyParser = require('body-parser');

const app = express();
const port = 3000; // Choose a suitable port

// Use middleware to parse JSON request bodies
app.use(bodyParser.json());

// Define a route for receiving email data from the frontend
app.post('/send-email', (req, res) => {
  // Access email data from the request body
  const emailData = req.body;

  // Forward the email data to the Django email service
  axios.post('http://django-email-service-url/send-email/', emailData)
    .then(response => {
      console.log('Email sent successfully');
      // Respond with a success message (customize as needed)
      res.status(200).json({ message: 'Email sent successfully' });
    })
    .catch(error => {
      console.error('Error sending email:', error);
      // Respond with an error message (customize as needed)
      res.status(500).json({ error: 'Error sending email' });
    });
});

app.listen(port, () => {
  console.log(`Backend server is running on port ${port}`);
});

Now the emailing part in django service
routes.py

from django.urls import path
from . import views

urlpatterns = [
    # Define the URL route for sending an email
    path('send-email/', views.send_email, name='send_email'),
]

views.py

from django.http import JsonResponse
from django.core.mail import send_mail
from django.views.decorators.csrf import csrf_exempt

@csrf_exempt
def send_email(request):
    if request.method == 'POST':
        to = request.POST.get('to')
        subject = request.POST.get('subject')
        message = request.POST.get('message')

        try:
            send_mail(subject, message, '[e-mail]', [to], fail_silently=False)
            return JsonResponse({'message': 'Email sent successfully'}, status=200)
        except Exception as e:
            return JsonResponse({'error': str(e)}, status=500)

    return JsonResponse({'error': 'Invalid request method'}, status=405)

So right now as we can see that we made two microservices communicate with each other booom.You have achieved your first microservices up and running.

But it looks kind of messy this way and it is not right as well cause if there are multiple requests to send an email there might be an issue of API throttling secondly it's not idle when we have a lot of data that needs to be communicated so and let's make it much cleaner using a messaging queue and for this tutorial we will be using Rabbit MQ.

Using RabbitMQ

• Step 1: Install RabbitMQ
  First, you'll need to install RabbitMQ. Follow the installation instructions for your specific operating system on the official RabbitMQ website: RabbitMQ Installation Guide

• Step 2: Set Up a Node.js Producer
  In this step, we'll create a Node.js service that acts as a message producer, sending email data to a RabbitMQ exchange.
  

npm install amqplib express body-parser

Create a Node.js script to send email data to RabbitMQ:

const express = require('express');
const amqp = require('amqplib');
const bodyParser = require('body-parser');

const app = express();
const port = 3000;

app.use(bodyParser.json());

// RabbitMQ server connection
const rabbitmqUrl = 'amqp://localhost';
const exchangeName = 'email_exchange';

app.post('/send-email', async (req, res) => {
  const emailData = req.body;

  try {
    const connection = await amqp.connect(rabbitmqUrl);
    const channel = await connection.createChannel();
    await channel.assertExchange(exchangeName, 'direct', { durable: false });

    // Publish email data to RabbitMQ
    channel.publish(exchangeName, '', Buffer.from(JSON.stringify(emailData));
    console.log('Email data sent to RabbitMQ');

    res.status(200).json({ message: 'Email data sent to RabbitMQ' });
  } catch (error) {
    console.error('Error sending email data:', error);
    res.status(500).json({ error: 'Error sending email data' });
  }
});

app.listen(port, () => {
  console.log(`Node.js service is running on port ${port}`);
});

• Step 3: Set Up a Django Consumer Now, let's create a Django service that acts as a message consumer, consuming email data from RabbitMQ and sending emails.

pip install pika

Create a Django script to consume email data from RabbitMQ:

# consumers.py

import pika
import json

def send_email(email_data):
    # Implement your email sending logic here
    pass

def process_email(channel, method, properties, body):
    email_data = json.loads(body)
    send_email(email_data)
    print(f'Email sent: {email_data}')

def consume_emails():
    rabbitmq_url = 'amqp://localhost'
    connection = pika.BlockingConnection(pika.URLParameters(rabbitmq_url))
    channel = connection.channel()
    channel.queue_declare(queue='email_queue')

    channel.basic_consume(queue='email_queue', on_message_callback=process_email, auto_ack=True)
    print('Email consumer is waiting for messages. To exit, press Ctrl+C')
    channel.start_consuming()

You'll need to call consume_emails to start the consumer, which will listen for email data and process it.

• Step 4: Running the System Start RabbitMQ: Run the RabbitMQ server on your local machine.

Start the Node.js producer: Run your Node.js service that sends email data to RabbitMQ.

Start the Django consumer: Run your Django service that consumes email data from RabbitMQ and sends emails.

With this setup, the Node.js service will send email data to RabbitMQ, and the Django service will consume the data from RabbitMQ and send emails.

Remember to implement the actual email-sending logic in the send_email function in your Django consumer. Also, adjust the RabbitMQ settings as needed for your production environment.

So with this, you can easily use a messaging queue to send out messages between services and work asynchronously. You can easily tell your interviewer how you have implemented a messaging queue in a normal college project.Giving you an edge over other candidates