DEV Community: puneet kumar

Microservices Ki Kahaani Part 2: Messaging - When REST & gRPC Need a Break!

puneet kumar — Thu, 01 May 2025 10:56:20 +0000

In our last episode (Microservices Ki Shaadi!), we saw how services can talk directly using REST (the reliable Raj) and gRPC (the speedy Simran). That's synchronous communication – like making a phone call 📞, you wait for the other person to pick up and reply.

But what if your service is too busy gossiping (processing requests) and can't answer right away? What if you just want to leave a message and carry on with your life, like sending a WhatsApp message 📱 instead of calling?

Welcome to the world of Asynchronous Messaging! This is how microservices can communicate without getting stuck waiting for each other, making your whole system more resilient and scalable – like having a reliable courier service handle deliveries instead of doing everything yourself.

How Messaging Works: The Asynchronous Chit-Chat ✉️

Imagine Service A needs Service B to do something. Instead of calling B directly and waiting, A just writes a message (like a digital chitthi) and drops it off. Service A then goes back to its own work (bhai, time nahi hai wait karne ka!).

Later, when Service B is free, it picks up the message, does the task, and maybe sends a reply message back if needed.

Caption: Service A sending a message and chilling, not waiting.

Usually, there's a middleman involved – a Message Broker (like Kafka, RabbitMQ). Think of it as a super-efficient post office 🏤 that takes messages from senders and delivers them to the right receivers. We'll talk more about these brokers later.

Message Channels: The Secret Tunnels of Communication 🚇

How do messages actually travel? A brilliant concept from the book Enterprise Integration Patterns describes Message Channels.

Think of a channel as a dedicated pipe or tunnel connecting senders and receivers.

Sender: Writes a message and puts it into the channel.
Receiver: Picks up the message from the channel.

Simple, right? The channel acts as the bridge.

Caption: The basic Message Channel concept.

Anatomy of a Message: Kya Hai Andar? 🤔

Every message is like a little package with two parts:

Header (Metadata): Like the address and stamps on an envelope. Contains info about the message:
- messageId: Unique ID (like a tracking number).
- timestamp: When it was sent.
- returnAddress: Where to send the reply (if any).
- Other custom details.
Body (Payload): The actual content – the data itself! This can be text (JSON, XML) or binary (like Protocol Buffers we saw with gRPC).

Types of Messages: Different Flavours of Data 🍬

Not all messages are created equal:

Document Message: Just carries data. The receiver figures out what to do. Think of sending someone your Aadhar card copy – they decide how to use it. Often used for replies.
Command Message: Asks the receiver to do something specific. Like telling your friend, "Order pizza!" 🍕 Includes necessary details (address, toppings). Similar to a remote procedure call, but async.
Event Message: Announces that something happened. "Order Placed!" 🎉, "User Registered!". Common in Event-Driven Architectures. Multiple services might react to the same event.

In modern microservices, Commands and Events are super common. They help keep services independent (loosely coupled) and reactive.

Channels Deep Dive: The Plumbing System 🔧

Let's look closer at how services interact with channels:

Sender Side:
- Business Logic -> Sending Port (Hides complexity) -> Sender Adapter (Actually sends) -> Channel
Receiver Side:
- Channel -> Handler Adapter (Listens) -> Receiving Port (Triggers logic) -> Business Logic

Multiple senders can write to the same channel, and multiple receivers can listen! Flexibility max.

Channel Types: Point-to-Point vs. Publish/Subscribe 📢

There are two main ways channels deliver messages:

Point-to-Point (P2P) Channel:
- What: Delivers each message to only one receiver, even if multiple are listening.
- Analogy: A task queue where only one worker picks up each job. Like a single cashier counter at a bank 🏦.
- Use Case: Perfect for Command messages. You want only one service to process the "Create Order" command.
Publish/Subscribe (Pub/Sub) Channel:
- What: Broadcasts each message to all interested receivers (subscribers).
- Analogy: A WhatsApp group announcement 📢. Everyone in the group gets the message.
- Use Case: Ideal for Event messages. When an "Order Placed" event happens, Billing, Inventory, and Notifications services might all need to know and react.

Caption: Point-to-Point (Task Queue) vs. Publish/Subscribe (Broadcast).

Messaging for Different Interactions (Even Replies!) 🔄

Messaging is super flexible and can handle all the interaction styles we discussed before (sync/async, one-to-one, one-to-many).

How does it handle Request/Reply asynchronously?

It's clever! The client doesn't block (wait).

Client Sends Command: Sends a command message to the service via a request channel (P2P). This message includes:
- A unique messageId.
- A returnAddress (the name of the client's reply channel).
- The request data in the body.
Service Processes: Reads the message, does the work.
Service Sends Reply: Sends a separate reply message (Document type) back to the client's specified returnAddress channel. This reply includes:
- A correlationId (which matches the original messageId from the request).
- The response data in the body.
Client Matches Reply: Listens on its reply channel. Uses the correlationId to match the response to the original request it sent earlier. Pata chal gaya kaunsa jawab kis sawaal ka hai!

Caption: Asynchronous Request-Reply using two channels and Correlation ID.

This way, the client never waits, but still gets the response eventually. Beautifully decoupled! One-way notifications (no reply needed) are even simpler – just send and forget! Pub/Sub handles one-to-many notifications easily.

Brokerless vs. Broker-Based: Direct Dial or Post Office? ☎️ vs 🏤

How do messages physically get from A to B? Two main approaches:

Brokerless:
- What: Services talk directly to each other (using libraries like ZeroMQ). No middleman.
- Pros: Low latency (direct path), less infrastructure to manage (no broker setup).
- Cons:
  - Services need to know each other's locations (requires Service Discovery).
  - If sender or receiver is down, message is lost! 😱 Poof!
  - Implementing reliability (retries, guaranteed delivery) is your headache. Complex!
Broker-Based:
- What: Uses a dedicated Message Broker (RabbitMQ, Kafka, ActiveMQ, cloud services like AWS SQS/SNS, Google Pub/Sub, Azure Service Bus) as an intermediary.
- Pros:
  - Loose Coupling: Sender just sends to the broker, doesn't care who receives it or where they are. No Service Discovery needed!
  - Reliability: Brokers handle retries, persistence (saving messages to disk), guaranteed delivery.
  - Availability/Buffering: Broker holds messages if the receiver is down or slow. Your app keeps working! (Like the post office holding mail if you're not home).
  - Scalability: Brokers are designed to handle high volumes.
- Cons:
  - Potential bottleneck (though modern brokers scale well).
  - Broker itself needs to be managed (setup, monitoring, HA). Adds operational overhead.
  - Slightly higher latency than direct connection (usually negligible).

Caption: Brokerless (Direct Chaos) vs. Broker-Based (Organized Delivery).

The Verdict? For most real-world, scalable, and resilient systems, Broker-Based is the way to go. The benefits usually far outweigh the operational cost. It's the standard for enterprise applications.

Broker Implementations: Different Names, Same Game?

Different brokers implement channels slightly differently:

JMS (e.g., ActiveMQ): Uses Queues (P2P) and Topics (Pub/Sub).
RabbitMQ (AMQP): Uses Exchanges that route messages to Queues. Flexible routing!
Kafka: Uses Topics (which can be consumed P2P-style with consumer groups or Pub/Sub style). Built for high throughput streaming.
AWS SQS: Only Queues (P2P). Pair with AWS SNS (Pub/Sub) for broadcast.
Google Pub/Sub, Azure Service Bus: Support both P2P and Pub/Sub via Subscriptions to topics.

While the names differ, most support both P2P and Pub/Sub communication styles.

Downsides? Haan Bhai, Kuch Toh Hai...

Messaging isn't perfect:

Broker Bottleneck: The broker can become a bottleneck if not scaled properly (but they are designed to scale).
Broker Failure: If your broker goes down (and isn't set up for High Availability - HA), communication stops! Running it reliably is crucial.
Operational Overhead: It's another piece of infrastructure to manage.
Complexity: Debugging distributed message flows can be trickier than direct calls.

But still, for building systems that need to be reliable and scale gracefully, the advantages are HUGE.

Conclusion & What's Next? 🤔

Asynchronous messaging is a powerful tool for building decoupled, resilient, and scalable microservices. It allows services to communicate without waiting, handle failures gracefully, and react to events independently. Using a message broker is typically the best approach for achieving this.

But wait, there's more! Handling messages isn't always simple. What happens if a message needs to be processed only once, but gets delivered twice? How do you keep messages in the right order? How do you handle transactions that involve multiple services and messages?

In the next part, we'll tackle these real-world challenges: duplicate messages, message ordering, idempotency, distributed tracing, and transactional messaging. Stay tuned for more microservice gyaan!

Found this useful? Learned something new? Let me know your thoughts or questions about asynchronous messaging in the comments! 👇

Choosing Your Microservice API Style: When REST Reigns vs. When gRPC Rules. 🤔

puneet kumar — Tue, 29 Apr 2025 10:11:10 +0000

So, you're building microservices? Awesome! Welcome to the party where your tiny little services need to talk to each other. Think of it like a big, fat Indian wedding – lots of different relatives (services) need to coordinate, right? If Uncle Sharma from Accounts doesn't talk properly to Aunty Rita from HR, chaos ensues! 😱 That communication between services is called Interprocess Communication (IPC), and choosing the right way for them to talk is super important. Get it wrong, and your app might run slower than Mumbai traffic during monsoon!

Your microservices trying to communicate without a clear plan.

Today, we're looking at two popular "matchmaking" styles for your services: the ever-reliable REST and the super-speedy gRPC. Which one is the perfect jodi for your microservices? Let's find out! 👇

Pehle Yeh Batao: Why Even Bother? (Sync vs. Async Basics)

Before we meet the contenders, let's understand the basic ways services chat:

Synchronous (Like a Phone Call 📞): Service A calls Service B and waits right there, holding the line, until Service B answers. Quick response needed! Think HTTP-based REST or gRPC.
Asynchronous (Like Sending a WhatsApp Message 📱): Service A sends a message to Service B and goes back to its chai. It doesn't wait for an immediate reply. Think message queues like Kafka or RabbitMQ (we'll talk about these superstars in another post!).

Choosing between these affects how fast and reliable your app is.

Interaction Styles: The Dance Moves of Microservices 💃🕺

How do services actually interact? It's not just about calling; it's about how they call:

One-to-One (Ek Se Ek):
- Request-Response: Client asks, waits, gets an answer (like ordering biryani and waiting for it 🍛). Simple, but can couple things tightly.
- Asynchronous Request-Response: Client asks, goes off to do other things, gets the answer later (like ordering online and getting a delivery notification 🚚). More flexible!
- One-Way Notification: Client sends info, doesn't expect anything back (like sending a "Good Morning" forward on WhatsApp - okay, maybe a bad example, but you get it! 🙏).
One-to-Many (Ek Se Anek):
- Publish/Subscribe (Pub/Sub): One service shouts out news (publishes), and anyone interested listens (subscribes). Think of a group chat announcement 📢.
- Publish/Async Responses: Client shouts out a question, waits a bit for anyone interested to reply.

Real apps mix and match these styles like adding different spices to a dish! 🌶️

API-First Design: The Shaadi Ka Card Pehle! 💌

Imagine planning that big fat Indian wedding without finalising the guest list or the venue first. Disaster, right? That's what happens when you build services without defining how they'll talk first.

API-First Design is like designing the shaadi ka card (wedding invitation) before booking the caterer.

Define the API (how services will interact, what data they need).
Share it with everyone involved (frontend team, other backend teams).
Get feedback, make changes.
Then, and only then, start building the actual service.

Why? It avoids that classic situation where the Java backend team and the Angular frontend team both say "Done!", but when they try to connect... dhappa! Nothing works because they expected different things from the API. Seriously, tension nahi lene ka, just define the API first! 🧘

Avoid this communication gap!

Message Formats: Hindi Mein Ya Code Mein? (Text vs. Binary)

When services talk, they exchange messages (data). The format of this message matters!

Text-Based (JSON, XML):
- Pros: Human-readable (like Hindi or English 🇮🇳), flexible (easy to add new stuff without breaking old things). Think of JSON like reading a menu.
- Cons: Can be a bit bulky and slower.
- Schema: You can define the structure using things like JSON Schema, like a grammar guide.
Binary (Protocol Buffers, Avro):
- Pros: Super-efficient, compact, fast (like speaking in a secret code 🤫). Forces API-First design! Great for performance.
- Cons: Not human-readable (need tools to understand).
- IDL (Interface Definition Language): You define the structure strictly upfront (e.g., using .proto files for Protocol Buffers).

Choose your message format wisely.

Contender #1: REST - The Reliable Raj of APIs 🤵

REST (Representational State Transfer) is like the Shah Rukh Khan of APIs – familiar, widely loved, and gets the job done reliably. ❤️

How it works: Uses standard HTTP (the language of the web). You treat everything as a "resource" (like a customer, an order). Use standard HTTP verbs (GET to fetch, POST to create, PUT to update, DELETE to remove).
- Example: POST /orders (Create a new order), GET /orders/123 (Get details of order 123).
Pros:
- Simple & Familiar: Everyone knows HTTP, like everyone knows how to make Maggi 🍜.
- Easy Testing: Test with your browser, Postman, curl - easy peasy!
- Firewall Friendly: Usually works without special network setup.
- No Broker Needed: Direct communication, simpler architecture initially.
Cons:
- Synchronous Only: Only supports the "wait for response" style. Can affect availability if a service is slow or down.
- Client Needs Location: Client needs to know where the service is (needs Service Discovery).
- Multiple Resources: Fetching lots of different data in one go can be inefficient (like making multiple trips to the market 🛍️).
- Mapping Complexity: Sometimes mapping complex actions to simple GET/POST/PUT is awkward.

Use REST when: Simplicity is key, you need wide compatibility (especially browsers), and basic request-response is enough. Think public-facing APIs, simple CRUD operations. Like ordering food from Swiggy/Zomato – a standard request-response interaction. 🍔

Contender #2: gRPC - The Speedy Simran of APIs 🚀

gRPC (Google Remote Procedure Call) is like the new-gen superstar – fast, efficient, and packed with features. Developed by Google. ✨

How it works: Uses Protocol Buffers (that efficient binary format) by default. Define your API strictly in a .proto file (API-First!). Uses HTTP/2 for speed.
Pros:
- High Performance: Way faster and uses less data than REST/JSON. Think Mumbai local train speed vs. road traffic 🚄💨.
- Streaming: Supports real-time data flows! Client-to-server, server-to-client, or both ways (Bi-directional). Imagine getting live cricket score updates pushed to your app! 🏏
- Code Generation: Automatically generates client/server code in many languages (Java, Go, Python, Node.js etc.) – less manual work, fewer integration bugs!
- Strict API Contract: The .proto file ensures everyone is on the same page.
Cons:
- Browser Support: Doesn't work directly in browsers easily (needs gRPC-Web or a gateway).
- Not Human Readable: Binary format is hard to inspect manually.
- Infrastructure: Might need network/firewall adjustments for HTTP/2.

Use gRPC when: Performance is critical, you have internal microservice-to-microservice communication, you need streaming capabilities, or you work in multiple programming languages. Think high-frequency trading systems, IoT data streams, internal backends. 📈

Toh Karna Kya Hai? REST ya gRPC? 🤔

Okay, the million-rupee question! Here’s a very simplified guide:

Simplified Flow:

Need direct browser communication?
- YES: Use REST (or consider gRPC-Web as an alternative).
- NO: Go to step 2.
Is high performance or streaming a MUST?
- YES: Use gRPC.
- NO: Go to step 3.
Is simplicity & broad familiarity the top priority?
- YES: Use REST.
- NO: gRPC is still a strong contender, weigh its

benefits.

Short Answer:

Public APIs / Browser Clients? Start with REST.
Internal Microservices / Performance Critical / Streaming Needed? Lean towards gRPC.
Mixed Environment? You might use both! REST for public/browser, gRPC for internal communication. It's not always either/or! 😉

Caption: The eternal dilemma.

Cliffhanger! What About Asynchronous? 🕒

Remember that WhatsApp message style (Asynchronous)? Sometimes, you don't want services waiting on each other. You want them to drop off a message and move on, letting things process in the background. That's where tools like Kafka and RabbitMQ come in! 🚀 But that, my friends, is a story for another day... (Stay tuned for the sequel!). 😉

The Final Word (For Now!)

Choosing between REST and gRPC isn't about which one is "better" – it's about which one is the better jodi for your specific needs, just like choosing the right life partner (okay, maybe not that serious, but still important!). 😂

Consider your performance needs, your communication patterns (request-response vs. streaming), your team's familiarity, and whether you're talking to browsers or just internal services.

Hopefully, this gave you a clearer (and more fun!) picture. Don't just blindly follow the crowd – understand the fundas and choose wisely! 🧠

What are your experiences? Team REST or Team gRPC? Let me know your thoughts and war stories in the comments below! 👇