DEV Community

Cover image for Mind Your Business (Not Your Message Broker)

Mind Your Business (Not Your Message Broker)

You built an application. It works. Messages are flowing between services, and everything looks fine.

Then one day, a queue starts growing unexpectedly. Memory usage increases. A connection drops. Or you need to upgrade your message broker without interrupting the application.

Suddenly, instead of working on the feature you planned to build, you're investigating infrastructure.

Sound familiar?

Message brokers are an important part of many applications. They help services communicate asynchronously, distribute work, and handle workloads without requiring every component to be available at the same time.

But there's something we don't always talk about when choosing a message broker: who is going to operate it?

Open source gives you freedom

Take LavinMQ, for example.

LavinMQ is an open-source message broker that supports AMQP and MQTT. You can download it, install it on your own infrastructure, and use it without paying a software license fee.

Similar to RabbitMQ, you can self-host it and decide how it fits into your application architecture.

That's one of the things I appreciate about open-source software. You have the freedom to choose where and how to run it.

You might deploy LavinMQ on a virtual machine, in a container, or on infrastructure you already manage.

And for many developers, especially when building a prototype or a small application, getting a broker running is relatively straightforward.

But what happens when that application becomes important to your business?

The broker is free. Your time isn't.

Installing a message broker is one thing. Keeping it reliable in production is another.

Once your application depends on messaging, there are several things you need to consider:

  • Monitoring: Are queues growing faster than consumers can process messages? Are memory or disk resources becoming a problem?
  • Availability: What happens if the broker or the machine running it becomes unavailable?
  • Recovery: How quickly can your messaging system recover after a failure?
  • Upgrades: How do you apply updates while minimizing disruption?
  • Security: Who can connect to the broker, and what are they allowed to access?
  • Scaling: What happens when message volume increases beyond what your current setup can comfortably handle?

None of these challenges is impossible to solve.

If you have an experienced infrastructure team, managing your own broker might be exactly what you want.

But if you're a small development team, every hour spent investigating infrastructure is an hour you could have spent improving your application.

That's the trade-off I find interesting.

Open source can remove software licensing costs, but it doesn't remove operational responsibilities.

What about clustering and high availability?

Let's say your application initially runs with a single message broker.

Everything works well until that broker becomes unavailable.

Now your application may be unable to publish new messages or consume existing ones until connectivity is restored.

Depending on your requirements, you might consider a multi-node setup to improve availability.

But running a cluster introduces another set of questions.

How do you handle node failures? Which queues are replicated? How do clients reconnect? What happens during maintenance? And how do you know whether the cluster is healthy?

A cluster can improve availability, but it doesn't automatically make an application reliable.

Your application still needs to handle connection interruptions, message acknowledgements, retries, and potentially duplicate processing.

These are application-level responsibilities, regardless of whether you operate the broker yourself or use a managed service.

What changes with a managed message broker?

This is where managed services become interesting.

I work with LavinMQ at 84codes, the company behind CloudAMQP, so I should be transparent about my connection to the products I'm discussing.

CloudAMQP provides managed LavinMQ and RabbitMQ.

Instead of deploying and maintaining the message broker yourself, you can create a managed instance and connect your application to it.

The idea is simple: you focus on how messaging works in your application, while the provider takes care of much of the underlying infrastructure.

For example, CloudAMQP provides monitoring, alarms, managed maintenance, and cluster configurations for workloads that need higher availability.

It also offers 24/7 technical support, which can be particularly valuable when you're dealing with an unexpected production issue.

Of course, managed doesn't mean you can forget about reliability.

You still need to design your producers and consumers properly, understand message delivery guarantees, and decide how your application should respond to failures.

The difference is that you don't have to take responsibility for every operational detail of the broker itself.

Does it matter which cloud you're using?

I think it does.

Imagine your application is already running on AWS.

Would you want your message broker hosted somewhere completely unrelated, with unnecessary network distance between your services?

Probably not, unless you have a good reason.

Network latency, connectivity, data location, and infrastructure requirements can all influence where you deploy your broker.

CloudAMQP offers deployments on AWS, Google Cloud, Azure, DigitalOcean, and Scaleway, allowing teams to choose a cloud provider and region that fit their applications.

This is useful because choosing a managed message broker doesn't necessarily mean changing the cloud platform where the rest of your application runs.

Still, it's worth evaluating the actual network architecture, security requirements, and costs rather than assuming that choosing the same cloud provider solves everything.

Reliability is a shared responsibility

There's one point I particularly want to emphasize.

Whether you're using self-hosted LavinMQ, RabbitMQ, or a managed message broker, reliable messaging doesn't happen automatically.

Consider a consumer that receives a message, processes it successfully, but loses its connection before acknowledging it.

The broker may deliver that message again.

If your application isn't prepared for duplicate delivery, the same operation could be performed twice.

In a banking application, for example, that could be a serious problem.

This is why concepts such as acknowledgements, idempotency, retries, and dead-letter queues are important.

A managed service can help with infrastructure availability and operations. But it cannot replace good application design.

I recently explored some of these failure scenarios while building a small banking demo using LavinMQ and two Node.js AMQP clients.

The exercise reminded me that connection recovery and message-processing reliability are related, but they're not the same problem.

You need to think about both.

So, should you self-host or use a managed service?

I don't think there's one correct answer.

Self-hosting might make sense when:

  • You want full control over your infrastructure and configuration.
  • You already have the experience and resources to operate message brokers.
  • You have specific deployment or compliance requirements.
  • You're experimenting, learning, or running a workload that doesn't justify a managed service.

A managed service might make sense when:

  • You want to spend more time developing your application.
  • Your team has limited infrastructure capacity.
  • You need operational support, monitoring, and managed maintenance.
  • You prefer to delegate broker operations while keeping responsibility for your application architecture.

Cost matters too.

A managed service has a price, while self-hosting has infrastructure and operational costs that are sometimes less obvious.

It's worth comparing both rather than assuming one is always cheaper.

Mind your business

The title might sound a little funny, but there's a serious point behind it.

I don't mean developers should stop caring about infrastructure.

Quite the opposite.

If your application depends on messaging, you should understand how your broker works, how messages are delivered, and what happens when things fail.

But understanding your infrastructure and operating every part of it yourself are two different things.

Sometimes self-hosting is the right choice.

Sometimes delegating the operational work gives your team more time to build what matters to your users.

And sometimes the right decision changes as your application grows.

I'm curious how other developers approach this.

Do you prefer self-hosting your message brokers or using managed services? What influences your decision most: cost, control, reliability, or operational effort?

I'd love to hear about your experience, especially if you've tried both approaches.


Further reading

Top comments (0)

Some comments may only be visible to logged-in visitors. Sign in to view all comments.