DEV Community

Cover image for Messaging and Madness: Sending Messages with AMQP and Amazon MQ
George Offley for AWS Community Builders

Posted on

7 1

Messaging and Madness: Sending Messages with AMQP and Amazon MQ

Originally posted on my personal blog

Table Of Contents

Introduction

How do software systems talk to each other? Back-end systems can scale into giant melted together Cronenberg monsters, often making up different tools and languages. So, communicating between these services can become an untenable challenge without some shared vocabulary. We can communicate in many ways, but today I wanted to talk about asynchronous messaging protocols and figure out how AWS can help.

AMQP

AMQP stands for Advanced Message Queuing Protocol. I’ve been working to implement it for some back-end software suites I’m building out to enable them to talk to each other. AMQP utilizes these things called brokers to publish messages on, then on the other end, a receiving service subscribed to the same “channel” that we posted to can pick up that message.

Hello World Visualization
via Rabbit MQ Tutorials

Let’s dive a little further down; the publisher service publishes a message to an exchange on a broker. This exchange has routes that lead to queues, or “channels,” where the payload is published. We make sure to include the sending information with our message to be routed to the correct queue. The broker cannot see the message, although it might look into any metadata attached to the message from the publisher. This workflow asynchronously sends messages. Imagine a server version of a mail sorting machine shooting letters into the correct mail slot based on the address.

Mail Sorting Gif from MIB II

When referring to a publisher, I mean some code that we utilize to connect and send a message. AMQP is programmable, so I can shape it to fit most situations. In this case, we need to send messages to our different software suites to trigger actions to happen. Learning this took some time, but it’s been simple to implement.

There are different types of exchanges that we can use to make these services fit our needs. I’m going to explain what we use briefly.

We use a direct exchange utilizing routing keys to bind queues to exchanges. Our code can use direct exchanges to distribute tasks to many different endpoints, but we used these direct exchanges to make direct routes between our services. Other types of exchanges can be used to broadcast messages. More information can be found here. For now, we’re going to focus on direct exchanges.

AMQP and Amazon MQ

We touched on all that because I wanted to talk about Amazon MQ. Amazon MQ is a fully managed platform for setting up message brokers. Amazon MQ utilizes both RabbitMQ and Apache Active MQ for creating brokers. We’re sticking with Rabbit MQ for the time being.

Amazon MQ Dashboard

Here above, you can see you can easily set up a broker in just a few clicks. I left most of the settings on default, except for choosing “RabbitMQ” for our broker engine and setting some security up for accessing our management console.

Rabbit MQ Management Console

Once we get that, we have access to the RabbitMQ dashboard Amazon MQ created and is managing. Now that we have a broker set up, we can play with some code.

Code

Above I use the library Kombu to create some connections and send some stuff. I started by setting up our environment variables. Then created exchange and queue objects. Finally, I made our connection object and the producer object, and then we sent a simple “Hello” message.

Serialization

Serialization is another blog post, but I chose to use JSON to serialize the payload. In the production software, I use a combination of JSON and Pickle to serialize things like image data.

Now we can see our message published on the queue I declared in our publisher service. An identical receiving service would be set up on the other side to read out messages sent to that queue.

Results

Conclusion

In conclusion, using Amazon MQ allows us to set up managed brokers for us to send messages. With AMQP as the broker engine, we have a lightweight message-sending workflow. Thanks for reading.

-George

Image of Timescale

🚀 pgai Vectorizer: SQLAlchemy and LiteLLM Make Vector Search Simple

We built pgai Vectorizer to simplify embedding management for AI applications—without needing a separate database or complex infrastructure. Since launch, developers have created over 3,000 vectorizers on Timescale Cloud, with many more self-hosted.

Read full post →

Top comments (0)

Best Practices for Running  Container WordPress on AWS (ECS, EFS, RDS, ELB) using CDK cover image

Best Practices for Running Container WordPress on AWS (ECS, EFS, RDS, ELB) using CDK

This post discusses the process of migrating a growing WordPress eShop business to AWS using AWS CDK for an easily scalable, high availability architecture. The detailed structure encompasses several pillars: Compute, Storage, Database, Cache, CDN, DNS, Security, and Backup.

Read full post