Yggdrasil as Distributed PubSub

#elixir #tutorial #pubsub #distributedsystems

Yggdrasil's default adapter supports multi-node subscriptions out-of-the-box thanks to Phoenix PubSub. This distributed capabilities can be extended to any adapter compatible with Yggdrasil v5.0 without writing a single line of extra code.

Before We Start

I've used this example project for the code in this article. You can skip this section safely as long as you remember the following:

Basic project has only Yggdrasil.
Rabbit project has Yggdrasil for RabbitMQ.
A RabbitMQ server is available.
The host name is called matrix. Your machine's will be different.

If you want to follow along with the examples in this article, you can download the example project using the following command:

git clone \
  --depth 2 \
  -b blog \
  https://github.com/alexdesousa/alexdesousa.github.io.git examples && \
cd examples && \
git filter-branch \
  --prune-empty \
  --subdirectory-filter examples/matrix HEAD

In the folder you'll find:

Basic project that has a basic version of Yggdrasil.
Rabbit project that has Yggdrasil for RabbitMQ.
A docker compose with a RabbitMQ server:

   $ cd rabbit && docker-compose up

Basic Message Distribution

Yggdrasil's default adapter piggybacks on Phoenix PubSub for the message delivery, inheriting its distributed capabilities e.g. let's say we have the following:

The node :neo@matrix using Basic project:

   $ iex --sname neo -S mix

The node :smith@matrix also using Basic project:

   $ iex --sname smith -S mix

Both nodes are interconnected:

   iex(smith@matrix)1> Node.connect(:neo@matrix)
   true

Then :smith@matrix can subscribe to any channel where :neo@matrix is publishing messages e.g:

In :smith@matrix, we subscribe to the channel "private":

iex(smith@matrix)2> Yggdrasil.subscribe(name: "private")
:ok
iex(smith@matrix)3> flush()
{:Y_CONNECTED, %Yggdrasil.Channel{...}}
:ok

In :neo@matrix, we publish a message in the channel "private":

iex(neo@matrix)1> channel = [name: "private"]
iex(neo@matrix)2> Yggdrasil.publish(channel, "What's the Matrix?")
:ok

Finally, we can flush :smith@matrix mailbox and find our message:

iex(smith@matrix)4> flush()
{:Y_EVENT, %Yggdrasil.Channel{...}, "What's the Matrix?"}
:ok

Distributed pubsub as simple as that.

Bridged Message Distribution

The bridge adapter makes a bridge between any Yggdrasil adapter and the default adapter. This allows adapters to inherit the distributed capabilities of the default adapter e.g. let's say we have the following:

The node :neo@matrix using Basic project:

   $ iex --sname neo -S mix

The node :trinity@matrix using Rabbit project:

   $ iex --sname trinity -S mix

The node :trinity@matrix has access to a RabbitMQ server.
Both nodes are interconnected:

   iex(trinity@matrix)1> Node.connect(:neo@matrix)
   true

So our final infrastructure would look like the following:

Through :trinity@matrix, the node :neo@matrix can now subscribe to
a RabbitMQ exchange:

iex(neo@matrix)1> channel = [name: {"amq.topic", "private"}, adapter: :rabbitmq]
iex(neo@matrix)2> Yggdrasil.subscribe(channel)
:ok
iex(neo@matrix)3> flush()
{:Y_CONNECTED, %Yggdrasil.Channel{...}}
:ok

Or even publish messages:

iex(neo@matrix)4> Yggdrasil.publish(channel, "What's the Matrix?")
:ok
iex(neo@matrix)3> flush()
{:Y_EVENT, %Yggdrasil.Channel{...}, "What's the Matrix?"}
:ok