Slimming down fat channels in Phoenix

Originally published on WyeWorks blog: https://www.wyeworks.com/blog/2019/07/19/slimming-down-fat-channels-in-phoenix/

In this article, we are going to discuss an approach we applied in a Phoenix project. Channels were an important part of the solution given that the application includes real-time collaboration features.

We had a specific channel module with an increasing number of message handler functions. In terms of design, we thought that having all the interactions managed by only one channel module still made sense. However, we started to notice the module itself was growing out of control, with a lot of non-related logic and a high number of lines of code. For this reason, we started to think about how to split the code across different modules, but in a way that still works using only one Phoenix channel.

The problem

Let's say we want to implement a multi-step wizard interface having some sort of collaborative interaction during each step. It means that everyone in the group will be working on the same step, at a given moment. Here is how we initially implemented this wizard.

defmodule MyExampleAppWeb.WizardChannel do
  use Phoenix.Channel

  def join("wizard:" <> id, _message, socket) do
    ...
  end

  # The message that begins the multi-step experience
  def handle_in("start_first_step", body, socket) do
    ...
  end

  # This message is valid in step 1
  def handle_in("send_info", body, socket) do
    ...
  end

  # This message is valid in step 1 but takes the group into step 2
  def handle_in("start_second_step", body, socket) do
    ...
  end

  # This message is valid in step 2
  def handle_in("pick_option", body, socket) do
    ...
  end
end

As you may have already noticed, this approach doesn't scale well. As more actions within steps and more steps themselves are added to the wizard, the channel module gets more and more complex.

Splitting the logic in different modules

Creating specific modules for each step is probably the most natural solution to our problem, so we tried it:

defmodule MyExampleAppWeb.WizardChannel do
  use Phoenix.Channel

  alias MyExampleAppWeb.WizardChannel.{FirstStep, SecondStep}
  ...

  def handle_in("start_first_step", body, socket) do
    FirstStep.start(data, socket)
  end

  def handle_in("send_info", body, socket) do
    FirstStep.send_info(data, socket)
  end

  ...
end

defmodule MyExampleAppWeb.WizardChannel.FirstStep do
  def start(body, socket) do
    ...
    broadcast(socket, "first_step_started", ...)
    {:noreply, socket}
  end

  def send_info(body, socket) do
    ...
    broadcast(socket, "info_received", ...)
    {:noreply, socket}
  end
end

Simple enough. We are just moving the logic to dedicated modules, and at the same time, adding some trivial code that delegates from the channel module. Well, there is a problem. This solution does not compile 🙈!

We have issues with the broadcast/3 function that is not found in the context of the FirstStep module. The same would happen if we were using any of the available functions provided by Phoenix.Channel like push/3, reply/2 and so on.

Looking for the missing functions

All those channel-specific functions are available in our WizardChannel because we have this line there:

use Phoenix.Channel

We can not simply do the same in our auxiliary modules, such as MyExampleAppWeb.WizardChannel.FirstStep, because this line does more than importing a bunch of functions: it defines a process that will be spawn on runtime and be responsible for handling all the messages going back and forth in our websocket connection.

The solution is quite easy. We can directly import the necessary functions defined in Phoenix.Channel module. There is no impediment to access to those functions and they're part of the public API of the framework (although it is easier to find examples of the full usage of Phoenix.Channel module in the official documentation)

A working solution for our FirstStep is the following:

defmodule MyExampleAppWeb.WizardChannel.FirstStep do

  #### Updated line ####
  import Phoenix.Channel, only: [broadcast: 3]

  def start(body, socket) do
    ...
    broadcast(socket, "first_step_started", ...)
    {:noreply, socket}
  end

  def send_info(body, socket) do
    ...
    broadcast(socket, "info_received", ...)
    {:noreply, socket}
  end
end

A new DSL is born

Let's look for a moment how our WizardChannel module looks after adding a few more steps and handler functions:

defmodule MyExampleAppWeb.WizardChannel do
  use Phoenix.Channel

  alias MyExampleAppWeb.WizardChannel.{FirstStep, SecondStep, LastStep}

  ...

  # First step messages
  def handle_in("start_first_step", body, socket), do: FirstStep.start(body, socket)
  def handle_in("send_info", body, socket), do: FirstStep.send_info(body, socket)

  # Second step messages
  def handle_in("start_second_step", body, socket), do: SecondStep.start(body, socket)
  def handle_in("pick_option", body, socket), do: SecondStep.pick_option(body, socket)
  def handle_in("notify_option", body, socket), do: SecondStep.notify_option(body, socket)

  ...

  # Last step messages
  def handle_in("confirm", body, socket), do: LastStep.confirm(body, socket)
  def handle_in("end_wizard", body, socket), do: LastStep.end_wizard(body, socket)

end

We noticed the code was mostly repetitive, having no much logic other than defining what the proper module and function are. Moreover, we were grouping the functions and adding those comment lines that refer to the different steps, so that the file was easier to navigate.

Here we started to see a chance to implement a custom DSL, in an attempt to have a more legible and maintainable piece of code. Elixir and its metaprogramming capabilities make possible to build DSLs with a few lines of code (although understanding that code would require to learn how macros work in Elixir).

Let's see how this module looked after implementing this fresh idea:

defmodule MyExampleAppWeb.WizardChannel do
  use Phoenix.Channel
  use MyExampleAppWeb.WizardChannel.MessagesHandler

  alias MyExampleAppWeb.WizardChannel.{FirstStep, SecondStep, LastStep}

  ...

  handle_step_messages [:start_first_step, :send_info], with_module: FirstStep
  handle_step_messages [:start_second_step, :pick_option, :notify_option], with_module: SecondStep
  handle_step_messages [:confirm, :end_wizard], with_module: LastStep
end

A lot of repetition has gone! And we're happy to see this code better communicates the intention (assuming certain familiarity with our custom DSL, of course).

How does it works? Note we have added the following to this module:

use MyExampleAppWeb.WizardChannel.MessagesHandler

Let's explore how the handle_step_messages macro is implemented, looking at the WizardChannel.MessagesHandler module code.

defmodule MyExampleAppWeb.WizardChannel.MessagesHandler do
  defmacro __using__(_opts) do
    quote do
      import unquote(__MODULE__)
    end
  end

  defmacro handle_step_messages(messages, with_module: module) do
    quote bind_quoted: [messages: messages, module: module] do
      Enum.each(messages, fn message ->
        def handle_in(unquote(Atom.to_string(message)), data, socket) do
          apply(unquote(module), unquote(message), [data, socket])
        end
      end)
    end
  end
end

The __using__ macro is invoked when the use directive is used. We're using that special macro just to make sure all the functions and macros from this module are available in our host module (which is MyExampleAppWeb.WizardChannel in our case).

Elixir macros are used to programmatically produce code that is injected where the macro is invoked at compile time. We have now a macro that generates all the functions we used to write by hand. It receives a list of atoms with the names of the messages and the module where the custom logic is implemented for each specific wizard step.

We are making use of a few conventions here. The functions in the different modules are equals to message names. For instance, the message type :send_info will be handled by the function FirstStep.send_info/2. Also, we're assuming that those functions have a fixed arity, receiving the message body and the Phoenix.Socket struct.

So, we are iterating over the message types list and generating a function definition for each one. The body of each generated function is the following

apply(unquote(module), unquote(message), [data, socket])

which is relying on Kernel.apply/3. It is possible to dynamically invoke the correct function from the specified module because the message names are passed now as variables instead of literals.

It's not the intention of this article to fully explain the metaprogramming aspect of this solution. If things like quotes and unquotes still puzzle you, I highly recommend reading the related documentation in the Elixir guides.

The result

After implementing this DSL we felt much better with this part of the code. We ended up with a decent way to divide the channel functionality in different modules, and also a clear way to write all the glue code in the channel module using our handle_step_messages/2 macro.

This approach opens new opportunities and challenges. For example, we have evolved the presented solution to support functions with different arity (some functions were not making use of the Phoenix.Socket struct parameter). In addition, we are now exploring some approaches to run shared validations depending on the target module. Anyway, taking the metaprogramming approach too far could lead to an over-engineered solution, hard to understand to other developers dealing with the code later, so it's clear we need to have a balance between code conciseness and simplicity.

Have you found similar issues with fat Phoenix channels? How did you manage it? Please comment if you have tried different solutions or if you find our story useful.

Happy coding with Elixir and Phoenix ‍👩🏽‍💻👨🏻‍💻!

Acknowledgements

Huge thanks to Nicolás Ferraro and Javier Morales for helping to write this article. Both of them were involved in the implementation of the described solution.