DEV Community: Jorge Bejar

Slimming down fat channels in Phoenix

Jorge Bejar — Thu, 08 Aug 2019 16:46:27 +0000

Originally published on WyeWorks blog: https://www.wyeworks.com/blog/2019/07/19/slimming-down-fat-channels-in-phoenix/

In this article, we are going to discuss an approach we applied in a Phoenix project. Channels were an important part of the solution given that the application includes real-time collaboration features.

We had a specific channel module with an increasing number of message handler functions. In terms of design, we thought that having all the interactions managed by only one channel module still made sense. However, we started to notice the module itself was growing out of control, with a lot of non-related logic and a high number of lines of code. For this reason, we started to think about how to split the code across different modules, but in a way that still works using only one Phoenix channel.

The problem

Let's say we want to implement a multi-step wizard interface having some sort of collaborative interaction during each step. It means that everyone in the group will be working on the same step, at a given moment. Here is how we initially implemented this wizard.

defmodule MyExampleAppWeb.WizardChannel do
  use Phoenix.Channel

  def join("wizard:" <> id, _message, socket) do
    ...
  end

  # The message that begins the multi-step experience
  def handle_in("start_first_step", body, socket) do
    ...
  end

  # This message is valid in step 1
  def handle_in("send_info", body, socket) do
    ...
  end

  # This message is valid in step 1 but takes the group into step 2
  def handle_in("start_second_step", body, socket) do
    ...
  end

  # This message is valid in step 2
  def handle_in("pick_option", body, socket) do
    ...
  end
end

As you may have already noticed, this approach doesn't scale well. As more actions within steps and more steps themselves are added to the wizard, the channel module gets more and more complex.

Splitting the logic in different modules

Creating specific modules for each step is probably the most natural solution to our problem, so we tried it:

defmodule MyExampleAppWeb.WizardChannel do
  use Phoenix.Channel

  alias MyExampleAppWeb.WizardChannel.{FirstStep, SecondStep}
  ...

  def handle_in("start_first_step", body, socket) do
    FirstStep.start(data, socket)
  end

  def handle_in("send_info", body, socket) do
    FirstStep.send_info(data, socket)
  end

  ...
end

defmodule MyExampleAppWeb.WizardChannel.FirstStep do
  def start(body, socket) do
    ...
    broadcast(socket, "first_step_started", ...)
    {:noreply, socket}
  end

  def send_info(body, socket) do
    ...
    broadcast(socket, "info_received", ...)
    {:noreply, socket}
  end
end

Simple enough. We are just moving the logic to dedicated modules, and at the same time, adding some trivial code that delegates from the channel module. Well, there is a problem. This solution does not compile 🙈!

We have issues with the broadcast/3 function that is not found in the context of the FirstStep module. The same would happen if we were using any of the available functions provided by Phoenix.Channel like push/3, reply/2 and so on.

Looking for the missing functions

All those channel-specific functions are available in our WizardChannel because we have this line there:

use Phoenix.Channel

We can not simply do the same in our auxiliary modules, such as MyExampleAppWeb.WizardChannel.FirstStep, because this line does more than importing a bunch of functions: it defines a process that will be spawn on runtime and be responsible for handling all the messages going back and forth in our websocket connection.

The solution is quite easy. We can directly import the necessary functions defined in Phoenix.Channel module. There is no impediment to access to those functions and they're part of the public API of the framework (although it is easier to find examples of the full usage of Phoenix.Channel module in the official documentation)

A working solution for our FirstStep is the following:

defmodule MyExampleAppWeb.WizardChannel.FirstStep do

  #### Updated line ####
  import Phoenix.Channel, only: [broadcast: 3]

  def start(body, socket) do
    ...
    broadcast(socket, "first_step_started", ...)
    {:noreply, socket}
  end

  def send_info(body, socket) do
    ...
    broadcast(socket, "info_received", ...)
    {:noreply, socket}
  end
end

A new DSL is born

Let's look for a moment how our WizardChannel module looks after adding a few more steps and handler functions:

defmodule MyExampleAppWeb.WizardChannel do
  use Phoenix.Channel

  alias MyExampleAppWeb.WizardChannel.{FirstStep, SecondStep, LastStep}

  ...

  # First step messages
  def handle_in("start_first_step", body, socket), do: FirstStep.start(body, socket)
  def handle_in("send_info", body, socket), do: FirstStep.send_info(body, socket)

  # Second step messages
  def handle_in("start_second_step", body, socket), do: SecondStep.start(body, socket)
  def handle_in("pick_option", body, socket), do: SecondStep.pick_option(body, socket)
  def handle_in("notify_option", body, socket), do: SecondStep.notify_option(body, socket)

  ...

  # Last step messages
  def handle_in("confirm", body, socket), do: LastStep.confirm(body, socket)
  def handle_in("end_wizard", body, socket), do: LastStep.end_wizard(body, socket)

end

We noticed the code was mostly repetitive, having no much logic other than defining what the proper module and function are. Moreover, we were grouping the functions and adding those comment lines that refer to the different steps, so that the file was easier to navigate.

Here we started to see a chance to implement a custom DSL, in an attempt to have a more legible and maintainable piece of code. Elixir and its metaprogramming capabilities make possible to build DSLs with a few lines of code (although understanding that code would require to learn how macros work in Elixir).

Let's see how this module looked after implementing this fresh idea:

defmodule MyExampleAppWeb.WizardChannel do
  use Phoenix.Channel
  use MyExampleAppWeb.WizardChannel.MessagesHandler

  alias MyExampleAppWeb.WizardChannel.{FirstStep, SecondStep, LastStep}

  ...

  handle_step_messages [:start_first_step, :send_info], with_module: FirstStep
  handle_step_messages [:start_second_step, :pick_option, :notify_option], with_module: SecondStep
  handle_step_messages [:confirm, :end_wizard], with_module: LastStep
end

A lot of repetition has gone! And we're happy to see this code better communicates the intention (assuming certain familiarity with our custom DSL, of course).

How does it works? Note we have added the following to this module:

use MyExampleAppWeb.WizardChannel.MessagesHandler

Let's explore how the handle_step_messages macro is implemented, looking at the WizardChannel.MessagesHandler module code.

defmodule MyExampleAppWeb.WizardChannel.MessagesHandler do
  defmacro __using__(_opts) do
    quote do
      import unquote(__MODULE__)
    end
  end

  defmacro handle_step_messages(messages, with_module: module) do
    quote bind_quoted: [messages: messages, module: module] do
      Enum.each(messages, fn message ->
        def handle_in(unquote(Atom.to_string(message)), data, socket) do
          apply(unquote(module), unquote(message), [data, socket])
        end
      end)
    end
  end
end

The __using__ macro is invoked when the use directive is used. We're using that special macro just to make sure all the functions and macros from this module are available in our host module (which is MyExampleAppWeb.WizardChannel in our case).

Elixir macros are used to programmatically produce code that is injected where the macro is invoked at compile time. We have now a macro that generates all the functions we used to write by hand. It receives a list of atoms with the names of the messages and the module where the custom logic is implemented for each specific wizard step.

We are making use of a few conventions here. The functions in the different modules are equals to message names. For instance, the message type :send_info will be handled by the function FirstStep.send_info/2. Also, we're assuming that those functions have a fixed arity, receiving the message body and the Phoenix.Socket struct.

So, we are iterating over the message types list and generating a function definition for each one. The body of each generated function is the following

apply(unquote(module), unquote(message), [data, socket])

which is relying on Kernel.apply/3. It is possible to dynamically invoke the correct function from the specified module because the message names are passed now as variables instead of literals.

It's not the intention of this article to fully explain the metaprogramming aspect of this solution. If things like quotes and unquotes still puzzle you, I highly recommend reading the related documentation in the Elixir guides.

The result

After implementing this DSL we felt much better with this part of the code. We ended up with a decent way to divide the channel functionality in different modules, and also a clear way to write all the glue code in the channel module using our handle_step_messages/2 macro.

This approach opens new opportunities and challenges. For example, we have evolved the presented solution to support functions with different arity (some functions were not making use of the Phoenix.Socket struct parameter). In addition, we are now exploring some approaches to run shared validations depending on the target module. Anyway, taking the metaprogramming approach too far could lead to an over-engineered solution, hard to understand to other developers dealing with the code later, so it's clear we need to have a balance between code conciseness and simplicity.

Have you found similar issues with fat Phoenix channels? How did you manage it? Please comment if you have tried different solutions or if you find our story useful.

Happy coding with Elixir and Phoenix ‍👩🏽‍💻👨🏻‍💻!

Acknowledgements

Huge thanks to Nicolás Ferraro and Javier Morales for helping to write this article. Both of them were involved in the implementation of the described solution.

To use or not to use the ++ operator in Elixir

Jorge Bejar — Tue, 05 Mar 2019 17:32:04 +0000

Originally published on WyeWorks blog: https://www.wyeworks.com/blog/2019/03/01/to-use-or-not-to-use-the-++-operator-in-elixir/

You may have already been warned about the risks of using the ++ operator to concatenate lists in Elixir. In fact, a good piece of advice can be found directly in the official documentation for this method:

The complexity of a ++ b is proportional to length(a), so avoid repeatedly appending to lists of arbitrary length, e.g. list ++ [item]. Instead, consider prepending via [item | rest] and then reversing.

For the rest of this article, we are going to dive into an example in an attempt to explain what could be problematic with using ++, while also checking if we can improve on things by refactoring as suggested in the documentation. Spoiler alert: life is not that easy 😅.

The problem

Let's say we need to implement a function that receives a list of integers and returns a new list that includes each number n times, where n equals is the integer's actual value. We also expect that the original order of these numbers is preserved in the resulting list. Check out the following examples:

my_func([1]) # it returns [1]
my_func([1,2]) # it returns [1,2,2]
my_func([1,3,2]) # it returns [1,3,3,3,2,2]

Our first naive implementation

A very simple solution to this problem could rely on using Enum.reduce/3 to duplicate each number and concatenate the duplicated values to the accumulated list.

def number_dups_list(numbers) do
  Enum.reduce(numbers, [], fn n, acc ->
    acc ++ List.duplicate(n, n)
  end)
end

Let's see how it performs by running this function with a large list of integers. Below is our first test (using :timer.tc to quickly measure the time elapsed to run the code):

iex> :timer.tc(fn -> number_dups_list(List.duplicate(1,100_000)) end)
{30785140, .... }

After running it a few times in order to get a few samples, I noted that it takes around 3 seconds (on my machine). That's pretty slow; there might be room for improvement.

Let's follow the advice from the documentation

If we look carefully at the execution of our previous test, where the input was a list containing a lot of 1s, we see that our implementation is using ++ repeatedly. In fact, the crucial factor here is that the list to the left of the operator is getting larger and larger with each iteration. At the same time, the list to the right is always [1]. As the documentation states, the overall performance depends on the length of the list to the left of this operator.

Let's see what happens if we reverse the order (then use Enum.reverse to re-sort).

def number_dups_list(numbers) do
  Enum.reduce(numbers, [], fn n, acc ->
    List.duplicate(n, n) ++ acc
  end)
  |> Enum.reverse
end

Let's run it again to see if we have enhanced performance:

iex> :timer.tc(fn -> number_dups_list(List.duplicate(1,100_000)) end)
{11570, .... }

On my machine, this executes in approximately 11ms! This is a huge difference! The larger the list is, the more is the difference with the previous approach.

All good so far, but... are we sure that we’re going to see better performance in all cases? Chances are, we're not out of the woods yet. Let's run our function again using different input.

iex()> :timer.tc(fn -> number_dups_list([10_000_000, 20_000_000]) end)
{2638101, .... }

You may have already noticed the issue! We're using a rather short list now, but the numbers are very big numbers. Thus, we're once again in a position where the list to the left of the ++ operator is always a large list, resulting in slower execution of the function (around 2 seconds).

The funny thing here is that our former implementation works a bit better for this new list (less than 1.5 seconds). It is only at the first stage of the execution where the list to the left of the ++ operator is a short list (an empty list). These results suggest that reversing makes a difference here; the previous implementation (where reversing is not needed) is preferable for this type of input.

At this point, we want to start measuring performance in a more robust way. We’ll use Benchee to handle all the benchmarking work for us. See below for the benchmarking (assume this function is included in the same module and note we are now using different function names for each implementation).

def run_benchmarks do
  inputs = %{
    "long list with low values" => List.duplicate(1,100_000),
    "short list with high values" => [10_000_000, 20_000_000]
  }

  Benchee.run %{
    "Using ++" =>
      fn(list) -> number_dups_list1(list) end,
    "Using ++ and reverse"  =>
      fn(list) -> number_dups_list2(list) end,
  }, time: 20, warmup: 5, inputs: inputs
end

Running this code on my computer gives me the following output:

##### With input long list with low values #####
Name                           ips        average  deviation         median         99th %
Using ++ and reverse        123.53      0.00810 s    ±18.33%      0.00824 s       0.0127 s
Using ++                    0.0366        27.35 s     ±0.00%        27.35 s        27.35 s

Comparison:
Using ++ and reverse        123.53
Using ++                    0.0366 - 3378.87x slower

##### With input short list with high values #####
Name                           ips        average  deviation         median         99th %
Using ++                      0.93         1.07 s    ±41.40%         1.00 s         2.38 s
Using ++ and reverse          0.50         2.01 s    ±29.48%         2.01 s         3.48 s

Comparison:
Using ++                      0.93
Using ++ and reverse          0.50 - 1.88x slower

This confirms what we saw before, specifically that our first implementation is faster with a short list of large numbers. As we mentioned previously, the difference is invoking Enum.reverse in the case of a long list (you can easily verify this by commenting out this part and checking that the benchmarking results are now quite similar, even though the functions don't produce the exact same results).

What’s the problem with the lists on the left?

A further explanation of this issue would require a blog post of its own 😀.

Let's just say that the Erlang VM is forced to make a full copy of the list to the left of the ++ when calculating the result of the operation, while this is not necessary for the list to the right. The latter list’s allocated memory can be reused by the resulting list, but the original list to the left needs to be copied to memory. Therefore, the larger the list to the left, the more costly the overall append operation.

You can find more information here.

Attempt at an implementation without using `++`

The official documentation suggests that we refactor the code utilizing ++ to include [item|rest]. Why don't we give it a try? One thing that is not explicitly stated here is that we probably need to use Enum.reduce/3 to iterate over the elements of the list to the left of the ++ operator in order to achieve the same results.

def number_dups_list3(numbers) do
  Enum.reduce(numbers, [], fn n, acc ->
    Enum.reduce(List.duplicate(n, n), acc, fn n2, acc2 ->
      [ n2 | acc2 ]
    end)
  end)
end

Time to run this new implementation and see what happens...

iex> fn -> number_dups_list3(List.duplicate(1,100_000)) end |> :timer.tc
{16279, .... }

iex> fn -> number_dups_list3([10_000_000, 20_000_000]) end |> :timer.tc
{2515518, .... }

We found that in the first case, where the values in the list are small, the code works well. However, it seems that there isn’t much of a difference when it comes to our second implementation relying on ++ (the one that grows the resulting list by prepending short lists).

Additionally, in the case of the input list, [10_000_000, 20_000_000], things are performing quite slowly. Let's take a look at the complete report after running the 3 functions with the assistance of Benchee:

##### With input long list with low values #####
Name                                       ips        average  deviation         median         99th %
Using ++ and reverse                    119.85        8.34 ms    ±24.50%        8.22 ms       15.47 ms
Without ++ ([h | t] and reverse)         70.75       14.13 ms    ±21.33%       13.07 ms       26.79 ms
Using ++                                0.0300    33361.65 ms     ±0.00%    33361.65 ms    33361.65 ms

Comparison:
Using ++ and reverse                    119.85
Without ++ ([h | t] and reverse)         70.75 - 1.69x slower
Using ++                                0.0300 - 3998.44x slower

##### With input short list with high values #####
Name                                       ips        average  deviation         median         99th %
Using ++                                  0.92         1.08 s    ±45.19%         1.01 s         2.51 s
Using ++ and reverse                      0.41         2.45 s    ±27.75%         2.29 s         3.72 s
Without ++ ([h | t] and reverse)          0.35         2.85 s    ±39.46%         2.39 s         4.82 s

Comparison:
Using ++                                  0.92
Using ++ and reverse                      0.41 - 2.26x slower
Without ++ ([h | t] and reverse)          0.35 - 2.64x slower

We were unable to beat our former implementations relying on ++ for any of our test cases. Hence, the option suggested by the documentation was not the best fit for us in either of our two situations.

Another implementation idea?

We can brainstorm further to come up with other ideas to determine a solution to our problem. If we stop thinking as much about performance, we can try to find additional ways to reimplement the code, possibly ending up with a more elegant solution.

For example, if we combine Enum.map/2 and Enum.flatten/1, we end up with the following implementation:

def  number_dups_list4(numbers) do
  Enum.map(numbers, &(List.duplicate(&1, &1)))
  |>  List.flatten
end

This is actually quite simple to understand. We're now creating a list that includes both of the intermediate lists with the duplicated items. In the end, everything is flattened in the final list.

How might it perform? If we try to guess what is going to happen, I would say that there will be a struggle between the need to keep the intermediate lists in memory across all iterations and the fact that there is no need to reverse the resulting list (as with the other solutions).

The final answer is revealed after running our benchmarks:

##### With input long list with low values #####
Name                                       ips        average  deviation         median         99th %
Using ++ and reverse                    120.03        8.33 ms    ±25.82%        8.11 ms       16.30 ms
map / flatten                           113.00        8.85 ms    ±21.77%        8.17 ms       17.72 ms
Without ++ ([h | t] and reverse)         76.24       13.12 ms     ±9.12%       12.93 ms       17.96 ms
Using ++                                0.0324    30821.21 ms     ±0.00%    30821.21 ms    30821.21 ms

Comparison:
Using ++ and reverse                    120.03
map / flatten                           113.00 - 1.06x slower
Without ++ ([h | t] and reverse)         76.24 - 1.57x slower
Using ++                                0.0324 - 3699.55x slower


##### With input short list with high values #####
Name                                       ips        average  deviation         median         99th %
Using ++                                  0.93         1.08 s    ±39.43%         1.03 s         2.38 s
map / flatten                             0.61         1.63 s    ±46.49%         1.53 s         3.90 s
Using ++ and reverse                      0.53         1.89 s    ±34.73%         1.84 s         3.61 s
Without ++ ([h | t] and reverse)          0.37         2.70 s    ±33.23%         2.37 s         4.16 s

Comparison:
Using ++                                  0.93
map / flatten                             0.61 - 1.51x slower
Using ++ and reverse                      0.53 - 1.75x slower
Without ++ ([h | t] and reverse)          0.37 - 2.50x slower

We can see that this solution ranks quite well, taking second place in both situations. This implementation wouldn't be a bad choice if we expected to have both types of inputs (especially if we discard our first attempt given the bad results where the accumulated list is always to the left of the ++ operator).

However, we would need to test additional cases to make a more informed decision. After all, we are testing two very different and, perhaps, extreme use cases. For instance, we can test our implementations with Enum.to_list(1..1000), or whatever could be considered an average case. I tested with this new input and I found it runs faster with the second implementation, the one with ++ and reverse (while the map and flatten version performs like the [item | rest] version).

Conclusion

Here are a few thoughts about working with the ++ operator in Elixir, especially in situations where the expected load is high and the performance impact could be an issue.

It is important to know the risks of using ++ to append to lists, especially when the list to the left of the operator could have a large number of elements.
If you're in the situation described above, you may consider using Enum.reduce and [item | rest], though you should verify if these adjustments actually improve the situation. Although I think the advice in the documentation is a good rule of thumb, you shouldn't blindly follow it. In fact, I would try to use ++ but invert the order of the appended lists (and reverse the result later). Moreover, you can also test different implementations using different language tools, as we did when we tried Enum.map and Enum.flatten, in order to see how they compare with the previous attempts.
Reflect on the shape and size of your expected inputs. Also think about the number of created lists and their length in the intermediate steps of the calculation. The best performing implementation will depend on usage patterns.
Keep in mind that it’s usually hard to predict the resulting performance...unless you are an expert at Elixir/Erlang internals 😀. For the rest of us mere mortals, it is crucial to run benchmarks in each case and make decisions based on that. Every situation is unique!

Lastly, you may want to take a look at the Erlang Efficiency Guide chapter dedicated to lists. It includes some good insights on list operators, including ++, flatten, etc. Elixir is mostly relying on those Erlang functions.

And, because I know you are curious, here are the complete code and benchmarking results mentioned in this post.

How to Quickly Deploy a VueJS App to Heroku

Jorge Bejar — Wed, 10 Jan 2018 18:06:52 +0000

Originally published in WyeWorks blog.

Recently, I have been investing some time learning VueJS and I found that it is a very interesting framework to play around with. In fact, I have been working on a new project prototype for the last few days and wanted to show it to some people, so I wanted to publish it somewhere in the Internet.

I decided to deploy the project on Heroku so I started to research what is the best way to do it. To my surprise, I did not find much about it apart from a few posts like Quick-n-clean way to deploy Vue + Webpack apps on Heroku and Easily deploy a Vue + Webpack App to Heroku in 5 Steps. Nevertheless, I ended up with a different setup and this is the topic of this post.

Assuming that a Heroku account is already created and the VueJS project already exists, the approach explained in the mentioned articles that I found in my research could be summarized in the following steps:

Write a minimal NodeJS web server using Express
Build the assets locally
Add the dist folder to the Git repository, so it is included when pushing to Heroku

What I did not like of these solutions was the need to build the site locally and check-in the changes within the dist folder. I wanted to have this step handled by Heroku when pushing a new version of my application.

Our solution

Let's assume we have a VueJS project generated using vue-cli with the webpack template. Just to be clear, the project was created using the following command:

vue init webpack <YOUR-PROJECT-NAME-HERE>

Of course, we also need a Heroku account and a new application created there. Heroku will use the NodeJS buildpack because our project contains a package.json in the root folder.

Step 1: Add a minimal NodeJS server

This is a step borrowed from the mentioned blog posts. We have to add a server.js file in the project's root folder containing the following code:

const express = require('express');
const path = require('path');
const serveStatic = require('serve-static');

let app = express();
app.use(serveStatic(__dirname + "/dist"));

const port = process.env.PORT || 5000;
app.listen(port, () => {
  console.log('Listening on port ' + port)
});

Since this code uses Express, we need to add this dependency to our project:

npm install express --save

You can test this server locally by running the following commands:

npm run build
node server.js

Step 2: Setup package.json scripts

We need to tweak the scripts section in the package.json. If the package file provided by the Vue Webpack template was not modified, it should include two important tasks, start and build:

"scripts": {
  ...
  "start": "npm run dev",
  ...
  "build": "node build/build.js"
},

By default, the start script will be executed by Heroku to start the server. For this reason, we will change the command associated to start to run our custom server script:

"scripts": {
  ...
  "start": "node server.js",
  ...
},

Please note you can not use npm run start anymore to run the development server in your computer. I decided to use npm run dev directly but you could add a new entry in the scripts section with an alias for that.

We still have to add something to make sure that the dist folder is built in our Heroku instance every time the code is deployed, otherwise the server script is not going to work properly. We will use a special script called heroku-postbuild which is documented here. The idea is to build the site using this special hook, so let's add it to our package.json:

"scripts": {
  ...
  "heroku-postbuild": "npm install --only=dev --no-shrinkwrap && npm run build",
},

Let's explain the command a bit. First of all, we need to install the dependencies that are used to build the assets. In a VueJS project created with the Webpack template, all the needed dependencies are in devDependencies, so we have to add the --only=dev option.

The —no-shrinkwrap option is used to avoid possible conflicts with the packages installed by Heroku during the installation process (where the production dependencies were installed). However, it could be an unnecessary option in most cases.

And of course, we are running npm run build to actually build the site before the server is started.

Step 3: Try it and enjoy!

We are now ready to deploy to Heroku. Assuming we already have a Git repository, we need to add the Heroku remote repo:

heroku git:remote -a <YOUR-HEROKU-APP-NAME-HERE>

And the command to deploy our application is:

git push heroku master

It will push the code, trigger the build steps and start the NodeJS script that will serve our site made with VueJS

Discussion

There might be some discussion around the decision of having a build step in Heroku instead of checking in the dist folder. Building the site locally would lead to a less complicated Heroku setup because we can just assume that the dist folder is always present. However, having the dist folder in our Git repository does not seem a good practice because it will make harder to read commit changes and deal with merge conflicts. Also, it will require some effort and discipline from every developer in the team to keep the right built version of the assets in the repository. For all these reasons, we prefer to build the site as an automated step into the deployment process.

Speaking about the heroku-postbuild hook, some people are actually using post-install which seems to also work on Heroku. The purpose of this npm hook is to be invoked when a package is installed and, in my opinion, it should be used in the context of the library project, not in an application project. I would rather use the most specific hook provided by Heroku.

Regarding the need to run npm install in the heroku-postbuild hook to install our devDependencies, we could discuss a few available alternatives to solve the problem:

Do not use `devDependencies`

The simplest approach would be to move everything to dependencies and do not use devDependencies at all.
In fact, I was comparing the process to deploy a React project created with create-react-app to Heroku and realized that all the scripts and dependencies needed to build the site are actually in the dependencies section. This is what you find in the package.json file in such cases (the react-scripts package contains all the dependencies used to build the site):

  "dependencies": {
    "react": "^16.2.0",
    "react-dom": "^16.2.0",
    "react-scripts": "1.0.17"
  },
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    ...
  }

Note that there is no devDependencies there. Therefore, it is safe to just run npm run build in the heroku-postbuild hook, since all the necessary packages to build the site were already installed at that point by the default deployment process in Heroku.

In any case, I think it is a good practice to keep dependencies well organized in both categories, dependencies and depDependencies. As a consequence, we opted for the inclusion of the additional npm install in the heroku-postbuild hook step instead of changing the default configuration provided by vue-cli.

Set NPM_CONFIG_PRODUCTION to false

Setting the environment variable NPM_CONFIG_PRODUCTION to false causes that packages from devDependencies will also be installed by default in the deployment process in Heroku. The default value is true because the most common case would be to install only the items from the dependencies list.

It would be a valid solution to tweak this value and have the heroku-postbuild script just running npm run build. Even so, note this change also affects the value of NODE_ENV as explained here . There is a chance that it could cause some side effect in the build process but this is unlikely to happen using the default Webpack configuration for VueJS projects, as far as I can tell.

Hope you find this post useful and have your VueJS project finally deployed to Heroku! If you have any problem following the steps please leave a comment so we can find a solution together and improve this article.

Unopinionated comparison of Glimmer and React

Jorge Bejar — Wed, 20 Dec 2017 18:50:45 +0000

Originally published in Wyeworks blog.

In this article, we will discuss how different (or similar) it is to use Glimmer as compared with React. Glimmer is a new library in the Ember ecosystem, which was released in March. Since the creation of this project, the Ember team has been experimenting with a new components API and the result of this work is expected to gradually be integrated into Ember itself in the future.

To some extent, Glimmer is comparable with React because the scope of both libraries is very similar: UI components for the web. However, let's take a look at some examples to highlight some of the similarities, as well as the differences.

Our first comparison

Let's start by comparing some basic stuff. Let's say we have a component containing a button. When said button is clicked, a random animal name is displayed.

This is how we could implement it with React:

import React, { Fragment } from "react";

const animals = ["Cat", "Dog", "Rabbit"];

class RandomAnimal extends React.Component {
  constructor() {
    super();
    this.state = { animal: null };
    this.setRandomAnimal = this.setRandomAnimal.bind(this);
  }

  setRandomAnimal() {
    const animal = animals[Math.floor(Math.random() * 3)];
    this.setState({ animal });
  }

  render() {
    let renderAnimal;

    if (this.state.animal) {
      renderAnimal = <h1>Hello, {this.state.animal}</h1>
    }

    return (
      <Fragment>
        <button onClick={this.setRandomAnimal}>Set Random Animal</button>
        { renderAnimal }
      </Fragment>
    );
  }
}

Try this code here

What would be the translation for Glimmer? Well, the following does the same (please note, we need two files):

template.hbs

<button onclick={{action setRandomAnimal}}>Set Random Animal</button>
{{#if randomAnimal}}
  Hello, {{randomAnimal}}!
{{/if}}

component.ts

import Component, { tracked } from '@glimmer/component';

const animals = ["Cat", "Dog", "Rabbit"];

export default class extends Component {
  @tracked randomAnimal = null;

  setRandomAnimal() {
    const animal = animals[Math.floor(Math.random() * 3)];
    this.randomAnimal = animal;
  }
}

Try this code here

Obviously, the HTML code is handled differently in each case: React relies on JSX to embed the HTML into the Javascript code whereas Glimmer requires a specific file for the template into which it is written using Handlebars.

You may have noticed that our component does not have a unique root element. Glimmer supports that out-of-the-box. In React, one of the possible ways to create a component with more than one root element is to use Fragment tags, as done in the example. Support for such a situation has been improving in the latest React releases, as explained here.

We have a button that is attached to an action which is invoked when the button is clicked. In React, we achieve this by passing a component function to the onClick property of the button. However, we need to make sure that the function is correctly bound to this (we are binding the function to the constructor). On the other side, Glimmer comes with an action helper and we use it when the component function setRandomAnimal is passed to the onclick property of the button.

The implementation of the setRandomAnimal function is quite similar, but differs a bit depending on how the component’s internal state is updated in each case. React comes with the setState function, and it must be used to alter any internal state if we want to cause a re-rendering of our component.

In Glimmer, we use tracked properties which are updated by assigning a new value using regular Javascript syntax (in our example, the relevant code is this.randomAnimal = animal;). However, this relies on property declaration. We must use the @tracked annotation so that Glimmer keeps track of these values and triggers a component rerender when modifications in those properties take place.

We can see that Glimmer follows a more declarative approach while React’s model is more imperative (due to the need to invoke setState) to manage when components are refreshed in the UI.

Iterating over arrays

Let's rewrite our React component to show a list of animals:

import React, { Fragment } from "react";

const animals = ["Cat", "Dog", "Rabbit"];

class RandomAnimal extends React.Component {
  constructor() {
    super();
    this.state = { animals: [] };
    this.setRandomAnimal = this.setRandomAnimal.bind(this);
  }

  setRandomAnimal() {
    const animal = animals[Math.floor(Math.random() * 3)];

    this.setState((prevState) => (
      { animals: prevState.animals.concat(animal) }
    ));
  }

  render() {
    const renderedAnimals = this.state.animals.map((animal, index) =>
      <li key={index}>{animal}</li>
    );

    return (
      <Fragment>
        <button onClick={this.setRandomAnimal}>Set Random Animal</button>
        <ul>{ renderedAnimals }</ul>
      </Fragment>
    );
  }
}

Try this code here

Here we changed the code in order to show a list of animals. The click event of the button will invoke a function which adds new animals to the list. The React documentation states that this.state must not be mutated directly because React is designed around the idea that the component's state must be updated only through setState. To avoid that, we are using concat to generate a new instance of our list of animals, including the added item.

Another piece of advice regarding setState that is worth knowing: we need to rely on the value of prevState to make sure we are not messing up the array of animals if React decides to batch multiple calls to setState as explained here.

All the above are fundamental React's gotchas to be aware of. In effect, our component could have been implemented by mutating the list or relying on this.state.animals in the setState callback and it would work...most of the time. Unfortunately, it would introduce subtle issues, which would be hard to track if race conditions actually exist.

Let's now explore how it might look in Glimmer:

template.hbs

<button onclick={{action setRandomAnimal}}>Set Random Animal</button>
<ul>
  {{#each randomAnimals key="@index" as |animal| }}
    <li>{{animal}}</li>
  {{/each}}
</ul>

component.ts

import Component, { tracked } from '@glimmer/component';

const animals = ["Cat", "Dog", "Rabbit"];

export default class extends Component {
  @tracked randomAnimals = [];

  setRandomAnimal() {
    const animal = animals[Math.floor(Math.random() * 3)];

    this.randomAnimals = this.randomAnimals.concat(animal);
  }
}

Try this code here

Here we have something that is actually very similar between Glimmer and React: we need to mutate the array in order to update the UI. Glimmer does not refresh the component if we mutate the array value directly, as it does not detect a value change in the tracked property. When tracked properties are arrays or objects, a new instance with the modified values must be provided. This is explained in the section, "The Immutable Pattern", here. However, if we ignore this recommendation and mutate the array of animals anyway, the component is not updated at all when we click the button. Since changes in the list of animals are not reflected on the web page, we know that something is not working as expected and, as a consequence, it is unlikely that a race condition problem would appear as in the React case.

Regarding how the user interface is built, we observe how different the mechanisms to produce HTML are in each case. In Glimmer, we use Handlebars, which comes with control structures such as if and each blocks to build dynamic content. Moreover, you have access to the component properties (all of them, tracked or not), and you never refer to the component itself via this. In fact, this is basically a declarative approach... yet again.

In React, you will end up forming the HTML output in a more programmatic (and imperative) way, probably splitting parts of the output calculation across several variables or auxiliary functions and finally concatenating everything in the value returned by the render function. Since our case was not so complicated, we just needed to use the local variable renderAnimals with the help of the map function to build the list of li elements.

Composing components

With the excuse of exploring the interaction between components, let's rewrite our React example to contain only one specific component for the button:

import React, { Fragment } from "react";

const animals = ["Cat", "Dog", "Rabbit"];

function AnimalButton(props) {

  function setRandomAnimal() {
    const animal = animals[Math.floor(Math.random() * 3)];
    props.onAnimalPicked(animal);
  }

  return (
    <button onClick={setRandomAnimal}>{props.caption}</button>
  );
}

class RandomAnimal extends React.Component {
  constructor() {
    super();
    this.state = { animals: [] };
    this.addAnimalToList = this.addAnimalToList.bind(this);
  }

  addAnimalToList(animal) {
    this.setState((prevState) => (
      { animals: prevState.animals.concat(animal) }
    ));
  }

  render() {
    const renderedAnimals = this.state.animals.map((animal, index) =>
      <li key={index}>{animal}</li>
    );

    return (
      <Fragment>
        <AnimalButton caption="Set Random Animal" onAnimalPicked={this.addAnimalToList}/>
        <ul>{ renderedAnimals }</ul>
      </Fragment>
    );
  }
}

Try this code here

Note that we added the stateless component AnimalButton, and we are determining the next animal in the list for this component. Then, we are passing a callback function which is invoked by the new animal. Adding the value to the list is the job of our main component. The caption text and the callback function are both values passed as properties.

Similarly, we can do the analogous refactor in Glimmer:

AnimalButton/template.hbs

<button onclick={{action setRandomAnimal}}>{{@caption}}</button>

AnimalButton/component.ts

import Component, { tracked } from "@glimmer/component";

const animals = ["Cat", "Dog", "Rabbit"];

export default class extends Component {
  setRandomAnimal() {
    const animal = animals[Math.floor(Math.random() * 3)];

    this.args.onAnimalPicked(animal);
  }
};

RandomAnimal/template.hbs

<AnimalButton @caption="Set Random Animal" @onAnimalPicked={{action addAnimalToList}} />

<ul>
  {{#each randomAnimals key="@index" as |animal| }}
    <li>{{animal}}</li>
  {{/each}}
</ul>

RandomAnimal/component.ts

import Component, { tracked } from '@glimmer/component';

export default class extends Component {
  @tracked randomAnimals = [];

  addAnimalToList(animal) {
    this.randomAnimals = this.randomAnimals.concat(animal);
  }
}

Try this code here

Looking at the Glimmer solution, we notice that it is very similar in how values are passed to the AnimalButton component (note that in Glimmer, arguments begin with the character @). In both cases, we are passing a string for the button element to the AnimalButton component, as well as a function which adds the animal to the list.

In our React solution, the callback is a property of props, so when the button is clicked, the addAnimalToList function from the parent component is finally invoked. In Glimmer, the passed function is available in the child component context as a property of this.args, making it very easy to invoke as well. The only difference here is the usage of the action keyword, which automatically binds the function to the parent component context, which in turn is usually the most convenient binding. Manually binding the callbacks to the component instance is usually required in React, as done in the last line of the constructor function in our example.

The most noticeable difference here is related to the very simple components like the AnimalButton. In React, it is possible to implement AnimalButton as a stateless component. This is a means of simplifying the definition of components that do not contain inner states. No similar concept exists in Glimmer, where the simplest option would be components containing only handlebars code. However, some Typescript code must be present to handle the click event and invoke the callback, hence both files are necessary in our case.

Didn't we mention before that Glimmer uses Typescript instead of Javascript by default? 🤠

Another important thing! In Glimmer, the location and names of files is very important. All components should be placed under ui/components in separate folders where the specific files template.hbs and component.ts exist. This convention must be adhered to in order to keep things properly connected. In the React world, one has more freedom to organize things because you are responsible for importing components as they are needed using the ES6 module import syntax.

Further comparison

Looking beyond the code examples above, there are other similarities between both libraries worth highlighting.

For example, both include support for defining inline content for components. Take a look at the alternative implementation below for the render function of the AnimalButton component:

function AnimalButton(props) {

  ...

  return (
    <button onClick={setRandomAnimal}>
      {props.children}
    </button>
  );
}

The props.children will be replaced by any content nested under the tag <AnimalButton>. Here is an example of how it would be invoked by the parent component:

function RandomAnimal() {
  return (
    <AnimalButton onAnimalPicked={this.addAnimalToList}>
      Set Random Animal
    </AnimalButton>
  );
}

In Glimmer, it’s possible to do the same using the yield keyword. Although it is not officially mentioned in the Glimmer documentation, it works as expected. Well, something that you might expect if you have experience with Ember 😌.

AnimalButton/template.hbs

<button onclick={{action setRandomAnimal}}>
  {{yield}}
</button>

RandomAnimal/template.hbs

<AnimalButton @onAnimalPicked={{action addAnimalToList}}>
  Set Random Animal
</AnimalButton>

Both libraries have additional similar features, such as the ability to render outside of the DOM hierarchy of components (see Portals in React and the {{in-element}} helper in Glimmer mentioned here).

Another important aspect is the availability of lifecycle hooks in the components API. In fact, Glimmer doesn’t include a lot of them; only didInsertElement , didUpdate, and willDestroy are present in the public API. React has a few more hooks, but I would expect to see similar capabilities in Glimmer in the near future (for instance, something similar to React’s componentWillReceiveProps would be a nice-to-have).

Conclusions

The examples in the article allowed us to cover similar aspects in both libraries. At the same time, we discussed problems that each library resolves differently. Although the comparison is not a complete review of the capabilities of each library, many relevant aspects have been highlighted and an overall idea of what you can expect from Glimmer and React has been provided.

We should note that Glimmer is a rather young library, and as a consequence, the component API that has been exposed to developers is still changing and being further established. On the other hand, this library is powered by the GlimmerVM, the rendering engine that shipped with later versions of Ember, so it is already in use in many existing applications. The Ember team is currently working on improvements for this engine, and that will impact Glimmer quite soon. For instance, they are trying to optimize the bytecode generated by the GlimmerVM, and are experimenting with incremental rendering and rehydration. Further reading about these topics can be found here: Glimmer Progress Report. Of course, React itself is a very active project and the rendering engine completely changed with the arrival of Fiber this year.

An interesting aspect of Glimmer is that it relies on ES6 classes for components, making it more like React than Ember in this way, and this explains some of the similarities. At the same time, both approaches differ in terms of component state management, templating language, and underlying mechanisms to manipulate the DOM. Naturally, React offers a broader set of features since the API is larger than Glimmer’s, the latter which only covers the most basic of needs, at least for now.

As a web developer, I find it interesting to understand how the different libraries compare with each other. We have seen how UI libraries in the frontend landscape had been consolidating the architecture of components, adopting ES6/ES7 idioms and goodies, and looking for a balance between an accessible API and solid rendering performance.

DEV Community: Jorge Bejar

Slimming down fat channels in Phoenix

The problem

Splitting the logic in different modules

Looking for the missing functions

A new DSL is born

The result

Acknowledgements

To use or not to use the ++ operator in Elixir

The problem

Our first naive implementation

Let's follow the advice from the documentation

What’s the problem with the lists on the left?

Attempt at an implementation without using ++

Another implementation idea?

Conclusion

How to Quickly Deploy a VueJS App to Heroku

Our solution

Step 1: Add a minimal NodeJS server

Step 2: Setup package.json scripts

Step 3: Try it and enjoy!

Discussion

Do not use devDependencies

Set NPM_CONFIG_PRODUCTION to false

Unopinionated comparison of Glimmer and React

Our first comparison

Iterating over arrays

Composing components

Further comparison

Conclusions

Attempt at an implementation without using `++`

Do not use `devDependencies`