Using Elixir's "with" statement.

Elixir has many features to make code readable, maintainable, and correct. One of them is the with statement. Unfortunately, the with statement can be confusing to work with, because there are some hidden assumptions on how it's used.

In this post, I will introduce you to why, where, and when you would want to use a with statement, and how you can deal with some of the challenges and underlying assumptions. Additionally, you’ll read about a readability and differentiation trick using annotation tuples.

1. The worst "case".

The use-case for with statements is apparent when we look at the worst case statement implementation. Nesting case statements quickly leads to a pyramid of unreadable code (literally! tilt your head 90°):

case is_email_address?(email) do
  true ->
    case String.length(code) === 6 do
      true ->
        case EmailConfirmations.get_email_confirmation(email) do
          %EmailConfirmation{} ->
            case EmailAddresses.get_email_address(email) do
              nil ->
                case Users.create_user(data) do
                  {:ok, user} ->
                    case EmailAddresses.create_email_address(user, email) do
                      {:ok, email_address} ->
                        ...

                      {:error, error} ->
                        :error
                    end

                  {:error, error} ->
                    :error
                end

              _email_address ->
                :error
            end

          nil ->
            :error
        end

      false ->
        :error
    end

  false ->
    :error
end

• What this code does well, is matching all possible returns from every function that is called; therefore the code is correct.

• What this code does badly, is it's unreadable for you, me, and any other developer looking at it.

We need something better to maintain readability, while keeping the case statement's functionality and correctness.

2. Simple "with" statements.

with statements work like case statements, but with a focus on successful function call results. The following statements are equivalent:

case Users.create_user(attrs) do
  {:ok, user} -> ...
  {:error, changeset} -> ...
end

with {:ok, user} <- Users.create_user(attrs) do
  ...
else
  {:error, changeset} -> ...
end

Let's see some examples.

Implicit handling of non-matching results.

with statements do not need to handle non-matching clauses. Instead, the with statement will return the non-matching clause directly to the parent scope.

The Phoenix framework comes with context generators that will output controller functions with with statements like this:

def create(conn, params) do
  with {:ok, user} <- Users.create_user(params) do
    conn      
    |> put_status(:accepted)      
    |> render("show.json", user: user)
  end
end

Where the function Users.create_user/1 can return either {:ok, user} or {:error, changeset}.

As you can see, the clause {:error, changeset} is not caught in the with statement. Let's see what this with statement does:

• When the expected clause matches, because it is {:ok, user}, we continue with the function inside the with statement and user is available within the scope.

• When the expected clause does not match, for example we receive {:error, changeset}, then the result is immediately returned to the parent scope (i.e. create/2).

Therefore, the function create/2 returns either the result of the successful clause (an updated conn) or {:error, changeset}.

Explicit handling of non-matching results.

We can explicitly handle the non-matching clauses in a with statement by using the else clause. The else clause requires us to pattern match on all non-matching results.

def create(conn, params) do
  with {:ok, user} <- Users.create_user(params) do
    conn      
    |> put_status(:accepted)      
    |> render("show.json", user: user)
  else
    {:error, changeset} ->
      {:error, changeset}
  end
end

This does the same as the previous example. It will return either the result of the successful clause (an updated conn) or {:error, changeset}.

Because we are explicitly handling non-matching clauses, we must add pattern matches for all non-matching clauses in the else block. If we miss a returned clause, we will receive a ** (WithClauseError) no with clause matching error.

This is the reason why Dialyzer will complain "The pattern can never match the type", because if a returned clause is not explicitly handled, it may crash the process at runtime.

3. Chained functions in "with" statements.

Chaining function calls in a with statement is fairly straightforward. We can rewrite the "case pyramid" from the introduction by chaining the consecutive expressions in a with statement:

with true <- is_email_address?(email),
     true <- String.length(code) === 6,
     %EmailConfirmation{} <- EmailConfirmations.get_email_confirmation(email),
     nil <- EmailAddresses.get_email_address(email),
     {:ok, user} <- Users.create_user(data),
     {:ok, email_address} <- EmailAddresses.create_email_address(user, email) do
  ...
end

Much cleaner and shorter than the original. And with syntax-highlighting in an IDE, this code will be much more readable.

Challenges.

In the previous examples, we always return :error when a function call does not return the desired result. But this is rarely what we want in practice.

• We may want to differentiate between similar clauses in the else block.

• We need to handle all non-matching clauses in the else block, of every chained function call, or we will introduce app-crashing bugs.

• Dialyzer will definitely complain if any of the functions does not return an expected clause (i.e. :error in this example).

4. Advanced "with" statements.

Let's add custom error-handling, differentiation of function calls and results, and readability improvements.

Custom error-handling.

Let's introduce some custom error-handling to the rewritten "case pyramid" in a with statement:

with true <- is_email_address?(email),
     true <- String.length(code) === 6,
     %EmailConfirmation{} <- EmailConfirmations.get_email_confirmation(email),
     nil <- EmailAddresses.get_email_address(email),
     {:ok, user} <- Users.create_user(data),
     {:ok, email_address} <- EmailAddresses.create_email_address(user, email) do
  ...
else
  false -> 
    {:error, :bad_request}

  nil -> 
    {:error, :not_found}

  %EmailAddress{} -> 
    {:error, :conflict}

  {:error, changeset} -> 
    {:error, changeset}
end

Great! Now we can return different error messages depending on which non-matching clause is returned.

But what if the with statement has multiple functions that return the same non-matching results, and we want to handle each return differently?

For example, when two functions can return false, and one version must return {:error, :bad_request}, and the other must return {:error, :conflict}, like so:

with true <- is_email_address?(email),
     true <- EmailAddresses.is_available(email) do
  ...
else
  false ->
    # Return either '{:error, :bad_request} or '{:error, :conflict}'
end

Let's see how we can differentiate the returned false clause.

Differentiating non-matching clauses.

We can use a simple mathematical trick to differentiate the returned results. If x * y = z, then x * y + c = z + c must be true as well. Therefore we can add some annotation tuples to the function calls and clauses, and write the previous example as:

with {:is_email, true} <- {:is_email, is_email_address?(email)},
     {:is_available, true} <- {:is_available, EmailAddresses.is_available(email)} do
  ...
else
  {:is_email, false} ->
    {:error, :bad_request}

  {:is_available, false} ->
    {:error, :conflict}
end

By using tuples on both sides of the function calls, we can differentiate the returning values.

Improving readability for large "with" statements.

We can improve readability of large with statements by using the trick from the previous section. That is, by annotating function calls and pattern matches in the with and else block with tuples.

Let's rewrite the "case pyramid" with statement with annotation tuples:

with {:is_email, true} <- {:is_email, is_email_address?(email)},
     {:is_code, true} <- {:is_code, String.length(code) === 6},
     {:confirmation_fetch, %EmailConfirmation{}} <- {:confirmation_fetch, EmailConfirmations.get_email_confirmation(email)},
     {:email_fetch, nil} <- {:email_fetch, EmailAddresses.get_email_address(email)},
     {:user_create, {:ok, user}} <- {:user_create, Users.create_user(data)},
     {:email_create, {:ok, email_address}} <- {:email_create, EmailAddresses.create_email_address(user, email)} do
  ...
else
  {_, false} -> 
    {:error, :bad_request}

  {:confirmation_fetch, nil} -> 
    {:error, :not_found}

  {:email_fetch, %EmailAddress{}} -> 
    {:error, :conflict}

  {_, {:error, changeset}} -> 
    {:error, changeset}
end

In this with statement we chain six functions, and add the following tuples to make the code more readable: {:is_email, _}, {:is_code, _}, {:confirmation_fetch, _}, {:email_fetch, _}, {:user_create, _}, and {:email_create, _}.

Again, in an IDE with syntax-highlighting this will look much better.

5. Common issues and how to solve them.

Be mindful of all the possible returned values from the functions you call in a with statement. Properly handling matching and non-matching clauses will solve most of your issues.

1. Use formatting to make with blocks more readable.

2. Use the else block in a with statement to handle non-matching results manually.

3. Always pattern match all possible returned non-matching values in else blocks to prevent app-crashing bugs.

4. Use pattern matches on structs. For example with %User{is_banned: false} <- Users.get_user(id) do.

5. To differentiate function calls and results, annotate the function call and their results with tuples (e.g. {:is_email, true} and {:is_available, true}).

6. To make large with statements more readable, consider annotating function calls and their results with tuples (e.g. {:user_fetch, user} and {:user_update, updated_user}).

7. Dialyzer may show the warning "The pattern can never match the type". This means you've missed a result from one (or multiple) of the called functions in the else block. Add the missing return clause, and the warning will resolve.

---

I hope you learned something today. I'm a big fan of Elixir's with clause, and of all the other readability, maintainability, and code correctness features that Elixir offers. The next post will be about Dialyzer, and why you should (always) use it for development. Follow me to get a notification in January!

Comments

[Simon McConnell]

Why jam the first two checks into the with statement and create noise?
You don't have to use with for everything just because you like it. I think point 6 is debatable.

You could just say "if not email address or length isn't six return bad_request, else ..."

[Martin Nijboer]

Fair point. Adding an if else statement increases scope nesting and could add noise too. If that improves the readability / perceived logic flow of your code, then go for it. :)

[Anwesh]

Check this out - good way to handle long lived transactions - github.com/Nebo15/sage#using-sagas

[Martin Nijboer]

Interesting pattern! I'll take a look. :)

[Michael Oshosanya]

Great article. Writing Elixir seems to make my brain freeze given that I have always worked with OOP languages. This is one of the things I struggle with.