DEV Community: Leandro Cesquini Pereira

Leveraging Exceptions to handle errors in Elixir

Leandro Cesquini Pereira — Sat, 01 Aug 2020 04:00:00 +0000

Returning a tagged tuple {:ok, result} | {:error, reason} is the de facto practice to handle errors in Elixir, but that may not be enough for all situations and this article will explore how to leverage Exceptions to enrich errors and the benefits of doing so.

Looks like that subject is a recurring source of discussion, as Martin Gausby asked the community how to deal with expected and unexpected errors, and turns out Michał Muskała has already introduced a clever technique to handle errors that was used by Andrea Leopardi on libraries Mint and Redix. He tweeted defending that exceptions are a great return value, so let’s dig in to find out how that works.

TLDR

Encapsulate all possible errors of a function into a custom Exception;
Leverage Exceptions message mechanism to avoid coupling;
Let the caller decide if the error is expected or not;

Code example:

Error exception lib/my_app/error.ex

defmodule MyApp.Error do
  @type t() :: % __MODULE__ {
          module: module(),
          reason: atom(),
          changeset: Ecto.Changeset.t() | nil
        }

  defexception [:module, :reason, :changeset]

  @spec wrap(module(), atom()) :: t()
  def wrap(module, reason), do: % __MODULE__ {module: module, reason: reason}

  @spec wrap(module(), atom(), Ecto.Changeset.t()) :: t()
  def wrap(module, reason, changeset) do
    % __MODULE__ {module: module, reason: reason, changeset: changeset}
  end

  @doc """
  Return the message for the given error.

  ### Examples

       iex> {:error, %MyApp.Error{} = error} = do_something()
       iex> Exception.message(error)
       "Unable to perform this action."

  """
  @spec message(t()) :: String.t()
  def message(% __MODULE__ {reason: reason, module: module}) do
    module.format_error(reason)
  end
end

Context lib/my_app/accounts.ex

defmodule MyApp.Accounts do
  @spec register(map()) :: {:ok, User.t()} | {:error, MyApp.Error.t()}
  def register(attrs) do
    # simulate a function that may return more than one type of error
    case has_permission?(attrs) do
      true ->
        # simulate that something wrong happened on register,
        # and note that changeset is just a regular changeset
        MyApp.Error.wrap( __MODULE__ , :register, changeset)

      false ->
        # another situation requires another type of error
        MyApp.Error.wrap( __MODULE__ , :insufficient_permisions)
    end 

    # and other errors could happen...
  end

  # translate the error case into a friendly message
  def format_error(:register), do: "Unable to register account."
  def format_error(:insufficient_permissions), do: "Unable to perform action due to insufficient permissions."
end

Caller LiveView, Controller, etc

case MyApp.Accounts.register(attrs) do
  {:ok, user} ->
    continue_happy_path()

  {:error, error} ->
    socket =
      socket
      |> put_flash(:error, Exception.message(error))
      |> assign(:changeset, error.changeset)

    {:noreply, socket}
end

You may be asking, why not just return {:error, :reason} or even {:error, "message"}? First of all, try to avoid returning a string because that will complicate the pattern matching and a simple change will break your system, on the other hand returning an atom is totally fine when your function doesn’t need to deal with different errors and messages. But usually context or complex functions has more outcomes than just a single possible error, and besides that they’re usually consumed by another layer that needs to transform that error into useful feedback for the user.

Some practical scenarios and benefits

In LiveViews or Controllers, you usually need to display a flash message to let the user know what’s happening. If you return {:error, atom()} that means you need to pattern match all possible atoms to create the proper message. With exceptions, all you need to do is to implement format_error in a single place and call Exception.message(error).
If that same function changes current atom or adds another return value, you have to go through all places where that function is called to update the pattern match. That’s error-prone as you may miss something and the compiler won’t help here
Shared and reused messages. Think about generic errors such as authorization, system errors, and others. Those errors produce the same message everywhere, which requires duplicating the pattern match and the message definition in different places. That can be solved by relying on the expcetion message mechanism.
Cohesion. Implementing a format_message/1 close to where the error happens will improve maintainability.
Transparency. By having the module and a reason in the error struct, you’ll have the origin of that error, from where it’s coming from. Stack traces won’t help when the error is expected and it’s not raised.

But what about that changeset in the middle of the error?

Regular apps, especially web apps, depends a lot on Changesets to return feedback to users but also has to deal with complementary errors on more complex scenarios where returning just an invalid changeset isn’t enough. Suppose a function that deals with form submission but also has to call an external service, check permissions, or deal with that crazy legacy rule. Many different errors or situations may happen: the changeset may be invalid, an external service may be offline, or maybe an error happened but the changeset is valid and needs to be updated to reflect changes on the template. That would require more complex return values like adding more values on the tagged tuple, a struct to store all values, or something. A better approach is to return either {:error, %MyApp.Error{reason: :invalid_input, changeset: changeset}}, {:error, %MyApp.Error{reason: :billing_service_offline}}, or whatever is needed. All you need will be encapsulated on the exception struct. With that return, you can either display inline errors on form if the changeset is valid, call Exception.message(error) to give proper feedback for the user or update the changeset while giving feedback for the user about another error. It’s very flexible and simple.

Error is part of your application

You’re not limited to a generic %MyApp.Error{reason: atom()}, in fact you can implement explicit errors like %MyApp.OfflineService{reason: atom(), status: integer()} instead of %MyApp.Error{reason: :billing_service_offline} to enrich errors specific to your app’s domain. Some benefits include leveraging pattern matching for control flow, explicit errors when raising or reading your code, and encapsulate metadata about specific errors, but not limited to those benefits.

Error is expected or not?

That depends on the caller because that usually is tied to the current situation. Suppose a function that calculates something based on a set of data, if that function is called by a Controller or LiveView probably the user is waiting for feedback, but if that function is called by an async process (mostly a background process) there’s no reason to present a message so raising to force the process to restart may be the best approach. In short, let the caller decide:

If the error is expected or needs to display feedback, call Exception.message(error)
If the error is not expected or there’s no way to recover from that, call raise error

Remember that error is in fact an exception, so raising it is simple as calling raise error, but remember to avoid using try/rescue for control flow and reserve that for situations where the function has reached the end of the line and there’s nothing else to do unless raising.

Notes

Thanks to:

Andrea Leopardi for reviewing this article and also for giving the inspiration to write it.
Michał Muskała for introducing this technique.
Eduardo Hernandes for giving the change to pair program and experiment.

Enforcing code quality in Elixir

Leandro Cesquini Pereira — Fri, 07 Jun 2019 03:00:00 +0000

Enforcing your Elixir code to be well formatted, with no warnings and hopefully free of bugs. I’m talking about a mix alias that will check the quality of the code, and that can be used during local development and also on CI/CD pipelines to enforce the team (or yourself) to keep a clean code. But keep in mind that there’s no magic, you’re still responsible for creating a code that’s well organized, performant, without security issues, and pleasant for human reading.

TL;DR

The complete example is at the end of the article.

Tools

Credo and Dialyzer

Credo is responsible for checking if the code is adherent to common good code practices established by the community. And dialyzer is a static analysis tool that identifies software discrepancies, such as definite type errors, code that has become dead or unreachable because of a programming error (paraphrasing the official docs), but we’ll use dialyxir, which is wrapper around dialyzer written in Elixir that simplify the use of dialyzer.

Sobelow

Sobelow is a security-focused static analysis tool for the Phoenix framework. If you’re not using Phoenix, just remove it from the deps, configs, and alias described on this article.

mix format

The mix format task was introduced in Elixir v1.6, and it’s used to format your code automatically. One of the main benefits of using it is to avoid boring and endless discussions like “how we should format the code ?”, “what should be the line length ?”, and so on. Elixir also uses it to enforce a standard format.

Warning as errors

That’s not a tool, actually, it’s an Elixir compile attribute that will treat warnings in the current project as errors and return a non-zero exit code, which means that your project won’t compile if it has any warning.

A note before we continue

Having all those tools and configs turned on may be annoying. A module without doc, a wrong spec or even an unused variable will stop you from compiling and shipping the code. The level of enforcement is up to you, so you should adapt the tools and configs as required for your project (and your stress level). But it’s a good ideia to plan for the long term.

Implementation

Ok, let’s change the files to implement the quality check. Let’s do that piece by piece, or better saying, function by function.

mix.exs

First, let’s install the deps. Add the following in deps list:

{:credo, "~> 1.0", only: [:dev, :test], runtime: false},
{:dialyxir, "~> 1.0.0-rc.6", only: [:dev, :test], runtime: false},
{:sobelow, "~> 0.7", only: [:dev, :test], runtime: false}

And let’s change the project’s config. Add this to the project function:

elixirc_options: [warnings_as_errors: true],
aliases: aliases(),
dialyzer: [
  plt_file: {:no_warn, "priv/plts/dialyzer.plt"},
  ignore_warnings: ".dialyzer_ignore.exs"
]

Which will be like:

defmodule YourProject.MixProject do
  use Mix.Project

  def project do
    [
      # current configs...

      elixirc_options: [warnings_as_errors: true],
      aliases: aliases(),
      dialyzer: [
        plt_file: {:no_warn, "priv/plts/dialyzer.plt"},
        ignore_warnings: ".dialyzer_ignore.exs"
      ]
    ]
  end

  defp deps do
    [
      # current deps...

      # dev, test
      {:credo, "~> 1.0", only: [:dev, :test], runtime: false},
      {:dialyxir, "~> 1.0.0-rc.6", only: [:dev, :test], runtime: false},
      {:sobelow, "~> 0.7", only: [:dev, :test], runtime: false}
    ]
  end
end

I recommend reading dialyxir doc to know more about its config, especially the Continuous Integration if you want to enforce it on your CI/CD pipeline.

Continuing, let’s create an alias function in your mix.exs file, if you don’t have this function already created, and add two aliases:

defp aliases do
  [
    # current aliases...

    quality: ["format", "credo --strict", "sobelow --verbose", "dialyzer", "test"],
    "quality.ci": [
      "test",
      "format --check-formatted",
      "credo --strict",
      "sobelow --exit",
      "dialyzer --halt-exit-status"
    ]
  ]
end

I like using one alias specific for local development (quality) and one for CI/CD (quality.ci) because I have more freedom on the order of the tasks and its arguments. For example, on local development I don’t want to check if the code is formatted, I just format the code directly; and I like to see test results at the end. Adapt as you want.

.dialyzer_ignore.exs

Create the file .dialyzer_ignore.exs in the root dir of the project:

[
  {":0:unknown_function Function ExUnit.Callbacks. __merge__ /3 does not exist."},
  {":0:unknown_function Function ExUnit.CaseTemplate. __proxy__ /2 does not exist."}
]

Remember I said those tools may be annoying ? Specially the dialyzer. Don’t get me wrong, dialyzer is amazing and I believe you should use it, but sometimes you need to ignore a warning or two, and that’s the file where you can do that.

About the content of this file: we’ll run our aliases on the test environment (MIX_ENV=test) and dialyzer reclaim that those functions are errors, but that’s not a big deal and let’s just ignore it.

Credo

You don’t need to change anything in order to make credo work, but the default rules may not be suitable for your project. You can change that using a .credo.exs file or using special comments.

.gitignore

Add to .gitignore:

# Dialyzer
/priv/plts/*.plt
/priv/plts/*.plt.hash

# Sobelow
.sobelow

Those files are environment-dependent.

Executing

Finally, you’re ready to go! 😅

First, let’s create the dir where dialyzer will store plt files:

mkdir -p priv/plts

And then just execute in your terminal:

MIX_ENV=test mix quality

Take some time off and grab a coffee, the first execution will take some time to build all dialyzer artefacts, but those files will be cached, don’t worry.

And change your CI/CD pipeline to execute:

MIX_ENV=test mix quality.ci

Whenever this command finds an issue in your code, the CI/CD pipeline will halt and return a failure.

Complete example files

mix.exs

defmodule YourProject.MixProject do
  use Mix.Project

  def project do
    [
      app: :your_project,
      version: "0.1.0",
      elixir: "~> 1.6",
      elixirc_paths: elixirc_paths(Mix.env()),
      elixirc_options: [warnings_as_errors: true],
      compilers: [:phoenix, :gettext] ++ Mix.compilers(),
      start_permanent: Mix.env() == :prod,
      aliases: aliases(),
      deps: deps(),
      dialyzer: [
        plt_file: {:no_warn, "priv/plts/dialyzer.plt"},
        ignore_warnings: ".dialyzer_ignore.exs"
      ]
    ]
  end

  # Configuration for the OTP application.
  #
  # Type `mix help compile.app` for more information.
  def application do
    [
      mod: {YourProject.Application, []},
      extra_applications: [:logger, :runtime_tools]
    ]
  end

  # Specifies which paths to compile per environment.
  defp elixirc_paths(:test), do: ["lib", "test/support"]
  defp elixirc_paths(_), do: ["lib"]

  # Specifies your project dependencies.
  #
  # Type `mix help deps` for examples and options.
  defp deps do
    [
      {:phoenix, "~> 1.4.3"},
      {:phoenix_pubsub, "~> 1.1"},
      {:phoenix_ecto, "~> 4.0"},
      {:ecto_sql, "~> 3.0"},
      {:postgrex, ">= 0.0.0"},
      {:phoenix_html, "~> 2.11"},
      {:phoenix_live_reload, "~> 1.2", only: :dev},
      {:gettext, "~> 0.11"},

      # dev, test
      {:credo, "~> 1.0", only: [:dev, :test], runtime: false},
      {:dialyxir, "~> 1.0.0-rc.6", only: [:dev, :test], runtime: false},
      {:sobelow, "~> 0.7", only: [:dev, :test], runtime: false}
    ]
  end

  # Aliases are shortcuts or tasks specific to the current project.
  # For example, to create, migrate and run the seeds file at once:
  #
  # $ mix ecto.setup
  #
  # See the documentation for `Mix` for more info on aliases.
  defp aliases do
    [
      quality: ["format", "credo --strict", "sobelow --verbose", "dialyzer", "test"],
      "quality.ci": [
        "test",
        "format --check-formatted",
        "credo --strict",
        "sobelow --exit",
        "dialyzer --halt-exit-status"
      ],
      "ecto.setup": ["ecto.create", "ecto.migrate", "run priv/repo/seeds.exs"],
      "ecto.reset": ["ecto.drop", "ecto.setup"],
      test: ["ecto.create --quiet", "ecto.migrate", "test"]
    ]
  end
end

.dialyzer_ignore.exs

[
  {":0:unknown_function Function ExUnit.Callbacks. __merge__ /3 does not exist."},
  {":0:unknown_function Function ExUnit.CaseTemplate. __proxy__ /2 does not exist."}
  ]

.gitignore

# The directory Mix will write compiled artifacts to.
/_build/

# If you run "mix test --cover", coverage assets end up here.
/cover/

# The directory Mix downloads your dependencies sources to.
/deps/

# Where 3rd-party dependencies like ExDoc output generated docs.
/doc/

# Ignore .fetch files in case you like to edit your project deps locally.
/.fetch

# If the VM crashes, it generates a dump, let's ignore it too.
erl_crash.dump

# Also ignore archive artifacts (built via "mix archive.build").
*.ez

# Ignore package tarball (built via "mix hex.build").
your_project-*.tar

# If NPM crashes, it generates a log, let's ignore it too.
npm-debug.log

# The directory NPM downloads your dependencies sources to.
/assets/node_modules/

# Since we are building assets from assets/,
# we ignore priv/static. You may want to comment
# this depending on your deployment strategy.
/priv/static/

# Files matching config/*.secret.exs pattern contain sensitive
# data and you should not commit them into version control.
#
# Alternatively, you may comment the line below and commit the
# secrets files as long as you replace their contents by environment
# variables.
/config/*.secret.exs

.elixir_ls

# Dialyzer
/priv/plts/*.plt
/priv/plts/*.plt.hash%

# Sobelow
.sobelow

References

How to update objects inside JSONB arrays with PostgreSQL

Leandro Cesquini Pereira — Mon, 15 Apr 2019 03:00:00 +0000

How to update a specific value on a JSONB array

Let’s say you decided to store data in the database as json or jsonb and discovered that you just created new problems for yourself that you didn’t have before. You’re not alone.

JSONB is a powerful tool, but it comes at some cost because you need to adapt the way you query and handle the data.

And it’s not rare to load the entire jsonb object into memory, transform it using your preferred programming language, and then saving it back to the database. But, you just created another problem: performance bottlenecks and resource waste.

In this article let’s see how to update a specific value of an object inside an array with one query.

TL;DR : the final query is at the end of the article, and you can check out a live example at DB Fiddle to copy & paste and play with.

Suppose you’re implementing a customer screen to store dynamic contacts for each customer. Then you come up with the idea of storing the contacts as a JSONB column because they’re dynamic, and thus using a not relational data structure makes sense.

Then you create a customers table with a JSONB contacts column and insert some data into it:

create table customers (name varchar(256), contacts jsonb);

insert into customers (name, contacts) values (
  'Jimi',
  '[
    {"type": "phone", "value": "+1-202-555-0105"},
    {"type": "email", "value": "jimi@gmail.com"}
  ]'
);

insert into customers (name, contacts) values (
  'Janis',
  '[
    {"type": "email", "value": "janis@gmail.com"}
   ]'
);

Pretty easy right? But how can you update a specific contact for a specific customer? How to change Jimi’s email or Janis’ phone? 🤔

Fortunately, PostgreSQL is your friend and provides the jsonb_set function:

jsonb_set(target jsonb, path text[], new_value jsonb[, create_missing boolean])

Given a jsonb column, you can set a new value on the specified path:

Reference: PostgreSQL Json functions

The above selects will return:

[{"type": "phone", "value": "+1–202–555–0105"}, {"type": "email", "value": "jimi.hendrix@gmail.com"}]
  [{"type": "email", "value": "janis.joplin@gmail.com"}]

To change Jimi’s email on the contacts list, you inform the path 1, value which means the second object on the array (starting at 0) and the key value. That’s the path. The same applies to change Janis’ email, but its email object is at index 0.

You may be thinking: I just have to use jsonb_set on an update statement and it’s all done? That’s the idea, but that’s not enough yet.

The problem with non-relational data is that they’re dynamic. Well, that’s one of the reasons for using JSONB but that brings a problem: see that Jimi’s email object is at index 1 and Janis’ email object is at index 0 in the array, and another customer could have a very different array with different indexes. So, how can you discover the index of each contact type? 🤔

The answer is ordering the elements of the array and getting its index:

select index-1 as index
  from customers
      ,jsonb_array_elements(contacts) with ordinality arr(contact, index)
 where contact->>'type' = 'email'
   and name = 'Jimi';

That query returns 1 , which is the index of the email object (type email) inside the contacts array of the customer Jimi.

Now we have all the pieces of the puzzle: we know how to update a jsonb value and how to discover the index of the object to be updated.

The only step left is the update itself. Putting it all together we have:

The most important part of this query is the with block. It’s a powerful resource, but for this example, you can think of it as a “way to store a variable” that is the path of the contact you need to update, which will be dynamic depending on the record.

Let me explain a bit about this part:

('{'||index-1||',value}')::text[] as path

It just builds the path as '{1, value}', but we need to convert to text[] because that’s the type expected on the jsonb_path function.

Wrapping up

JSONB is a great and valuable tool to resolve a lot of problems. But keep in mind that you also need to query and update this kind of data. That brings a cost that you have to consider when deciding which tools you pick to use.

Side note: that solution came out of a pair programming session with Lucas Cegatti.

Implementing Bounded Contexts in Elixir

Leandro Cesquini Pereira — Thu, 25 Oct 2018 03:00:00 +0000

What if we could write loose coupling services without the overhead of common microservices architecture? In 2015 Valim wrote an article entitled Elixir in times of microservices, where he elaborated his idea on how Elixir fits in a microservice architecture. That’s the kind of mind-blowing post that challenges us to experiment different ways to write software, and that post is my practical approach on this matter.

TL;DR

Create an umbrella project;
One app for each bounded context;
An app should never call internal code from another app;
Use a module on each app to act as a public API to exchange data between apps, and also to call functions to mutate data;
An app should never return its internal data structures on public apis, it should return only raw data like maps and lists;

First of all, what’s a Bounded Context ?

If you know nothing about it, I recommend to read this Martin Fowler article before continuing.

According to the great book Domain Modeling Made Functional:

[…] domains and subdomains in the problem space are mapped to what DDD terminology calls bounded contexts — a kind of subsystem in our implementation. Each bounded context is a mini domain model in its own right. We use the phrase bounded context instead of something like subsystem because it helps us stay focused on what’s important when we design a solution: being aware of the context and being aware of the boundaries. Why context? Because each context represents some specialized knowledge in the solution. Within the context, we share a common language and the design is coherent and unified. But, just as in the real world, information taken out of context can be confusing or unusable. Why bounded? In the real world, domains have fuzzy boundaries, but in the world of software we want to reduce coupling between separate subsystems so that they can evolve independently. We can do this using standard software practices, such as having explicit APIs between subsystems and avoiding dependencies such as shared code.

That give us some hints on how we should implement a bounded context:

Should be independent
Should have an explict API to exchange data
Avoid shared code

Umbrella Projects

If you know the concept of umbrella projects, you may be thinking that we can implement each bounded context as applications of an umbrella project. And you’re right, but some rules are required in order to keep clean organization and avoid creating a mess that is hard to evolve.

But what’s an umbrella project and apps ?

An app is just a package of code, it could be a tiny library, a component, a subsystem or a full application. An ERP system can be composed of apps like Customer, Billing, Delivery and so on.

And an umbrella project is a project composed of those apps, which lives in the same project and can be dependent on each other.

Want to read more about it ? Go to Using an Elixir Umbrella post.

Show me the code

Enough talking, let’s see how it works. And let’s use the same example from Martin Fowler article to establish an example that we can use for our system.

Bounded Context example from Martin Fowler article

In this example we have 2 bounded contexts, Sales and Support, with their own rules and knowledge. Remember that each context should be independent, so let’s start by creating a new Elixir umbrella project called “SystemX” with two apps:

➜ mix new system_x --umbrella
➜ cd system_x/apps
➜ mix new sales
➜ mix new support

You’ll end up with the following directory structure:

➜ tree
.
├── README.md
├── apps
│ ├── sales
│ │ ├── README.md
│ │ ├── config
│ │ │ └── config.exs
│ │ ├── lib
│ │ │ └── sales.ex
│ │ ├── mix.exs
│ │ └── test
│ │ ├── sales_test.exs
│ │ └── test_helper.exs
│ └── support
│ ├── README.md
│ ├── config
│ │ └── config.exs
│ ├── lib
│ │ └── support.ex
│ ├── mix.exs
│ └── test
│ ├── support_test.exs
│ └── test_helper.exs
├── config
│ └── config.exs
└── mix.exs

Entities

As you can see in our domain, we have the entity Customer on both Sales and Support context. That’s because each context may have specific rules or data and evolve independently, which give us leverage to modify each one of them without breaking the whole system, because it’s limited to the inner context.

Let’s define each Customer:

# apps/sales/lib/domain/customer.ex
defmodule Sales.Domain.Customer do
  defstruct [:id, :name, :address, :score, :has_support_ticket]
end

# apps/support/lib/domain/customer.ex
defmodule Support.Domain.Customer do
  defstruct [:id, :name, :last_ticket_id, :email]
end

Public API

And what happens if the Sales context needs to get some information from the Customer on the Support context ? That’s when the explict API comes in handy. Let’s say we need to know if a Customer has a support ticket, then we need to ask this information for the Support context. So, let’s create the Support explicit API to return a Customer:

# apps/support/lib/support.ex
defmodule Support do
  def get_customer(customer_id) when is_integet(customer_id) do
    find(customer_id) # get customer from db
    |> Map.from_struct()
  end
end

Returning a map instead of a struct is what give us loose coupling between apps. We will talk more about it in a moment.

Putting into practice

Now, let’s create a service in the Sales context that calculates the current customer’s score. The business rule is: It gives a higher score if the customer has no ticket support (I know it’s not the best example in the world…)

# apps/sales/lib/client/support.ex
defmodule Sales.Client.Support do
  alias Sales.Domain

  def get_customer(%Domain.Customer{id: customer_id} = customer) do
    customer_has_support_ticket =
      customer_id
      |> Support.get_customer()
      |> has_support_ticket()

    %Domain.Customer{customer | has_support_ticket: customer_has_support_ticket}
  end

  defp has_support_ticket(%{last_ticket_id: last_ticket_id}) when is_nil(last_ticket_id), do: false
  defp has_support_ticket(_), do: true
end

# apps/sales/lib/service/customer.ex
defmodule Sales.Service.Customer do
  alias Sales.Domain
  alias Sales.Client

  def current_score(%Domain.Customer{} = customer) do
    customer = Client.Support.get_customer(customer)

    case customer.has_support_ticket do
      false -> 100
      true -> 50
    end
  end
end

Ok, there’s more than just a simple service. Let me explain.

Apps are allowed to comunicate with each other only by a public API. Boxes in gray are for internal use only.

A Customer entity in the Sales context knows nothing about support tickets, as expected because this information is of responsability of the Support context, however we need it to build our service Sales.Service.Customer.

To overcome this lack of information in the Sales context we can ask the Support context for this information, but remember that we can’t share code between apps nor call internal code from another app. That’s the objective of a public API and the reason it can only return raw data like maps and lists instead of returning an internal struct from Support app.

But we don’t want a meaningless map being used internally in Sales app, we actually want that our domain reflect the real life by mapping it to a struct, which is done in Sales.Client.Support — receives a map, use the information it needs and fullfill Sales.Domain.Customer. Note that in Sales.Domain.Customer we don’t keep the attribute last_ticket_id because we don’t care about the actual ticket id in the Sales context, we just need to know if it’s empty or not. Remember that the same entity from the real world may have different attributes according to the context.

Converting the data structure on the boundaries makes the service layer cleaner and easier to write and understand because in layers below the public API we deal with validated data structures prepared for the current context, which means you don’t need to worry with dirty data, which frees us from writing conditional code.

Maintenance

And there’s one more reason to do this kind of data convertion on the boundary: maintenance.

What would happen if you used the struct Support.Domain.Customer all over your Sales context in services, domain, tests and so on… and for some reason you had to rename the attribute last_ticket_id to any other name ? Yeah, that’s right… you’d have to change a lot of code. But using a client allows you to handle this scenario in the client while keeping the struct Sales.Domain.Customer exactly the same, avoiding a major refactor. Just the client layers changes. It’s a simple example but I think it’s enough to get the ideia.

How to organize the internal context code is not in the scope of this article.

Contract

You may be wondering: if the public API returns a map, how can I know the content (contract) of this map ? Which attribures are present or not, which data type I should expect and so on ? And how can I know if the call was successful ? The answer is: use Typespecs.

defmodule Support do
  @type customer :: %{id: integer, last_ticket_id: integer}
  @spec get_customer(integer) :: {:ok, customer} | {:error, binary}
  def get_customer(customer_id) when is_integet(customer_id) do
    # your logic here ...
    # return the common pattern {:ok, data} or {:error, data}
    {:ok, customer}
  end
end

Typespecs is about transparency. You know what to expect from calling the function and also what that function expects as parameter. And it also has the advantage to generate an useful documentation using ex_doc library or even a live documentation by using text editors that supports that.

However it does not enforce an “automatic data validation”, that means you can pass an Integer even if the typespec says that it’s expecting an String. To assist you in the task of obeying the contract, I recommend using Dialyxir that tool will do a static code analysis and warn you if anything is out of expected.

When should I use an umbrella app ?

Putting everything inside just one app or splitting each part into umbrella apps is a question of balance and following some rules. Creating many apps may be overkill, but it’s worth considering creating a new app in some cases:

That part of the system is developed by another team, be it an internal or offshore team in the same company or an outsourced team.
That part of the system is shared between more than one app. That’s a more common scenario for shared libs.
That part of the system needs to be deployed separately in another server or in a different permission schema (some are public and some are private). — Distillery releases can help you with that.
Need to use more than one framework in the same system.

Everything else can live in the same app perfectly.

A note about umbrella projects

An umbrella project may get out of control if you don’t follow some rules like we discussed here. Think in each app as being isolated, ie: you can remove it from umbrella without breaking too many things. That’s the reason we can’t call internal functions from another app and need to rely on a public API.

There’s nothing in Elixir that forbids or prevents you from calling internal code from another app, instead of only using the public API. That’s a convention you’d have to adopt and always review the code to not break this rule.

Another commom issue with umbrella projects is circular dependencies that happens when app A depends on app B and for some reason you also try to make B depends on A. Then you get a circular dependency error. You have to carefully design the relationship between your bounded contexts to avoid this situation.

Another observation it that not everybody likes working with umbrella projects, and that’s fine. The same ideia and rules applies if you work with separated apps.

Summing up, umbrella project sits between monolithic and microservices. And if done right, you gain the best of both worlds.

Bonus: distributed apps

Umbrella project is a good way to start a new project and get things running, but it has known issues that may cause problems in the future. But don’t worry too much about it because the approach discussed here can evolve to break apart those apps into distributed apps by implementing each public API as a GenServer, which can be global registered using tools like Swarm or Horde. But that’s subject to another post, let me know if you want to hear more about it.

What I didn’t say

Domain Types, Anti-Corruption Layers, CQRS, DTOs, Sagas, CRDT and some other techniques and patterns can be useful in this kind of architecture, but I can’t talk about everything in just one article and you may not need all of it. Keep in mind the first and most important step is to organize your project to enable it to evolve in a agile way.

Thanks Eduardo Hernandes for the hours discussing Elixir, DDD and some crazy stuff and also for reviewing this article.

Elixir — quick reference for debugging techniques

Leandro Cesquini Pereira — Thu, 17 May 2018 03:00:00 +0000

Much has been said about Elixir debugging techniques, but in this post, I’d like to give a quick overview of all possible options to serve as a go-to reference when you need to debug Elixir code. Enough talking, let’s check each of them:

IO.inspect

The simplest technique:

my_list = [1, 2, 3]
IO.inspect(my_list)

# Outputs
[1, 2, 3]

IO.inspect/2can also be used inside pipelines because it returns the item passed to be inspected. And the tip here is to use the option label:to output a string identifying each inspect:

[1, 2, 3]
|> IO.inspect(label: "before")
|> Enum.map(&(&1 * 2))
|> IO.inspect(label: "after")
|> Enum.sum

# Outputs
before: [1, 2, 3]
after: [2, 4, 6]

IO.inspect with binding

binding/1 is very useful when you want to see all variable names and values of the current function:

def some_fun(a, b, c) do
  IO.inspect binding()
  ...
end

iex> some_fun(:foo, "bar", :baz)
[a: :foo, b: "bar", c: :baz]

apex

Similar to IO.inspect/2, apex is a lib worth mentioning especially because of its adef macro:

# import apex adef macro
import Apex.AwesomeDef

# change def to adef
adef test(data, options \\ []) do
  data
end

# when you call your function, you'll receive detailed information about its execution
iex(1)> Apex.test "foo"
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Function Elixir.Apex.test was called
defined in /Users/bjoernrochel/Coding/Laboratory/apex/lib/apex.ex:12
----------------------------------------------------------------------------------------------------
Parameters:
----------------------------------------------------------------------------------------------------
[
  [0] "foo"
  [1] []
]

----------------------------------------------------------------------------------------------------
Result:
----------------------------------------------------------------------------------------------------
"foo"

IEx.pry

But it can be very tedious and be limiting to debug with just data inspection likeIO.inspect or apex, that’s when IEx.pry/0 comes at hand because it allows you to pry into the current code. Put the following code at the line you want to pry:

def some_fun(a, b, c) do
  require IEx; IEx.pry
  ...
end

And now execute your code inside an IEx session: iex -S mix or iex -S mix phx.server if you are using Phoenix Framework. Tip: if you are running tasks like database seeds, you can pry into that code by running iex -S mix run priv/repo/seeds.exs or any other script.

Once the code execution gets to the point of IEx.pry , an interactive shell opens and allow you to interact with the current code.

Go to Debugging Phoenix with IEx.pry if you want more tips about debugging Phoenix with IEx.pry.

:debugger

Pretty much the same as IEx.pry which stops the execution at the break point and allows you to inspect the current code, but :debugger gives you a nice visual interface, like the ones in IDEs:

gif from http://blog.plataformatec.com.br/2016/04/debugging-techniques-in-elixir-lang

To open this debugger, you need to start it and set a break point:

# given that you have this module
defmodule MyApp.Example do
  def sum(x, y) do
    x + y
  end
end

# start a new iex session
$ iex -S mix

# then start :debugger
iex> :debugger.start()

# prepare your module for debugging
iex> :int.ni(MyApp.Example)

# set a break point at the line you want to capture
iex> :int.break(MyApp.Example, 3)

# and finally call your function
iex> MyApp.Example.sum(1,2)

:sys.get_state and :sys.get_status

This one works only for processes. As the name suggests, :sys.get_state/1gets the current state of a process, and not only from a GenServer but also from any kind of process:

iex(1)> defmodule Example, do: use GenServer
iex(2)> {:ok, pid} = GenServer.start_link(Example, %{ping: "pong"})
iex(3)> :sys.get_state(pid)
%{ping: "pong"}

Snippet extracted from Looking at the state of processes in Elixir.

If you need more data than just the current state, just call :sys.get_status/1, which will return whole process information:

{:status, #PID<0.134.0>, {:module, :gen_server},
  [
    [
      "$initial_call": {Sequence.Server, :init, 1},
      "$ancestors": [#PID<0.118.0>, #PID<0.57.0>]
    ],
    :running,
    #PID<0.118.0>,
    [statistics: {{{2017, 12, 23}, {14, 11, 13}}, {:reductions, 14}, 3, 0},
    [
      header: 'Status for generic server <0.134.0>',
      data: [
        {'Status', :running},
        {'Parent', #PID<0.118.0>},
        {'Logged events', []}
    ],
    data: [{'State', 103}]
  ]

# Excerpt From: Dave Thomas. “Programming Elixir ≥ 1.6"

Bonus: you can replace a process state at runtime using :sys.replace_state/2, which can be very handy to test some specific situation.

Process.info

You can also use Process.info/1 to get information about a specific process:

iex> defmodule Example, do: use GenServer
iex> {:ok, pid} = GenServer.start_link(Example, %{ping: "pong"})
iex> Process.info pid
[
  current_function: {:gen_server, :loop, 7},
  initial_call: {:proc_lib, :init_p, 5},
  status: :waiting,
  message_queue_len: 0,
  messages: [],
  links: [#PID<0.610.0>],
  dictionary: [
    "$initial_call": {Example, :init, 1},
    "$ancestors": [#PID<0.610.0>, #PID<0.70.0>]
  ],
  trap_exit: false,
  error_handler: :error_handler,
  priority: :normal,
  group_leader: #PID<0.60.0>,
  total_heap_size: 233,
  heap_size: 233,
  stack_size: 10,
  reductions: 27,
  garbage_collection: [
    max_heap_size: %{error_logger: true, kill: true, size: 0},
    min_bin_vheap_size: 46422,
    min_heap_size: 233,
    fullsweep_after: 65535,
    minor_gcs: 0
  ],
  suspending: []
]

:sys.trace

Still talking about process debug, another resource we have is :sys.trace/2to trace process calls, which will display each call and state change of the calling process:

iex> :sys.trace pid, true
:ok
iex> GenServer.call(pid, :next_number)
*DBG* <0.69.0> got call next_number from <0.25.0>
*DBG* <0.69.0> sent 105 to <0.25.0>, new state 106
105

# Excerpt From: Dave Thomas. “Programming Elixir ≥ 1.6” iBooks.

Distillery commands

After you deploy your application as a package release, it may be very helpful to get information about it, specially if something is not working as expected. If you have deployed using Distillery, you’ll soon discover that mix does not work the same way as in your local machine, and that’s expected. But you can execute some commands on that release by calling bin/<app_name> <command>, the two more useful commands are: remote_console to start an IEx session on the running release and help to give you a list of all commands.

:observer

Although it’s not exactly a debug tool, it’s worth mentioning that :observer helps you get an overview of the running system. I’ll not dig into details because there are a lot of articles that already do that, so I’ll just leave here the function call to start the observer:

iex> :observer.start

Notes and sources

If you have more tips and techniques, please leave a comment so I can update this post and help spread more Elixir knowledge. :)

Some code snippets were extracted from the great official documentation or from the official getting started tutorial.

The book Programming Elixir was also used as source.

The Today I Learned from Hashrocket is also a great source of tips.

Resenha do livro Direto ao Ponto

Leandro Cesquini Pereira — Mon, 04 Jan 2016 02:00:00 +0000

Resenha do livro Direto ao Ponto - Criando produtos de forma enxuta

Se você ou sua empresa irá iniciar um projeto ou mesmo está no meio de um projeto mas está meio perdido com todas atividades necessárias pra tirar a ideia da cabeça e disponibilizar um produto ao seu usuário final, então este livro é mais que recomendado, é praticamente obrigatório.

Como o próprio autor justifica, o livro traz uma receita de bolo que o ajudará a identificar a visão e objetivos do produto, personas, features e jornadas. Com estas valiosas informações, planejar as entregas enxutas, curtas e priorizadas. Em outras palavras irá te ajudar a construir um produto em etapas, o chamado MVP.

Mas meu produto é muito grande ou minha empresa não é uma startup !

O autor Paulo Caroli deixa claro no Prefácio que o livro nasceu da sua vasta experiência com muitas empresas, de startups a grandes corporações e também com o relacionamento com pessoas influentes na área. E o conteúdo do livro esclarece esta experiência, portanto é seguro dizer que este livro lhe trará benefícios, independente do tamanho ou tipo de projeto.

MVP

O livro inicia com uma explicação completa e muito didática do conceito de MVP, que pode ser resumido em:

“Produto mínimo viável (em inglês, Minimum Viable Product ─ MVP) é a versão mais simples de um produto que pode ser disponibilizada para a validação de um pequeno conjunto de hipóteses sobre um negócio.”

Inception Enxuta

Conceitos e técnicas ágeis são muito úteis para se usar no decorrer do projeto, porém fica a dúvida: como usar ? O que é preciso fazer ? Como juntar tudo e gerar resultado ?

E são justamente estas dificuldades que o autor demonstra como resolver de forma simples no formato de um workshop colaborativo com atividades passo-a-passo, que compreende desde a ideia do produto até o planejamento das entregas priorizadas pelo valor de negócio. É o caminho para o desenvolvimento do seu produto.

Você pode conferir a lista de atividades desempenhadas no workshop no sumário do livro.

Conclusão

O livro é de leitura fácil e com muitas fotos e desenhos para exemplificar as atividades realizadas no workshop. Inclusive o apêndice inclui um exemplo real da aplicação dos conceitos ensinados no livro.

É indicado para qualquer pessoa que queira organizar ou mesmo otimizar o seu processo de desenvolvimento de produtos pois o livro consegue ensinar áreas do planejamento que envolvem análise, riscos, estimativas e outras que você iria se desinteressar se fosse um livro clássico de gerenciamento de projetos.

Lotus DB Migrations Tips

Leandro Cesquini Pereira — Mon, 24 Aug 2015 03:00:00 +0000

Lotus, just like Rails, also supports database migration through Sequel Migrations. It´s pretty simple but some things are not so clear.

Create a migration

lotus g migration create_my_table

Enable UUID (PostgreSQL only)

Lotus::Model.migration do
  up do
    execute 'CREATE EXTENSION "uuid-ossp"'
  end

  down do
    execute 'DROP EXTENSION "uuid-ossp"'
  end
end

Use UUID as Primary Key

Lotus::Model.migration do
  up do
    create_table :my_table do
      column :id, :uuid, null: false, default: Sequel.function(:uuid_generate_v4), primary_key: true
    end
  end

  down do
    drop_table :my_table
  end
end

Use UUID as Foreign Key

Lotus::Model.migration do
  up do
    create_table :my_table do
      column :id, :uuid, null: false, default: Sequel.function(:uuid_generate_v4), primary_key: true
      foreign_key :author_id, :authors, type: 'uuid', null: false
    end
  end
end

Index a column

Lotus::Model.migration do
  up do
    create_table :my_table do
      column :code, :integer, null: false
      index :code
    end
  end
end

Table documentation

Lotus::Model.migration do
  up do
    create_table :my_table do
      column :code, :integer, null: false
    end

    execute %Q(COMMENT ON TABLE my_table IS 'You should do it')
    execute %Q(COMMENT ON COLUMN my_table.code IS 'Easy and useful')
  end
end

Migrate up

lotus db migrate

Migrate down

There´s no db migrate down command, although you can specify a version to migrate. Just specify 0 to migrate to initial version (before any migration).

lotus db migrate 0