Introducing Dialyzer & type-specs to an Elixir Project

At Contact Stack we use Elixir & Phoenix as the core of the backend. I am fascinated by the design & power of the BEAM, the virtual machine that Elixir targets, and I want to learn more about it. I also love strongly & statically typed languages, particularly due to my recent history working with Elm which has a great type system and a helpful compiler.

Elixir is dynamically typed language which leaves me a little nervous about the long term future of the project. I really like types and I find them incredibly useful when the code base grows larger and you're returning to parts of it that you haven't looked at in a while. Often I can't remember exactly the kind of data that I'm passing around and strong types really help with that understanding. Plus I've grown to love having a compiler guide you through the code base when doing refactoring as it points out all the places you need to change.

One way to bring a little more type safety to Elixir projects is to use Dialyzer. Dialyzer performs static analysis on your code and tries to warn you about anything that is almost certainly going to fail. It can be helped by providing type annotations and it also derives its own from inspecting the code. It isn't as good as a solid type system but it is better than nothing.

So, how do we go about introducing Dialzyer and type annotations through out our codebase? Here are the steps that I've followed for Contact Stack.

Install Dialyzer

Maybe a bit of an obvious first step! We can use the dialxyir project to add a dialyzer task to mix. At the time of writing that involves adding this line to your mix.exs deps:

{:dialyxir, "~> 1.0", only: [:dev], runtime: false},

Run Dialyzer & Fix issues

We can now run dialyzer with:

mix dialyzer

On the command line. This can take a long time as dialyzer analyses and creates PLT caches for all the Elixir & Erlang standard libraries. Fortunately it is much quicker on that next run though still not as fast a many compilers out there.

The errors can honestly be quite hard to understand. I would recommend googling as much as possible. I've learned a few of things though that I'll share.

Firstly, in my projects I've had to config dialyzer to make it away of ex_unit and mix otherwise it issues warnings. I suspect I only needed to add mix because I have a custom mix task. You can configure dialyzer by adding the following to your the project config in mix.exs:

dialyzer: [plt_add_apps: [:ex_unit, :mix], ignore_warnings: "config/dialyzer.ignore"]

Here we add ex_unit & mix and also indicate an ignore file.

Secondly, some errors don't seem to be clearly fixable by code that is under our control. In which case we want to add those errors to the ignore file. If you run dialyzer with:

mix dialyzer --format dialyzer

Then you can copy the start or all of the error that you want to ignore onto a new line in the ignore file we've just set up and dialyzer will skip it on future runs.

Thirdly, if you get a warning that says that some function has no local return then it normally means there is another warning within the function itself that makes dialzyer think that the function will never work. You can skip the 'no local return' warning at look at the next one in the output so see what the issue might be.

Introduce Credo

Credo is a linting tool for Elixir that can warn if you don't provide a type annotation on a function. Part of our goal here is to make sure that we provide type annotations for all the functions in our code base. This helps Dialyzer understand our intent and is self-documenting which helps ourselves and others on the code base.

So we add:

{:credo, "~> 1.2", only: [:dev, :test], runtime: false}

To our mix.exs deps.

Configure & Fix Credo Warnings

Credo checks for a lot of different things. It does not check for type annotations by default and we're going to enable that but first, as long as they are manageable, fix any current warnings that Credo has in place. You can do this by changing your code or suppressing some of Credo's warnings. I have the following config for my preferences:

%{
  configs: [
    %{
      name: "default",
      files: %{
        included: ["lib/"],
        excluded: []
      },
      checks: [
        {Credo.Check.Design.AliasUsage, false},
        {Credo.Check.Readability.AliasOrder, false}
      ]
    }
  ]
}

Extend Credo Rules to Include Type Annotations

We extend our config to include the Specs check by adding this line:

      checks: [
        {Credo.Check.Design.AliasUsage, false},
        {Credo.Check.Readability.AliasOrder, false}
+       {Credo.Check.Readability.Specs, []}
      ]

Silence the Huge Number of Credo Warnings

Now we can run credo again to get a sense of the number of missing type annotations. If you have a small number then go ahead and try to add them now. If, like me, you have something like 200 warnings or more then you might prefer to silence them for now and attend to them gradually.

To do this we can run the following command:

mix credo --strict | grep lib | cut -d " " -f 8 | cut -d ":" -f 1 | sort -u | xargs sed -i '1s/^/# credo:disable-for-this-file Credo.Check.Readability.Specs\n/'

This is going to add # credo:disable-for-this-file Credo.Check.Readability.Specs to the top of every file in your codebase that has missing type annotations.

When we run mix credo --strict again we should see a clean sheet. Now you can go into the files at your leisure and remove the comment at the top and add type annotations. Credo will warn about any that are missing and warn by default for any new files we add to the system.

Final Thoughts

I worry that dialyzer is not very useful in comparison to the type systems & compilers that I have used before but it feels good to be trying to make the code as tight and error free as possible. I am at the beginning of this journey so I can't really report on whether it is worth it long term but we'll see.

Comments

[Dennis Dang]

How has the Dialyzer turned out for you? Has it been sufficient? I’m also making this debate to introduce Elixir or some other fp/fp-like language for our backend.

[Michael Jones]

Good question :) I was thinking about this a bit the other day. I don't think I particularly recommend it though that might be due to my workflow. I currently get stuff working by manually poking around in the browser, and then I might write tests for it and then I run my script which recompiles with warnings as errors, runs the tests, runs credo and then runs dialyzer.

So by the time I'm running dialzyer, due to its nature & place in the chain, it is isn't really telling me anything useful. It might catch an edge case. More likely it complains about my types as they are a bit off or too general maybe and so I dutifully correct them and commit the changes.

I have the credo set up to tell me about missing types and so I try to go about adding those but they are often just copying and pasting from other controllers or putting some basic get_by(Model.id) :: Model.t in there or something. Nothing that in depth.

Unlike Elm or even Typescript where I'd do some kind of "type driven development", dialyzer feels like I'm doing "type homework" or something.

I think I have worked with the elixir language server a little and so I might get dialyzer warnings in my editor but the warnings are not that informative and it can take a while for the errors to refresh or something so you get left wondering if you've failed to fix the issue or if the cache has just not recalculated yet. I didn't love it.

I can't speak to its use in teams as I've only used Elixir for personal projects. Perhaps the documentation aspect of it pays off in teams or using a CI set up helps drive more value. Not sure.

[Dennis Dang]

Honest review! Thanks, Michael. I really love the simplicity and all-in-one feel of mix and the documentation culture.

But your experience aligns with my own experience, albeit it was a short 3 months. It’s still a little too risky then. My team relies heavily on JSON schema validation and type generation (from the schema) to keep our system tied up. Maybe it’ll just be too tough to trust Dialyzer to keep instep for now.

Has OTP and fault tolerance of the BEAM VM come in handy?

[Michael Jones]

I guess my previous comment should also state that I have dialyzer where it is in the workflow because it takes so long to run. It is neither quick enough nor helpful enough to warrant a higher position.

As to OTP & fault tolerance, Fred Hebert's description of the "Let it crash" philosophy (ferd.ca/the-zen-of-erlang.html) was the whole thing that got me into the Erlang ecosystem. But Erlang looks a bit weird so Elixir it was :)

That said, I've not done any OTP myself in anger. I understand that it probably underpins Phoenix and some of the other libraries that I might lean on but I haven't yet found myself with tasks that have been best solved by building my own supervisor tree or agents or anything so I'm still a bit inexperienced with it all.

I came from Django and have enjoyed how snappy Phoenix seems to feel and I have faith in the underpinnings of it and the BEAM and functional programming with immutable data in a way that I don't really have faith in the stack required to get Django serving requests but I can't say I dug deep into it all.

At the moment it is the types that holds me up from truly embracing it. I had a happy 4 years of using Elm for my day job and I find it a bit unsettling to head back to a dynamic language like Elixir and the kind trial & error approach to programming that I have with it.

I am excited by Gleam which adds an Elm like experience (though with a curly brace syntax) to the Erlang ecosystem but at the same time I find myself finding enough value in Phoenix and now specifically Phoenix Live View that I struggle to figure out how much I could incorporate or switch to Gleam for the few projects that I have. I look forward to that community growing though and the libraries & frameworks that will come with it.