DEV Community

Cover image for Perfect Elixir: Environment Setup
Jon Lauridsen
Jon Lauridsen

Posted on • Updated on

Perfect Elixir: Environment Setup

#elixir #tutorial #development #webdev

We need Erlang and Elixir installed, which might sound simple but there are tradeoffs to consider when setting up a development environment such that it can be shared across a team. And because most projects need to persist data we'll also add a PostgreSQL database so our explorations can stay relevant to real-world complexities.

Let's try some different approaches first, and then discuss their pros and cons at the end.

Table of Contents

Just… Install the Dependencies?

Why not just install the dependencies as each tool's website suggests? Why make this any more complicated?

I’m on MacOS and erlang.org, elixir-lang.org, and postgresql.org all suggest installation via Homebrew, which is a very popular package manager for MacOS.

So, let's first install Homebrew:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
Enter fullscreen mode Exit fullscreen mode

🖥️ Terminal

Install Homebrew

And then we install our tools:

$ brew install erlang elixir postgresql@15
Enter fullscreen mode Exit fullscreen mode

🖥️ Terminal

Brew install erlang, elixir, and postgresql

And… we’re done!?

But there are critical problems with this approach:

  • No versioning - There's no control over what versions just got installed, so depending on when a person runs the install command they'll end up with whatever unpredictable mix of versions are available at that time. This alone is enough to entirely discard this approach.

  • Globally installed - The tools all got globally installed, which means all other projects are impacted as well. This is a recipe for disaster as we need to be in control of versions per project.


So “just installing” isn’t viable: We need to fully specify our environment such that the exact same versions are installed on all our machines. Thankfully there are entire projects made specifically to solve this, so let's go explore those.

asdf

asdf is a version manager with a lot of plugins (for our needs: Erlang, Elixir, and Postgres). The asdf installation guide says to first install some system dependencies via Homebrew and then clone the asdf repository:

$ brew install coreutils curl git
…
$ git clone https://github.com/asdf-vm/asdf.git ~/.asdf --branch v0.13.1
$ echo '. "$HOME/.asdf/asdf.sh"' >> ~/.zshrc
Enter fullscreen mode Exit fullscreen mode

🖥️ Terminal

Brew install coreutils, curl, and git

ℹ️ BTW I find it quite strange to install asdf via git clone. And asdf can actually be installed directly via Homebrew, but according to the asdf installation guide it’s “highly recommended” to use the Git method so that’s what I’m sticking with here 🤷

And then in a fresh terminal we add plugins:

$ asdf plugin add erlang https://github.com/asdf-vm/asdf-erlang.git
$ asdf plugin-add elixir https://github.com/asdf-vm/asdf-elixir.git
$ asdf plugin-add postgres
Enter fullscreen mode Exit fullscreen mode

🖥️ Terminal

Add plugins to asdf

And then we need to go to each plugin’s project page and note down which dependencies and configuration options they need, which aggregates into these two lines:

$ brew install autoconf openssl@1.1 openssl libxslt fop gcc readline zlib curl ossp-uuid
$ echo 'export KERL_CONFIGURE_OPTIONS="--without-javac --with-ssl=$(brew --prefix openssl@1.1)"' >> ~/.zshrc
Enter fullscreen mode Exit fullscreen mode

🖥️ Terminal

Add plugin dependencies

I don't really know (or care to know) why those exact dependencies are needed, and I surely won't notice if a plugin suddenly changes its requirements… but we'll just push onwards and now create a .tool-versions file which is how asdf is configured to install certain tools:

$ cat << EOF >> .tool-versions
erlang 26.2.1
elixir 1.16.0
postgres 15.5
EOF
Enter fullscreen mode Exit fullscreen mode

And now we just have to invoke asdf install to install everything:

$ asdf install
...
$ which erl
/Users/cloud/.asdf/shims/erl
$ which elixir
/Users/cloud/.asdf/shims/elixir
$ which psql
/Users/cloud/.asdf/shims/psql
Enter fullscreen mode Exit fullscreen mode

🖥️ Terminal

Run asdf install

We now have all our tools installed 🎉

direnv

Next let's think on how we can keep our environment in sync across the team… i.e. if we update .tool-versions how will anyone know to re-run asdf install?

One common solution is to handle this automatically via direnv which is made for running certain commands upon entering a folder.

Direnv even comes in a special asdf flavor, which we'll install by adding it to the .tool-versions file, then activate it for our shell, and finally create an .envrc file (that's how direnv gets configured to execute a command every time the folder is entered):

$ asdf plugin-add direnv
$ echo direnv 2.30.0 >> .tool-versions 
$ asdf install
$ asdf direnv setup --shell zsh --version 2.30.0
$ asdf direnv local
$ echo "use asdf" > .envrc
Enter fullscreen mode Exit fullscreen mode

🖥️ Terminal

Add asdf direnv

If we now simulate a change to .tools-versions by changing the required erlang version, we'll see a prompt to re-install asdf dependencies. This way the whole team will keep our environments synchronized:

$ cd perfect-elixir/
direnv: loading ~/perfect-elixir/.envrc
direnv: using asdf
direnv: Creating env file /Users/cloud/.cache/asdf-direnv/env/1510633598-1931737049-390094659-716574907
direnv: erlang 26.2.2 not installed. Run 'asdf direnv install' to install.
direnv: referenced  does not exist
$ asdf direnv install
Downloading 26.2.2 to /Users/cloud/.asdf/downloads/erlang/26.2.2...
...
$ which erl
/Users/cloud/.asdf/installs/erlang/26.2.2/bin/erl
Enter fullscreen mode Exit fullscreen mode

🖥️ Terminal

Show how asdf direnv automatically checks for dependencies on entering the folder

With this we have the essence of a workflow where asdf helps us keep our environments synced 🎉

We could keep improving this solution, but I'm not sure it's worth it compared to exploring the other popular tools that are available.

Nix

Nix is a tool "for reproducible and declarative configuration management", which is a fancy way of saying it'll manage environments. It's available for MacOS and Linux, and it's full of packages and a very active community. Let’s give it a try!

First step, as always, is to install:

sh <(curl -L https://nixos.org/nix/install)
Enter fullscreen mode Exit fullscreen mode

🖥️ Terminal

Installing nix

ℹ️ BTW the installer requires sudo, and it does some exotic things so be aware of that before running. E.g. it creates a new “Nix Store” drive-volume, and also creates 32 hidden new users 🤷

Nix is based around a custom pseudo programming language that allows for programmatically specifying dependencies. It took me a while to even get the most basic thing going (various articles all seem to point in different directions, and there's some complicated language to understand), but I'm pretty sure we're supposed to put Nix code into a file called flake.nix because "Flakes" are how Nix deals with reproducible environments. The nix CLI suggests an example we can follow:

$ nix --help
...
Examples

      · Create a new flake:

          | # nix flake new hello
          | # cd hello
...
Enter fullscreen mode Exit fullscreen mode

Before creating a new flake file we must first enable some "experimental features" because I guess flakes are still new:

$ mkdir -p ~/.config/nix && echo "experimental-features = nix-command flakes" > ~/.config/nix/nix.conf
Enter fullscreen mode Exit fullscreen mode

And then we can generate a flake file:

$ nix flake new .
wrote: /Users/cloud/Documents/nix/flake.nix
$ cat flake.nix
{
  description = "A very basic flake";
  outputs = { self, nixpkgs }: {
    packages.x86_64-linux.hello = nixpkgs.legacyPackages.x86_64-linux.hello;
    packages.x86_64-linux.default = self.packages.x86_64-linux.hello;
  };
}
Enter fullscreen mode Exit fullscreen mode

I don't know why that generated file references "legacyPackages", but err… let's just ignore that. And I don't know why it picked the "x86_64-linux" flavor when my platform is "x86_64-darwin", but we can edit our way out of that. And while we're editing let's also specify the dependencies we want, so the file looks like this:

$ cat flake.nix
{
  description = "A flake";
  outputs = { self, nixpkgs }: {
    devShells.x86_64-darwin = {
      default = nixpkgs.legacyPackages.x86_64-darwin.mkShell {
        buildInputs = [
          nixpkgs.legacyPackages.x86_64-darwin.erlangR26
          nixpkgs.legacyPackages.x86_64-darwin.elixir_1_16
          nixpkgs.legacyPackages.x86_64-darwin.postgresql_15
        ];
      };
    };
  };
}
Enter fullscreen mode Exit fullscreen mode

ℹ️ BTW I found what versions are available by running the command nix search nixpkgs darwin.erlang, for each of the tools.

And now we can activate this flake by running nix develop, which starts a new shell that has all our dependencies available:

$ nix develop
...
macOS-14:perfect-elixir cloud$ which erl
/nix/store/49qw7cw30wszrfn3sa23qnlskyvbnbhi-erlang-26.2.2/bin/erl
macOS-14:perfect-elixir cloud$ which elixir
/nix/store/rr6immch9mp8dphv1jvgxym35za4b7jy-elixir-1.16.1/bin/elixir
macOS-14:perfect-elixir cloud$ which psql
/nix/store/v5ym92k3kss1af7n1788653vis1d6qsc-postgresql-15.5/bin/psql
Enter fullscreen mode Exit fullscreen mode

🖥️ Terminal

Run nix develop

Cool!

And if we exit the shell the tools are no longer available:

macOS-14:perfect-elixir cloud$ which erl
/nix/store/49qw7cw30wszrfn3sa23qnlskyvbnbhi-erlang-26.2.2/bin/erl
macOS-14:perfect-elixir cloud$ exit
exit
$ which erl
erl not found
Enter fullscreen mode Exit fullscreen mode

🖥️ Terminal

Exiting the Nix shell makes tools unavailable again

So now we have this safe, reproducible specification of our environment that is guaranteed to be the same across any computer. Nice!

direnv

Starting a new shell to do any work is a bit cumbersome though, can we somehow automate all this so we can "just" sit down to develop and trust the environment-details are taken care of?

Let's once again turn to direnv.

First we need to install it, which can be done directly via Nix:

$ nix-env -iA nixpkgs.direnv;
$ echo 'eval "$(direnv hook zsh)"' >> ~/.zshrc
$ source ~/.zshrc
Enter fullscreen mode Exit fullscreen mode

🖥️ Terminal

Install direnv via Nix

ℹ️ BTW this isn't done via the flake.nix file because we need direnv to be available globally, not just when the flake is active. You might also notice this command doesn't specify a version (in fact Nix doesn't track versions of direnv at all), but it's very unlikely this imprecision will become a problem because direnv is very unlikely to change its basic interface.

And with direnv installed we can now automatically activate the flake:

$ echo "use flake" > .envrc
$ direnv allow
direnv: loading ~/Documents/nix/.envrc
direnv: using flake
…
$ which erl
/nix/store/rp1c50s0w039grl22q086h0dyrygk0p2-erlang-26.2.1/bin/erl
$ which elixir
/nix/store/66f9b1d1c4fmhz6bd3fpcny6brjm0fk7-elixir-1.16.0/bin/elixir
$ which psql
/nix/store/zhk6mf2y5c07zqf519zjkm3fm2nazmvj-postgresql-15.5/bin/psql
Enter fullscreen mode Exit fullscreen mode

🖥️ Terminal

Activate Nix Flake via Direnv

So now whenever we go into the folder our environment will automatically be activated, and because we can trust nix to track the versions our entire team will now have the same replicated environment. Awesome! 🎉

ℹ️ BTW direnv insists we run direnv allow whenever the .envrc file changes because it's a vector for malicious code. Always audit what has changed in .envrc before re-allowing it.

Bootstrapping

At this point there's just one more aspect to consider: How does someone "hook into" all this, when they come in with a new computer? We need some kind of bootstrap script that installs Nix and makes direnv available to get people into our ecosystem.

A simple first incarnation of such a script could be something like this, which installs and configures nix and direnv if they aren't already installed:

$ cat install.sh
#!/bin/zsh
set +euo pipefail

# If nix isn't installed, install it
command -v nix-build &> /dev/null || {
  echo "Installing Nix..."; sh <(curl -L https://nixos.org/nix/install);
  echo "Nix installed, please restart terminal & re-run this script"
  exit 1
}

# Configure nix
mkdir -p ~/.config/nix && echo "experimental-features = nix-command flakes" > ~/.config/nix/nix.conf

# If direnv isn't installed, install it
command -v direnv &> /dev/null || {
  echo "Installing direnv..."; nix-env -iA nixpkgs.direnv;
}

# Ensure direnv is activated in the shell
grep -q 'eval "\$(direnv hook zsh)"' ~/.zshrc || {
  echo 'eval "$(direnv hook zsh)"' >> ~/.zshrc
  echo "Hook added to .zshrc, please restart terminal & re-run this script"
  exit 1;
}

echo "Setup complete"
Enter fullscreen mode Exit fullscreen mode

That is a simple script, and just for illustration. It only works in zsh shells, and makes other questionable assumptions, but only you know your context so only you can write the script that fits your requirements. But this simple script at least illustrates how we can now have a workflow where new joiners can onboard themselves, and everyone the same identical environment made available to them. Great! 🎉

ℹ️ BTW there is a package called nix-direnv which I intended to use. But its readme recommends installation via something called Home Manager using Nix, which describes itself as a way to "Manage a user environment using Nix". Unfortunately Home Manager's readme offers words of warning that says it's a difficult system that can possible mess up your home directory. So… I just installed direnv directly via Nix 👀

pgkx

pkgx has the tagline "RUN ANYTHING", and it's not kidding around: pkgx node --help is all it takes to run Node.JS, and pkgx also promises to have support for development environments. And it's by the people behind Homebrew so they surely know a thing or two about installing tools. Let's give it a try!

First we install and activate pkgx:

brew install pkgxdev/made/pkgx
…
eval "$(pkgx integrate)"
Enter fullscreen mode Exit fullscreen mode

🖥️ Terminal

Install pkgx via Homebrew, then activate pkgx

Then we write a .pkgx.yml file to specify the dependencies we want:

$ cat .pkgx.yml
dependencies:
  erlang.org: =26.2.1
  elixir-lang.org: =1.16.0
  postgresql.org: =15.2.0
Enter fullscreen mode Exit fullscreen mode

And then the docs say we just have to run the built-in companion-command called dev to activate everything:

$ dev
env +erlang.org=26.2.1 +elixir-lang.org=1.16.0 +postgresql.org=15.2.0
$ which erl
/Users/cloud/.pkgx/erlang.org/v26.2.1/bin/erl
Enter fullscreen mode Exit fullscreen mode

🖥️ Terminal

Running dev command

And… that’s it?! All our tools are now available. And they even disappear when we leave the folder:

$ cd ..
env -erlang.org=26.2.1 -elixir-lang.org=1.16.0 -postgresql.org=15.2.0
$ which erl
erl not found
$ cd perfect-elixir
env +erlang.org=26.2.1 +elixir-lang.org=1.16.0 +postgresql.org=15.2.0
$ which erl
/Users/cloud/.pkgx/erlang.org/v26.2.1/bin/erl
Enter fullscreen mode Exit fullscreen mode

🖥️ Terminal

Leaving folder disables pkgx-provided tools

That is really as simple and magic as it gets! 🤯

Bootstrapping

Let's wrap that up in a nice install-script, to make it easy for others to get started:

$ cat install
#!/bin/zsh
set +euo pipefail

command -v brew &>/dev/null || /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
command -v pkgx &>/dev/null || brew install pkgxdev/made/pkgx
command -v dev &>/dev/null || eval "$(pkgx integrate)"
command -v erl &>/dev/null || dev

echo "Setup complete."
$ source install # we source it to let it fully activate pkgx in our current shell, so we don't have to restart the terminal
Enter fullscreen mode Exit fullscreen mode

🖥️ Terminal

Sourcing the pkgx bootstrap script

Again we've written a very simple bootstrapping script here, and you'll have to imagine all the ways we could modify it to be more resilient and flexible. But it conveys the essence of easily onboarding into our shared environment, because all a new joiner has to do is source that script. Doesn't get much easier than that!

In Conclusion

asdf Thoughts

I think asdf is a pretty popular choice, but I must say I didn't have much joy working my way through this chapter. Maybe it's a personality thing, but I feel the installation was awkward right out the gate with the system dependencies and git cloning, and then having to cater to each plugin's unique requirements nosedived my joy.

It's not a bad tool at all, but I end up feeling it's a bit cumbersome or old-fashioned, if that makes sense?

If you think I used it wrong or have simpler ways of using asdf please reach out in the comments, I'm happy to learn!

Nix Thoughts

Wow, it was both really pleasing to get Nix working and a stress-test of my patience with the rough edges I encountered. The more I read the more I started feeling nix is like an entire lifestyle I can invest my life into. It's a really impressive solution for being 100% in charge of dependencies, I can see how it must be amazing for setting up an entire operating system and creating ultra-super-flexible installation definitions where every detail of every dependency is controllable. Nix will definitely support any level of parameterized and reusable sub-trees of dependencies that can be combined and aggregated to perfectly express any ultra-sophisticated environments, and I really am genuinely impressed by all of it.

But it's also not at all simple. And I sure didn't feel joy working through it. I'm not even sure I'm using Nix correctly at all, and part of me fears getting flamed in the comments by whatever obvious things I've missed.

Just to illustrate with an example, this is the kind of experience I had getting into Nix where it almost seems helpful but then doesn't follow through:

$ nix-env
error: no operation specified
Try 'nix-env --help' for more information.
$ nix-env --help
Unknown locale, assuming C
No manual entry for nix-env
Enter fullscreen mode Exit fullscreen mode

But even with the rough edges aside, it also seems like quite a lot of effort to get some system dependencies installed. I mean, it's an entire new domain to learn, with a whole custom pseudo programming language, and backed by a pretty invasive sudo-based installation. It's possible I'll end up needing the flexibility of Nix one day and then I won't hesitate to pick it up, but for now I'll politely move on.

Please do comment if you have alternative Nix solutions though, I'm happy to learn!

pkgx Thoughts

I am quite frankly blown away at how little effort it took to learn and use pkgx: A simple non-sudo installation, immediately straight-forward specification of dependencies, and an incredibly easy way to get it all activated. I'm just so fond of how pkgx feels: It does exactly what it needs for dependencies, and gets out of the way for anything else. Each distinct use case feels clearly expressed, and that dev command is almost too good to be true right?? What a joy to use!

And we didn’t even cover how pgkx also supports adding one-off dependencies to individual scripts, here's a small example that uses the GitHub CLI:

$ cat gh-script
#!/bin/zsh
eval "$(pkgx --shellcode)"
# ^^ integrates `pkgx` during this script execution
env +cli.github.com
gh --help
Enter fullscreen mode Exit fullscreen mode

Running it, it transparently makes the GitHub CLI available inside the script, in this case showing its help text:

$ ./gh-script
 ✓  ~/.pkgx/cli.github.com/v2.44.1
Work seamlessly with GitHub from the command line.

USAGE
  gh <command> <subcommand> [flags]
Enter fullscreen mode Exit fullscreen mode

🖥️ Terminal

Running gh-script, showing GitHub CLI help output

It's a great technique to prevent the root .pkgx.yml from having to be a superset of all possible dependencies the project needs, because that makes it difficult to upgrade anything because unknown amount of code could break as a result. Instead when a script specifies its own dependencies they can be upgraded piecemeal and as needed. Quite cool!

Of course everything is a tradeoff, and pkgx is quite new so it doesn’t have the largest set of packages available yet (even if it is growing fast). Nothing time won't solve, but right now there are some shades being on the "cutting edge" by adopting it, which isn't for everyone.

A the end of the day I'm not here to declare "winners" in any objective sense, because only you know your own context well enough to decide what tradeoffs are worth it. But I'll say for my own preferences and needs I clearly feel drawn towards pkgx, so that's what I'll be going with.

Top comments (3)

Collapse
 
03juan profile image
Juan Barrios •

Great breakdown of the contemporary options. I use asdf with direnv and know of nix, but its configuration seemed pretty daunting and just another rabbit hole to go down. Thanks for introducing pkgx, looks very sleek and will definitely try it out.

I wonder what the journey is like with nix and pkgx to install erlang and elixir with compiled source docs?

In asdf, erlang docs are installed by setting some env variables, and the elixir plugin will compile any github version tag or sha ref from scratch with e.g. asdf install elixir ref:v1.17.0-dev. This way you can navigate to any erlang or elixir function's source from your own codebase in an IDE.

Collapse
 
jonlauridsen profile image
Jon Lauridsen •

Thanks for the feedback, and yeah I also wonder about that journey. We can see the pkgx Erlang package page here which leads to the actual pkgx package.yml definition here, and in that we can spot the familiar environment flags of --without-javac & friends. I guess we could make a PR that adds a configuration variant?

Very interesting to hear about the ability to navigate to built-in source, I can't quite find a link that describes this so if you have any resources (or can write a blog about it 😅) I'd be very happy to receive.

Collapse
 
03juan profile image
Juan Barrios •

Yes absolutely, you're very welcome to write about it! Wish I had the time... 🫠

I posted a reply on the Elixir forum about this and as luck would have it the link was in my notifications as a recent popular link.

Please link your blog post to the form thread when you're done, I'm sure people would want to read it, too.

Read next

anmolbaranwal profile image

🚀 21 Tools to take your dev skills to the moon 🌝

Anmol Baranwal -

kigazon profile image

The Tale of Tailwind CSS and React

Josef Held -

ayush010890 profile image

3 Ways to Add a Table of Contents to Your Blog Post

Ayush Sharma -

alexanderlamdan profile image

What is best tips for improving your development skills?

Alexander -