A Few Caveats to Running Elixir Tests in Containers and CI

#docker #elixir #devops #thisweekilearned

As a fitting follow up to last week's post dealing with proper database configuration and Docker containers, this week I ran into a related snag while trying to get my tests to run in the app container. When I ran the mix test command through docker-compose:

docker-compose exec web mix test

It failed with the following error:

	** (Mix) The database for MyApp.Repo couldn't be created: killed

view raw week_sept7_error01 hosted with ❤ by GitHub

Although this wasn't the exact same error from my previous experience, it's in a similar vein. I checked out the test.exs configuration, and low and behold, in the Repo configuration I found:

	config :my_app, MyApp.Repo,
	username: "postgres",
	password: "postgres",
	database: "my_app_test#{System.get_env("MIX_TEST_PARTITION")}",
	hostname: "localhost", << this is the culprit
	pool: Ecto.Adapters.SQL.Sandbox

view raw week_sept7_config01.ex hosted with ❤ by GitHub

I updated hostname from "localhost" to "db" -the name of the database container- and re-ran:

docker-compose exec web mix test

It successfully compiled, ran the tests, and everything was good to go, or so I thought. After creation of the pull request the CI spun up a new check... and it failed. To my surprise, it was failing at the same step and showing the same error as I discussed in last weeks post, only this time instead of the error happening on my local machine, it was happening on the CI build:

	** (DBConnection.ConnectionError) tcp connect (db:5432): non-existing domain - :nxdomain

view raw week_sept7_error02 hosted with ❤ by GitHub

After changing the test.exs configuration and pushing that up to the pull request to confirm, I concluded that the CI workflow needed the testing database hostname to be "localhost" while it also needed to be "db" for the tests to run in the app container.

The solution I found for this utilized System.get_env/1. I found a post that shows how to set configurations based on environment variables. When the app is being run in the CI workflow, there is a GITHUB_ACTION environmental variable set, so a conditional can be built around that:

	if System.get_env("GITHUB_ACTIONS") do
	config :my_app, MyApp.Repo, hostname: "localhost"
	end

view raw week_sept7_conditional01.ex hosted with ❤ by GitHub

Placing that conditional under the initial configuration will override the line hostname: "db" when the app is running in a CI workflow. The entire setup looks like this:

	config :my_app, MyApp.Repo,
	username: "postgres",
	password: "postgres",
	database: "my_app#{System.get_env("MIX_TEST_PARTITION")}",
	hostname: "db",
	pool: Ecto.Adapters.SQL.Sandbox

	if System.get_env("GITHUB_ACTIONS") do
	config :my_app, MyApp.Repo, hostname: "localhost"
	end

view raw week_sept7_config02.ex hosted with ❤ by GitHub

With that conditional set, the tests successfully run in the local app container using "db" and in the CI workflow using "localhost".

But there's more, a potential for refactoring! While researching and digging deeper into this, I came across a post that utlized environment variables in a slightly different way, which I actually preferred and refactored my work to emulate. He suggested setting an environmental variable in the CI workflow file itself, then injecting that variable as a conditional in the original database configuration. My original mix test command for the CI workflow was this:

	- name: Run tests
	run: mix test

view raw week_sept7_yml01.yml hosted with ❤ by GitHub

Adding an environmental variable looks like this:

	- name: Run tests
	run: mix test
	env:
	DB_HOSTNAME: "localhost"

view raw week_sept7_yml02.yml hosted with ❤ by GitHub

Now, in test.exs, instead of that entire conditional under the original configuration, the hostname field can be set conditionally as a one-liner:

	config :my_app, MyApp.Repo,
	username: "postgres",
	password: "postgres",
	database: "my_app_test#{System.get_env("MIX_TEST_PARTITION")}",
	hostname: System.get_env("DB_HOSTNAME") \|\| "db",
	pool: Ecto.Adapters.SQL.Sandbox

view raw week_sept7_config03.ex hosted with ❤ by GitHub

But that one-liner can be refactored even further. Theres also a System.get_env/2 that allows for two arguments to be passed, the first is the environmental variable, and the second is a default value if that environmental variable cannot be found. So, instead of using || to set the hostname to "db" if System.get_env("DB_HOSTNAME") returns nil, it can be all be done in the single function call: