Ivan Iraci

Posted on Jul 10, 2019

Deploy a Phoenix app with Docker stack

#docker #elixir #phoenix

The mandatory introductions

Hi everybody, this is my very first post on DEV and I want it to be as short as possible, so that anyone could be able to go straight to the point, if deploying a Phoenix app on docker is the problem at hand.

We will take advantage of the new "mix release" feature released with Elixir 1.9.

I will assume your app needs a Postgres DB. If your architecture is more complex than this (Redis, Mongo, whatever) the deployment strategy for any other piece of software included in your architecture is beyond the scope of this article.

Ok, let's go!

Releasing locally...

... first without docker

In the following examples, our app's name is Demo (so replace any occurrence of "demo" with your app's real name).

First of all we have to make sure our app will "mix release" locally with a production environment setup.

Run the following command in you console, at the root of your project:

mix release.init

Then create a releases.exs file inside your project's /config dir:

import Config

secret_key_base = System.fetch_env!("SECRET_KEY_BASE")
app_port = System.fetch_env!("APP_PORT")
app_hostname = System.fetch_env!("APP_HOSTNAME")
db_user = System.fetch_env!("DB_USER")
db_password = System.fetch_env!("DB_PASSWORD")
db_host = System.fetch_env!("DB_HOST")

config :demo, DemoWeb.Endpoint,
  http: [:inet6, port: String.to_integer(app_port)],
  secret_key_base: secret_key_base

config :demo,
  app_port: app_port

config :demo,
  app_hostname: app_hostname

# Configure your database
config :demo, Demo.Repo,
  username: db_user,
  password: db_password,
  database: "demo_prod",
  hostname: db_host,
  pool_size: 10

We are going to keep all of our production "secrets" in an .env file in the root of our project:

APP_PORT=4000
APP_HOSTNAME=localhost
DB_USER=postgres
DB_PASSWORD=pass
DB_HOST=localhost
SECRET_KEY_BASE=Y0uRvErYsecr3TANDL0ngStr1ng

The APP_HOSTNAME will be localhost for testing your app locally but later it will need to be set to your real domain name (e.g.: myverycoolapp.com), as you see in the comments of /config/prod.exs which needs to be edited as follows, in order to get host and port from our .env file. Make sure to uncomment the last line and to remove the "import_config "prod.secret.exs" from the file (since our "secrets" are in .env):

use Mix.Config

# Don't forget to configure the url host to something meaningful,
# Phoenix uses this information when generating URLs.
config :demo, DemoWeb.Endpoint,
  load_from_system_env: true,
  url: [host: Application.get_env(:demo, :app_hostname), port: Application.get_env(:demo, :app_port)],
  cache_static_manifest: "priv/static/cache_manifest.json"

# Do not print debug messages in production
config :logger, level: :info

# Which server to start per endpoint:
#
config :demo, DemoWeb.Endpoint, server: true

Remember to edit init/2 in /lib/demo_web/endpoint.ex:

  @doc """
  Callback invoked for dynamically configuring the endpoint.

  It receives the endpoint configuration and checks if
  configuration should be loaded from the system environment.
  """
  def init(_key, config) do
    if config[:load_from_system_env] do
      port = Application.get_env(:demo, :app_port) || raise "expected the PORT environment variable to be set"
      {:ok, Keyword.put(config, :http, [:inet6, port: port])}
    else
      {:ok, config}
    end
  end

To be able to manage our migrations in the released app, we need to create a /lib/demo/release.ex:

defmodule Demo.Release do
  @app :demo

  def migrate do
    for repo <- repos() do
      {:ok, _, _} = Ecto.Migrator.with_repo(repo, &Ecto.Migrator.run(&1, :up, all: true))
    end
  end

  def rollback(repo, version) do
    {:ok, _, _} = Ecto.Migrator.with_repo(repo, &Ecto.Migrator.run(&1, :down, to: version))
  end

  defp repos do
    Application.load(@app)
    Application.fetch_env!(@app, :ecto_repos)
  end
end

Ok, we are ready to try to release our app locally.

All you have to do is execute the following commands in your console:

mix phx.digest
MIX_ENV=prod mix release

If nothing went wrong, you will be welcomed with the following instructions:

* assembling demo-0.1.0 on MIX_ENV=prod                                                                                                               
* using config/releases.exs to configure the release at runtime                                                                                       
* skipping elixir.bat for windows (bin/elixir.bat not found in the Elixir installation)                                                               
* skipping iex.bat for windows (bin/iex.bat not found in the Elixir installation)                                                                     

Release created at _build/prod/rel/demo!

    # To start your system
    _build/prod/rel/demo/bin/demo start

Once the release is running:

    # To connect to it remotely
    _build/prod/rel/demo/bin/demo remote

    # To stop it gracefully (you may also send SIGINT/SIGTERM)
    _build/prod/rel/demo/bin/demo stop

To list all commands:

    _build/prod/rel/demo/bin/demo

But before we can start and try our released app, we need to migrate our database, typing the following command in our console:

source .env
_build/prod/rel/demo/bin/demo eval Demo.Release.migrate

Then, you can start your app as suggested above:

_build/prod/rel/demo/bin/demo start

Is your app working as intended? I hope so. If yes we can move on.

... and then with docker

We want our app to be as light as it can be, so we are going to use two docker images based on elixir:alpine and, of course, alpine.

We are going to have a multistage Dockerfile. In the first stage we are going to build our release, in the second one we are going to deploy our released app and a postgres client (that we will use to know if the database is ready to accept connections and to run our migrations).

This is our Dockerfile and I suggest you to read it very carefully:

# ---- Build Stage ----
FROM elixir:alpine AS app_builder

# Set environment variables for building the application
ENV MIX_ENV=prod \
    TEST=1 \
    LANG=C.UTF-8

RUN apk add --update git && \
    rm -rf /var/cache/apk/*

# Install hex and rebar
RUN mix local.hex --force && \
    mix local.rebar --force

# Create the application build directory
RUN mkdir /app
WORKDIR /app

# Copy over all the necessary application files and directories
COPY config ./config
COPY lib ./lib
COPY priv ./priv
COPY mix.exs .
COPY mix.lock .

# Fetch the application dependencies and build the application
RUN mix deps.get
RUN mix deps.compile
RUN mix phx.digest
RUN mix release

# ---- Application Stage ----
FROM alpine AS app

ENV LANG=C.UTF-8

# Install openssl
RUN apk add --update openssl ncurses-libs postgresql-client && \
    rm -rf /var/cache/apk/*

# Copy over the build artifact from the previous step and create a non root user
RUN adduser -D -h /home/app app
WORKDIR /home/app
COPY --from=app_builder /app/_build .
RUN chown -R app: ./prod
USER app

COPY entrypoint.sh .

# Run the Phoenix app
CMD ["./entrypoint.sh"]

Create an entrypoint.sh (and make it executable) at the root of your project:

#!/bin/sh
# Docker entrypoint script.

# Wait until Postgres is ready
while ! pg_isready -q -h $DB_HOST -p 5432 -U $DB_USER
do
  echo "$(date) - waiting for database to start"
  sleep 2
done

./prod/rel/demo/bin/demo eval Demo.Release.migrate

./prod/rel/demo/bin/demo start

Now we can build our image:

docker build -t demo-app .

Since we need to have a postgres instance running, here is a docker-compose.yml that will take care of both our app and a database:

version: '3.1'

services:

  database:
    image: postgres
    restart: always
    environment:
      POSTGRES_USER: ${DB_USER}
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_DB: demo_prod

  web:
    image: demo-app
    restart: always
    ports:
      - ${APP_PORT}:${APP_PORT}
    environment:
      APP_PORT: ${APP_PORT}
      DB_USER: ${DB_USER}
      DB_PASSWORD: ${DB_PASSWORD}
      DB_HOST: ${DB_HOST}
      SECRET_KEY_BASE: ${SECRET_KEY_BASE}
    depends_on:
      - database

Now you need to edit your .env and change at least the DB_HOST (you can leave the db credentials unchanged, the database container will take care of creating the user and db for you):

APP_PORT=4000
APP_HOSTNAME=localhost
DB_USER=postgres
DB_PASSWORD=pass
DB_HOST=database
SECRET_KEY_BASE=Y0uRvErYsecr3TANDL0ngStr1ng

Now you can start your containers:

docker-compose -f docker-compose.yml up

If all is well, you can point your browser to http://localhost:4000 and your application will be there waiting for you.

Now we are ready to write our docker-stack.yml, so that we can deploy our app in production (in a DigitalOcean droplet, on AWS, on your own server, ...):

version: '3.1'

services:

  database:
    image: postgres
    deploy:
      restart_policy:
        condition: on-failure
    environment:
      POSTGRES_USER: ${DB_USER}
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_DB: demo_prod
    networks:
      - backend

  web:
    image: foobar/demo-app:latest
    deploy:
      restart_policy:
        condition: on-failure
    ports:
      - ${APP_PORT}:${APP_PORT}
    environment:
      APP_PORT: ${APP_PORT}
      APP_HOSTNAME: ${APP_HOSTNAME}
      DB_USER: ${DB_USER}
      DB_PASSWORD: ${DB_PASSWORD}
      DB_HOST: ${DB_HOST}
      SECRET_KEY_BASE: ${SECRET_KEY_BASE}
    depends_on:
      - database_migrator
    networks:
      - backend

networks:
  backend:

Before deploying, we need to publish our application to the docker-hub of our choice (in the example above it is published as a fictional foobar/demo-app:latest). Publishing an image to a docker-hub is out of the scope of this article, but if you are here I am positive that you already know how to do it...

Now we have to create an .env-stack for our deploy:

APP_PORT=4000
APP_HOSTNAME=mycoolapp.com
DB_USER=postgres
DB_PASSWORD=pass
DB_HOST=database
SECRET_KEY_BASE=Y0uRvErYsecr3TANDL0ngStr1ng

Finally, after you set up you swarm and connect to it (out of this scope, see the docs on docs.docker.com), you can deploy your app as follows:

source .env-stack
docker stack deploy -c docker-stack.yml demo-app

Given that your swarm is configured to respond to the hostname mycoolapp.com, point your browser to http://mycoolapp.com:4000 and that's all!

Easy, uh? :-)

I'm looking forward for all your constructive (but not only) criticisms and suggestions.

Thanks and to the next.

Top comments (12)

voger • Jan 24 '20

Using bash, during local migrate, I got this error

ERROR! Config provider Config.Reader failed with:
** (ArgumentError) could not fetch environment variable "SECRET_KEY_BASE" because it is not set
 ...

In bash the line

source .env
_build/prod/rel/demo/bin/demo eval Demo.Release.migrate

must be translated to

set -o allexport; source .env; set +o allexport
_build/prod/rel/demo/bin/demo eval Demo.Release.migrate

I am posting this to save time to someone who might stumble upon this problem.

khosimorafo • Feb 26 '20

Thanks for this!

Artur Uklejewski • Nov 22 '19

Hello, i can't make it run. I've got the following error when trying to run eval locally:

** (UndefinedFunctionError) function Ecto.Migrator.with_repo/2 is undefined or private

My endpoint.ex and release.ex are updated with you fragments and application name is changed to mine. Postgres is running on port 5432. Have you got any idea why is that ?

Ivan Iraci • Nov 22 '19

Ecto.Migrator.with_repo/2 was introduced in ecto_sql v3.1.2, maybe you have an older version.

Take a look at your mix.lock.

Eduardo Ribeiro da Silva • Jan 8 '20

I am using those dependencies:

defp deps do
[
{:phoenix, "~> 1.3.4"},
{:phoenix_pubsub, "~> 1.0"},
{:phoenix_ecto, "~> 3.2"},
{:postgrex, ">= 0.0.0"},
{:phoenix_html, "~> 2.10"},
{:phoenix_live_reload, "~> 1.0", only: :dev},
{:gettext, "~> 0.11"},
{:plug_cowboy, "~> 1.0"},
{:comeonin, "~> 4.0"},
{:bcrypt_elixir, "~> 1.0"},
{:guardian, "~> 1.0"},
{:number, "~> 1.0.1"}
]
end

And the error is occurring too.
Any idea?

Ivan Iraci • Jan 9 '20

grep ecto_sql mix.lock

ecto_sql version has to be at least 3.1.2, eg:

"ecto_sql": {:hex, :ecto_sql, "3.1.5", ...
                               ^^^^^

Artur Uklejewski • Nov 22 '19

Yes that was a problem. Thank you

chaitanyapi • May 26 '20

i tried to replicate the above working example to my learning project(which is working fine on localhost including DB activity). however while starting docker-compose, DB container started without any issue, but while the web container is started following error is shown. anyone any idea? please help....

2020-05-26 19:05:51.137 UTC [57] FATAL: database "learn_liveview_dev" does not exist
web_1 | 19:05:51.139 [error] Postgrex.Protocol (#PID<0.2891.0>) failed to connect: ** (Postgrex.Error) FATAL 3D000 (invalid_catalog_name) database "learn_liveview_dev" does not exist

chaitanyapi • May 26 '20

Hi Ivan. this is a very nice demonstration of how to get on and start using docker with phoenix app. i followed through the steps and able to create containers without any errors. while starting the app using the docker compose (with dependency on database), the database container started without any issue and is ready to accept connection. however the web container when it ran the entrypoint.sh, it echos the statement that it is still waiting for database to start. i could not understand this. can you please help me out where could i have probably gone wrong or when i can check to find the issue.
Thank you.