DEV Community: Alastair Measures

Dockerizing Phoenix Development (6) Installing Phoenix

Alastair Measures — Sun, 30 May 2021 18:51:43 +0000

The database is on tap, the elixir image is ready and sharp observers, on their third espresso, will notice: Phoenix is still not present!

To get Phoenix installed we need to:

have the containers running
sign in to the web container as devuser
install the Erlang and Elixir package managers
install the Phoenix package (and dependencies).

These things go into hidden folders within the (container) /apps folder because it is the home folder for the devuser account used for development.

After this, the standard mix commands can be used to initiate projects, run migrations, run the web servers and so on.
From this point, the standard text books for Phoenix 1.5 (e.g. McCord, Tate & Valim) can be followed without problems.

Installing

Start the containers:

From the command line:

docker-compose  up  -d

Connect to the web container shell

From the command line:

docker exec  -u devuser  -ti  myp_dev_web_1  /bin/ash

Install Erlang package manager called Rebar

mix local.rebar --force

Install Elixir package manager called Hex

mix local.hex --force

Install Phoenix tool packages

mix archive.install hex phx_new 1.6.10 --force

By the time you read this: version 1.6.10 may no longer be ideal so review what is the latest stable edition: https://hexdocs.pm/phoenix/Phoenix.html.

Run up a fresh Phoenix project

mix phx.new myp

All being well, your project is now rolling and you will need to adjust the database configuration for the project.

The database configuration should reflect the database user (myp_user) and database instance (myp_dev) created earlier.
The database configuration should also reflect that the database host should be db (the service name from the docker-compose.yml file) rather than localhost!

The Elixir driver for PostgreSQL is postgrex and there are some relevant extensions to consider at this point:

Remarks

This tutorial has been run through to test that it works. Any problems, leave a comment and I will respond.
This tutorial works through a few awkward challenges; these solutions may have saved you some time. If you are feeling generous, you could buy me a coffee!

Dockerizing Phoenix Development (5) Scripting the images

Alastair Measures — Sun, 30 May 2021 18:50:07 +0000

This extends the docker-compose.yml that appeared in step 3.

The Dockerfile (./myp_dev/docker-compose.yml)

version: "3"

volumes:
  postgis13_data:

networks:
  myp_net:

services:

  db:
    image: postgis/postgis:13-3.1-alpine
    networks:
      - myp_net
    environment:
#     - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
#     - POSTGRES_DB=postgres
    ports:
      - 5432:5432
    volumes:
      - postgis13_data:/var/lib/postgresql/data/

  web: 
    build:
      context: .
      dockerfile: Dockerfile.alpine
    networks:
      - myp_net
    ports:
      - 4000:4000
    depends_on: 
      - db
    volumes:
      - ./source:/apps

# end

As docker-compose scripts go: this is about as simple as it gets. Even someone with a passing awareness of docker should be able to read this.

Build image from Dockerfile(s)

docker-compose --build

Remarks

The system defined by this script is unlikely to conflict with other running systems; other than where it offers service on ports exposed onto the localhost.

Dockerizing Phoenix Development (4) The Dockerfile

Alastair Measures — Sun, 30 May 2021 18:49:14 +0000

This step assembles a Dockerfile in order to develop a Phoenix web server. It lists the Dockerfile and then explains key lines. You will probably need to make some adjustments for your own use, so check the explanations.

The Dockerfile (./myp_dev/Dockerfile.alpine)

FROM elixir:1.12-alpine

LABEL maintainer="me_myself_i@work"
LABEL version_date="2021-05-20"

RUN apk add --no-cache --update \
        git \
        inotify-tools \
        libpq postgresql-client \
        nodejs npm yarn

ENV  USER="devuser"  UID="1000"  GID="1000"

RUN addgroup  -g $GID  -S $USER  && \
    adduser  -h /apps  -g ""  -G $USER  -S  -D  -u $UID "$USER"

USER $UID:$GID

ENV PATH="/usr/local/lib/elixir/bin:/usr/local/lib/erlang/bin:$PATH"
ENV PATH="/apps/.cache/rebar3/bin:$PATH"

VOLUME /apps
WORKDIR /apps
EXPOSE 4000/tcp

CMD sleep 50000

Lines Explained

FROM elixir:1.12-alpine

Note this means use of the apk package manager for Alpine Linux rather than the apt utility used with Debian and Ubuntu. There is a full searchable library of packages available for Alpine Linux.
By the time you read this, later editions may be available so adjust as necessary.

RUN apk add --no-cache --update \
        git \
        inotify-tools \
        libpq postgresql-client \
        nodejs npm yarn

Packages as follows:

git is useful for automated download of libraries, as well as uploading code to github et cetera.
inotify-tools adds into the container kernel so that changes to files can be detected by a running program within the container.
libpq is the shared object/ DLL for access to PostgreSQL.
postgresql-client adds in the PostgreSQL command line tools such as psql. This is not strictly needed and adds ~9MB to the image, but it is convenient to have on hand.
nodejs Javascript interpreter.
npm & yarn package managers for JavaScript.

ENV  USER="devuser"  UID="1000"  GID="1000"

Set some environment variables for use in setting up a user account for you as developer.
Matching the UID & GID between your host user account and the developer account in the image is key to using IDE's and other tools. You MUST use your own UID and GID here!

RUN addgroup  -g $GID  -S $USER && \
RUN adduser  -h /apps  -g ""  -G $USER  -S  -D  -u $UID "$USER"

These lines add a new group and a new user within that group.
Alpine Linux uses the BusyBox tools to do this; which are different to the Debian/ Ubuntu ones! Documentation is available.

USER $UID:$GID

The remaining few lines are completed as the devuser. So signing in as devuser will have the $PATH, and start in the WORKDIR that you would expect.

ENV PATH="/usr/local/lib/elixir/bin:/usr/local/lib/erlang/bin:$PATH"
ENV PATH="/apps/.cache/rebar3/bin:$PATH"

Adds Elixir and Erlang into the $PATH.

VOLUME /apps
WORKDIR /apps

The /apps/ folder within the container will be the $HOME folder of the devuser account.
The same folder will also be mapped as a volume to the ./myp_dev/source/ folder of the host user account (configured via the docker-compose.yml).

EXPOSE 4000/tcp

This allows the default Phoenix web server port to be mapped to a known port number of the host machine. Precisely which port number is configured in the docker-compose.yml.

CMD sleep 50000

Once a container runs, it wants to run to completion and then shutdown in an orderly fashion. We need it to hang around so we can attach a console and carry out our development work!

Remarks

An equivalent Dockerfile based on Debian or Ubuntu can readily be put together. Note however that the image will be near to twice the size(!).
The inotify-tools capability will pick up changes originating inside the container, but may not notice changes originating outside the container (e.g. your IDE). It does however need to be present as Phoenix squeals if it isn't.

Dockerizing Phoenix Development (3) PostgreSQL + PostGIS

Alastair Measures — Sun, 30 May 2021 18:47:49 +0000

This part is intended to pull together a running PostgreSQL DBMS, with the PostGIS extensions on hand, as a container co-ordinated by docker-compose.

Approach

This involves these parts:

Selecting an image from Docker Hub,
Figuring out settings for this database image,
Assembling a 'docker-compose.yml' file
Run the container via 'docker-compose'
- Establish a database instance & grant permissions
- Install PostGIS into a separate SQL schema

Selecting an image from Docker Hub

Visit https://hub.docker.com and look around for a reasonable PostgreSQL image that includes PostGIS extensions. One choice that is current (June 2022): postgis/postgis:14-3.2-alpine.

If you really don't want or need PostGIS extensions, you could use: postgres/postgres:14-alpine

Figuring out settings for database image

https://registry.hub.docker.com/_/postgres/ provides good documentation, in the ‘Overview’ tab, and allows you to look at the Dockerfile that generated the postgis image.

Assembling a 'docker-compose.yml' file

This version will be extended in a later part of this series to include the Phoenix web server container.

version: "3"

volumes:
  postgis14_data:

services:

  db:
    image: postgis/postgis:14-3.2-alpine
    ports:
      - 5432:5432
    environment:
#     - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
#     - POSTGRES_DB=postgres
    volumes:
      - postgis14_data:/var/lib/postgresql/data/

Note: the port 5432 being exposed allows access from tools on the host machine such as, say, BeeKeeper Studio.

Run the container

docker-compose  up  -d

Note the -d flag which detaches the action from the terminal and lets you have the command prompt back.

Establish a fresh database instance & grant permissions

Connect to container as 'root' user:

docker  exec  -it myp_dev_db_1  /bin/ash

You should now have a shell prompt within the container.
Note the use of 'ash' command shell rather than the 'bash' command shell.

Within the container go into 'psql' as 'postgres':

psql   -h localhost   -d postgres   -U postgres

You should now have a SQL command prompt.

Create a fresh user and database instance for development:

CREATE USER myp_user WITH ENCRYPTED PASSWORD 'Drowssap1' ;
CREATE DATABASE myp_dev ;
GRANT ALL PRIVILEGES ON DATABASE myp_dev TO postgres ;
\q

Note: On some GUI tools: if you try to paste these statements in together, they will be handled as a transaction which will then fail/ rollback from part way. So probably best to do these statements one at a time!

Enable PostGIS into a separate SQL schema

Connect into 'myp_dev' database as 'postgres':

psql   -h localhost   -d myp_dev   -U postgres

You should have a SQL command program at this point.

Establish the 'postgis' extension into it's own SQL schema:

CREATE SCHEMA postgis ;
ALTER DATABASE myp_dev SET search_path = postgis,public ;
\q

Re enter database for changed search_path to take effect:

CREATE EXTENSION postgis;
ALTER DATABASE myp_dev SET search_path = public,postgis ;
\q

Note that 'postgis' items are installed into the first schema in the 'search_path'.
There are further PostGIS related extensions - see https://www.postgis.net for more details.

Close things

Drop the container command line:

exit

Shutdown the running container from the host command line:

docker-compose  down

Note this needs you to be in the same folder as when the 'up' command was issued.

Remarks

This setup is for development only; testing and deployment will each be different.
Some would develop while working as the 'postgres' user (with postgres administrator rights), and it can be convenient. Although everything here is within a sandbox (aka container): developing as 'root' user or database administrator is avoided as being bad practice.
Good practice here includes ensuring that the 'postgis' tables, indexes, functions, etc are created into their own SQL schema rather than being mixed in with the rest of your application tables.
Docker-compose does some automated naming of containers, networks and volumes that seems really intuitive and useful!

Dockerizing Phoenix Development (2) Folders & Files

Alastair Measures — Sun, 30 May 2021 18:45:42 +0000

The overall objective is to construct a development environment within Docker for a website based on Phoenix and Elixir backed by PostgreSQL with PostGIS.

We are working through 'docker-compose', with project parameters in an explicit, indented YAML file rather than a bash script.

This step is intended to set out the organisation of files and folders being used, and explain each.

Hierarchy

./base_folder/ (folder - might be $HOME)
- /myp_dev/ (folder of entire project)
  - /docker-compose.yml (text file scripting images to run together)
  - /Dockerfile.alpine (text file defining image for web server)
  - /source/ (folder where your Elixir code goes)

./myp_dev

This is a folder that encapsulates our whole exercise. The docker-compose commands should be issued from this folder.
Note: when docker-compose starts our collection of containers: it names them including this folder name as a prefix. This feature is helpful and makes it easier to run, say, a beta version alongside a development version for comparison purposes.

./myp_dev/docker-compose.yml

This is a text file; which scripts the necessary images required to bring up a group of containers as a system.

./myp_dev/Dockerfile.alpine

This is a text file, we use to build an image for Elixir and Phoenix.

./myp_dev/source

This folder will contain all our Elixir and Phoenix coding. This folder will be mounted as a volume into the container. In this way we can run our web server inside the container, and, at the same time, use editors and IDE software on the host system (from outside the container).

Remarks

By default: the user inside the container is 'root' (for that container) and will not let you edit files created if you are anything less!
To make this work smoothly, the 'user id' and 'group id' used for development within the container will need to match those of the development user on the host machine.
This exercise has been assembled under Linux and will very probably work hosted on a Mac. Windows, however, formats text files differently and trying to use the same folders from a container (Linux) and the host (Windows) is likely to give you gremlins! (End of line and end of file are coded differently). Some IDE software such as KomodoIDE may have preference settings that help, however it remains a courageous practice.

Dockerizing Phoenix Development (1) Introduction

Alastair Measures — Sun, 30 May 2021 18:45:22 +0000

The overall objective here is to construct a development environment within Docker for a website based on: Phoenix and Elixir backed by PostgreSQL with PostGIS.

The plan is to have the Elixir and Phoenix work entirely within the container, even allowing projects to be started from scratch; thereby removing the need to have Elixir and Phoenix installed on the host machine.

The intention here is that these steps will get a relative newcomer up and running: with some understanding. It is written to be relatively gentle on the reader, if a bit slow for some.

For those not yet familiar: PostGIS is a group of extensions for PostgreSQL which include geographical capabilities that are both sophisticated and standards compliant .

Approach

Using Docker so that the project:

can be reproduced on demand for colleagues or others.
is largely prepared for deployment through Docker.
can co-exist with other projects using different version permutations.

Choosing/ constructing Docker images so that they are lean and targetted to the purpose in hand.

To enable IDEs and other tools (on the host machine) to work with source code inside a container.

Using 'docker-compose' so that parameters are scripted in the YAML format for clarity and simplicity.

Prerequisites and Platforms

This exercise was constructed and tested on the following platform:

Ubuntu 20.04 LTS with Docker and Docker Compose.
There are a couple of ways of installing docker on Ubuntu; the recommended one is documented on the Docker website
Reproducing this exercise should work just fine on Linux and Mac platforms. Windows however, would be tricky for reasons explained in step 2.

Recommended Extras

It is good to have a graphical view into what is going on within the docker server. Installing Portainer Community Edition as a container is a good option, allowing interaction through a browser.
Given that a database is involved it is also good to have a GUI based SQL tool on hand. BeeKeeper Studio is one option (also available among the snap applications on Ubuntu).

Remarks

This overall exercise reflects lessons learned the hard way by the author - a journey that took time. This route works; however there may be better techniques - it is humbly requested that such wisdom is shared constructively in the comments.
You are expected to use this as a kind of template. Use your own project acronym, use later editions of PostgreSQL, PostGIS, Elixir and/ or Phoenix; as you need.
If you don't need PostGIS: you can either leave it in or figure out which parts to adjust as you go through.