Free & simple CI/CD for Elixir Phoenix app

In this article, I will show the implementation of simple cost- and dependency-free Continuous Integration / Continuous Development (CI/CD) flow to auto-deploy Phoenix app without downtime.

This article is based on Tom Delalande’s YT video with very little of my contribution to make it work with Phoenix application.

---

After good amount of hard work, you finally decide to deploy your app and realize that deploying your app is not an easy task. But, thanks to Docker, the deployment can be simplified a lot.

In this article, I will show how to implement simple ci/cd flow where you can push your code to your repo and have newest version of your Phoenix app re-deployed with 0 downtime and auto-configured SSL. The VM will check the Github repo every minute to see if anything has changed. If yes, new docker image will be built and deployed.

Virtual Machine & DB

Pick any cloud provider you like and spin up a database and virtual server with ubuntu. I am using DigitalOcean (Love it, hate aws and gcp) and PostgreSQL database on it. Make sure you allow connection from your virtual server to your database.

ssh into your virtual machine and set up connection with your Github account so that your VM can clone and pull your private repos. Basically, you will have to generate ssh key on virtual machine, then add the key to your Github account through Setting > Developer settings. Here is the link: https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent

Install Docker using official guide here https://docs.docker.com/engine/install/

Go to your domain provider, and set type A @ (or any subdomain) record to forward to the public IP of your VM.

Create deployment folder that will contain the log file:

cd ~ (in DigitalOcean Ubuntu 20.04 VM, this results to /root)

mkdir deployment (=> /root/deployment)

VM part is mostly set and ready. We will come back to VM later with very small task.

Preparing Phoenix project

Generate release with —docker flag

Inside of your Phoenix project, generate release with Dockerfile by running the following command:

mix phx.gen.release --docker


If you are using NPM packages, then add these lines to install Node and all your dependencies after COPY assets assets.

Lines that handle assets:

COPY assets assets

ENV NVM_DIR=/root/.nvm
ENV NODE_VERSION 20.9.0

RUN curl https://raw.githubusercontent.com/creationix/nvm/v0.39.5/install.sh | bash \
  && . $NVM_DIR/nvm.sh \
  && nvm install $NODE_VERSION \
  && nvm alias default $NODE_VERSION \
  && nvm use default
ENV PATH="/root/.nvm/versions/node/v${NODE_VERSION}/bin/:${PATH}"

RUN mix cmd npm install --prefix assets
RUN mix assets.deploy

Add Caddyfile

We are going to use Caddy as a reverse proxy that can also generate SSL certificates for us. On our VM, Caddy will listen to incoming traffic from the user on ports 80 and 443 (HTTP and HTTPS), then forward that request to port 4000 where we will have our Phoenix running. Phoenix response will then be forwarded thru Caddy to the user. Meanwhile, Caddy will make sure all connection is secure and uses HTTPS.

Create /caddy folder, and Caddyfile inside:

#/caddy/Caddyfile

your_cool_domain.com {
    reverse_proxy server:4000

}


Why server:4000? Phoenix app will have the name server as docker service (see below).

Configure docker-compose file

services:
    server:
        build:
            context: .
            dockerfile: Dockerfile
        env_file:
            - ".env"
    caddy:
        image: "caddy:2.7-alpine"

        restart: unless-stopped
        ports:
            - "80:80"
            - "443:443"
            - "443:443/udp"

        volumes:
            - $PWD/caddy/Caddyfile:/etc/caddy/Caddyfile

add magical ci/cd scripts

Create a /scripts folder and add the following scripts:

deployer.sh

This script makes sure no two deployments are running at the same time, takes cares of logging deployment info, and runs a conditional-deploy scripts.

(Note: /root/deployment folder must already exist. Feel free to change it to whatever you like)

#!/usr/bin/env bash

export DOCKER_CONTEXT=default

LOCK_FILE="$(pwd)/your-app.lock"
cd /root/yetkazuv
flock -n $LOCK_FILE ./scripts/conditional-deploy.sh >>/root/deployment/deploy-app.log 2>&1

conditional-deploy.sh

This script checks if remote repo is different from local and runs server-deploy if necessary

#!/usr/bin/env bash

echo "$(date --utc +%FT%TZ): Fetching remote repository"
git fetch

UPSTREAM=${1:-'@{u}'}
LOCAL=$(git rev-parse @)
REMOTE=$(git rev-parse "$UPSTREAM")
BASE=$(git merge-base @ "$UPSTREAM")

if [ $LOCAL = $REMOTE ]; then
    echo "$(date --utc +%FT%TZ): No changes detected in git"
elif [ $LOCAL = $BASE ]; then
    BUILD_VERSION=$(git rev-parse HEAD)
    echo "$(date --utc +%FT%TZ): Changes detected, deploying new version: $BUILD_VERSION"
    ./scripts/server-deploy.sh
elif [ $REMOTE = $BASE ]; then
    echo "$(date --utc +%FT%TZ): Local changes detected, stashing"
    git stash
    ./scripts/server-deploy.sh
else
    echo "$(date --utc +%FT%TZ): Git is diverged, this is unexpected."
fi

server-deploy.sh

This scripts build a new version of our app inside docker. After new app is built, it then stops old one and substitutes it with new one

#!/usr/bin/env bash

git pull

BUILD_VERSION=$(git rev-parse HEAD)

echo "$(date --utc +%FT%TZ): Releasing new server version. $BUILD_VERSION"
echo "$(date --utc +%FT%TZ): Running build..."
docker compose rm -f
docker compose build

OLD_CONTAINER=$(docker ps -aqf "name=server")
echo "$(date --utc +%FT%TZ): Scaling server up..."
BUILD_VERSION=$BUILD_VERSION docker compose up -d --no-deps --scale server=2 --no-recreate server

sleep 30

echo "$(date --utc +%FT%TZ): Scaling old server down..."
docker container rm -f $OLD_CONTAINER

docker compose up -d --no-deps --scale server=1 --no-recreate server

echo "$(date --utc +%FT%TZ): Reloading caddy..."
CADDY_CONTAINER=$(docker ps -aqf "name=caddy")
docker exec $CADDY_CONTAINER caddy reload -c /etc/caddy/Caddyfile

Commit and push to remote Github repo all new files

Simplest way is:

git add .
git commit -m "add ci/cd after reading this article"

VM: final touches**

ssh into your VM

git clone (SSH method) your project by running

cd into your project folder

create .env file and add all ENV variables you are using while having at least these:

DATABASE_URL=
PHX_SERVER=true
PHX_HOST=your_domain.com
SECRET_KEY_BASE=

run docker compose up to have your app up and running

Now, let’s set a cron job that will run our deployer.sh every minute.

in your VM, run

crontab -e

Then choose the editor you are most comfortable with (i go with VIM), add this line:


* * * * * /root/repo_name/scripts/deployer.sh


Voila, now you can just push your code to Github and your server will pull the updates and re-deploy latest changes.

—

Discussion:

• I strongly recommend the YouTube video by Tom
• You can also deploy your DB, Plausible Analytics, and everything you would like in the same VM as other docker containers
• you probably do not want to store all your ENVs in .env file permanently
• log file can grow infinitely, so it’s a good idea to create cron job with another script that will delete old log files every X amount of time (like couple of weeks, months)

Comments

[Mohammed Zeglam]

sorry for that but using systemd will be a better option. even for performance reason and too easy to write systemd service.

and in work we use gcp and cloud-sql-proxy it is so much faster

[aziz abdullaev]

What part do you think can be substituted by systemd?