Deploying Phoenix via Dokku

Matthew Platts — Sat, 11 May 2019 09:53:43 +0000

Deploy early and deploy often.

I like making MVPs that could potentially turn into a proper startup. Obviously I want to save on costs. Non tech entrepreneurs often end up with a server on Amazon EC2 with an accompanying Postgres instance and end up paying over $100 per month. This is because they've hired a developer to do it who doesn't care about costs.

You should only pay this much when your startup is validated and you have traction. Until then you are essentially in find-product-market-fit mode. It's unlikely many people will be using your site in this mode and you should ideally be on a $5/month VPS. Once you hit traction you can upgrade easy enough - export your DB and upgrade to Amazon or GCP. Phoenix on $5/month will handle a large amount of traffic if your app isn't overly complicated.

I don't really like dev ops work so I'm going to use Dokku, which basically mimics Heroku in that it acts as a PaaS and you just do git remote pushes to deploy your code. I use this for all the projects I build with my Phoenix boilerplate template.

Deploying Phoenix on Vultr

Note that this could apply to any VPS (eg Digital Ocean). I use Vultr because it has an Australian location option.

Create a blank phoenix app using their install guide.

Run the server mix phx.server

Get Ubuntu up and running somewhere in the cloud. For Vultr just register and click the big blue plus button.

Server Location: Pick one closest to you or your target audience.
Server Type: Ubuntu 16.04 (could probably try newer versions - up to you)
Server Size: Might as well start with a $5/month one
Additional Features: All unchecked
Startup script: Leave it empty
SSH Keys: Add your computers ssh key (cat ~/.ssh/id_rsa.pub) to see it. Or look up SSH keys if you don't have one.
Server Hostname & Label: Whatever you want

Once that's installed, click on it and get the IP address and password so we can login.

Mine is 45.76.112.118.

In your terminal login:

ssh root@45.76.112.118

Are you sure you want to continue connecting (yes/no)? yes
root@45.76.112.118's password:

Copy and paste your password.

Now that we're inside the Ubuntu instance we should first ensure all the linux packages are up to date.

sudo apt update
sudo apt upgrade

Now we can install dokku. Make sure you check those docs as the version might be higher than what I'm using.

wget https://raw.githubusercontent.com/dokku/dokku/v0.20.4/bootstrap.sh;
sudo DOKKU_TAG=v0.20.4 bash bootstrap.sh

Now in your browser copy and paste your server's IP address into the address bar and follow the web installer.

Just add your ssh key like before - cat ~/.ssh/id_rsa.pub. And I just leave the rest as defaults. Click finish.

Now that Dokku is installed we can start using its command line interface (CLI) to setup a database and link it to our app.

Still on your remote server run these commands (note that I'm calling my app card-tracker).

# Create a dokku app called card-tracker
dokku apps:create card-tracker

# Dokku has lots of plugins - this one helps us create a postgres db
dokku plugin:install https://github.com/dokku/dokku-postgres.git

# Use the pluging to create a db instance called 'db'
dokku postgres:create db

# Linking just creates a global ENV config variable in card-tracker called 'DATABASE_URL`
dokku postgres:link db card-tracker

Your dokku app has settable global ENV config variables. You use these to set stuff you don't want to commit into git - important stuff like your database credentials, third party api keys, etc.

You can check the currently set ENV vars with dokku config:export card-tracker. There should be only one set - DATABASE_URL. This was set when you ran dokku postgres:link db card-tracker.

Go back to your phoenix app codebase and look at the file prod.secret.exs:

We can see Phoenix by default will look for an ENV var called DATABASE_URL and set up your database to use it for the credentials. Handy.

The Endpoint config is oddly split over prod.exs and prod.secret.exs (when you write config :app, AppWeb.Endpoint, blah, it's just adding to the existing config, not overwriting it).

I ended up deleting the Endpoint config in prod.exs and putting it completely in prod.secret.exs for simplicity.

# I deleted this from prod.exs:
config :app, AppWeb.Endpoint,
  url: [host: "example.com", port: 80],
  cache_static_manifest: "priv/static/cache_manifest.json"

And in the file prod.secret.exs:

config :app, AppWeb.Endpoint,
  cache_static_manifest: "priv/static/cache_manifest.json",
  http: [port: String.to_integer(System.get_env("PORT") || "4000")],
  url: [
    scheme: "https",
    host: System.get_env("WEB_HOST"),
    port: 443
  ],
  force_ssl: [rewrite_on: [:x_forwarded_proto]],
  secret_key_base: secret_key_base

Now back to our remote server we can set the WEB_HOST global ENV variable:

dokku config:set --no-restart card-tracker WEB_HOST=45.76.112.118

Also looking back at prod.secret.exs down the bottom:

secret_key_base =
  System.get_env("SECRET_KEY_BASE") ||
    raise """
    environment variable SECRET_KEY_BASE is missing.
    You can generate one by calling: mix phx.gen.secret
    """

config :app, AppWeb.Endpoint,
  http: [:inet6, port: String.to_integer(System.get_env("PORT") || "4000")],
  secret_key_base: secret_key_base

We should generate a SECRET_KEY_BASE by running mix phx.gen.secret. Do that in your console on your local computer (not remote).

Then set it on the remote:

dokku config:set --no-restart card-tracker SECRET_KEY_BASE="TGISHf+6hJiQgEuroRY29k8IWnmY9MzggnPY86x16AYkJnMoPZDBcRuVgiUkT/Zu"

Now if you run dokku config:export card-tracker again you should have ENV vars for:

DATABASE_URL
SECRET_KEY_BASE
WEB_HOST

Now go back to local terminal inside your phoenix app. We need to add our server as a remote to push to.

# Add a remote called dokku. In the dokku setup it created a user called dokku
git remote add dokku dokku@45.76.112.118:card-tracker

We're still not quite ready to deploy though. Like I said Dokku is like Heroku - and even Heroku doesn't support all languages and frameworks. Instead it relies on "buildpacks", which are basically install scripts for different environments. There is one for Phoenix. If you are serving static content (like JS/CSS) then you also need a Phoenix static buildpack.

There are a couple of ways to tell Dokku what buildpacks to use, but we'll use the method where you create a .buildpacks file in the root of your Phoenix project.

In you .buildpacks file add:

https://github.com/HashNuke/heroku-buildpack-elixir.git
https://github.com/gjaldon/heroku-buildpack-phoenix-static

The elixir buildpack allows you to set the elixir and erlang versions. To do this create another file called elixir_buildpack.config in the root of your Phoenix project and add the contents:

elixir_version=1.9.0
erlang_version=21.1.1

The versions above worked for me as of 23 July 2019 - maybe google the latest or check yours with brew info erlang and brew info elixir. Although I found if I used the latest versions the deploy process failed. So if you get random deploy fails then move the versions back until it works.

Finally, create another file called Procfile in your root directory. Inside it add:

web: ./.platform_tools/elixir/bin/mix phx.server

Read up on Procfiles here. They are a Heroku thing that Dokku also honours. I'm not really sure why we need to add this but I found it in a forum somewhere because without it my server wasn't working. The elixir buildpack says this "If your app doesn't have a Procfile, default web task mix run --no-halt will be run." So maybe that mix run --no-halt isn't good enough anymore.

Anyway, commit those 3 files and then let's try deploying!

git add -A
git commit -m 'Add dokku config files'
git push dokku master

It will print out a bunch of install logs - might take a while. It should finish with something like:

=====> Application deployed:
       http://45.76.112.118:31894

Paste that URL into your browser (or CMD-click it if in iterm2) and you should see your app online.

Migrations

By default our elixir buildpack doesn't run Phoenix migrations. To run them, we'll need to utilise Dokkus post deploy hook. To do this we add another file in the root directory of our Phoenix folder called app.json:

{
  "scripts": {
    "dokku": {
      "postdeploy": "mix ecto.migrate"
    }
  }
}

Commit that and deploy and you should see that command being run in the logs.

Domains

By default Dokku is just using the IP address and using ports to show your app to the world.

You could add another app dokku apps:create blah and it would just live on another port.

http://45.76.112.118:31894 => card-tracker
http://45.76.112.118:32939 => blah

Anyway I registered cardtracker.com.au on Godaddy and I'd like this domain to point to my IP address.

To do that I go into DNS settings in Godaddy and make sure there are no A records except for one:

Type: A
Name: @
Value: 45.76.112.118

This means cardtracker.com.au is now pointing to my remote server. Since I want www to also point there I can create a cname record to point to my root record:

Type: CNAME
Name: www
Value: @

Now on the remote server add the domain to Dokku:

dokku domains:add card-tracker cardtracker.com.au
dokku domains:add card-tracker www.cardtracker.com.au

Wait a bit of time for the changes to propagate. You can check what's happening in the terminal with host -a cardtracker.com.au.

Trying "cardtracker.com.au"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 20867
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;cardtracker.com.au.        IN  ANY

;; ANSWER SECTION:
cardtracker.com.au. 599 IN  A   45.76.112.118
cardtracker.com.au. 3599    IN  NS  ns69.domaincontrol.com.
cardtracker.com.au. 3599    IN  NS  ns70.domaincontrol.com.
cardtracker.com.au. 3599    IN  SOA ns69.domaincontrol.com. dns.jomax.net. 2019050807 28800 7200 604800 600

Remember to update the WEB_HOST environment variable with your new host.

dokku config:set card-tracker WEB_HOST=cardtracker.com.au

Redirect www to root

When someone hits www.cardtracker.com.au we want the response to be a 301 redirect to cardtracker.com.au. We need a server to be able to give this response - luckily Dokku has a plugin to manage this for us.

dokku plugin:install https://github.com/dokku/dokku-redirect.git
dokku redirect:set card-tracker www.cardtracker.com.au cardtracker.com.au

SSL

SSL is the norm these days. Dokku makes it easy with the dokku-letsencrypt plugin:

dokku plugin:install https://github.com/dokku/dokku-letsencrypt.git
dokku config:set --no-restart card-tracker DOKKU_LETSENCRYPT_EMAIL=your_email@mail.com

# Free SSL for 90 days
dokku letsencrypt:enable card-tracker

# Add a monthly CRON job to refresh your free SSL certificate each month
dokku letsencrypt:cron-job --add

# View your SSL status
dokku letsencrypt:ls

You should now be able to view your site with https. Make sure you check in Chrome incognito - mine wasn't working on my normal Chrome, probably due to some caching issue.

Reading the database

You can use any database tool to connect to your remote database. I use Table Plus.

Run these commands:

dokku postgres:expose db
dokku postgres:info db

In the output you can find these lines:

Exposed ports:       5432->24321  
postgres://postgres:xxxxxxxxxxxxxxxx@dokku-postgres-db:5432/db

This is in the format:

postgres://USERNAME:PASSWORD@dokku-postgres-db:5432/DATABASE_NAME

So overall we have the information:

IP Address: the IP address of your server
Port: 24321
Username: postgres
PW: xxxxxxxxxxxxxxxx
Database name: db

Just plug that into your database app and you're away.

Backing up the database to S3

Sign up to Amazon.
Create a bucket "card-tracker-backups" - no public access.
Go to IAM and create a new user with programmatic access and full access to S3.
Grab the access key and secret.

dokku postgres:backup-auth db AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY

Test backing up the db service to the BUCKET_NAME bucket on AWS.

dokku postgres:backup db BUCKET_NAME

=> 2021-05-06-00-59-06: The backup for xxx finished successfully.

Schedule a backup. CRON_SCHEDULE is a crontab expression, eg. "0 3 * * *" for each day at 3am

dokku postgres:backup-schedule db "0 3 * * *" BUCKET_NAME

File uploads

I found that if you allow file upload then you need to increase the file upload max limit in the nginx configuration. You can do it by sshing into the server and running:

echo 'client_max_body_size 50m;' > /home/dokku/<APP NAME>/nginx.conf.d/upload.conf

dokku ps:restart <APP NAME>

So for me:

echo 'client_max_body_size 50m;' > /home/dokku/card-tracker/nginx.conf.d/upload.conf

dokku ps:restart card-tracker

Security

Note: do the following at your own risk. I recommend researching security yourself instead of trusting this.

Check out VPS Harden, which will make your server much harder to crack for hackers.

I ran the install command on the remote server:

sudo git clone https://github.com/akcryptoguy/vps-harden.git && cd vps-harden && sudo bash get-hard.sh

When asked I created a non-root user and disabled password login and root access. I kept the SSH port on 22 though as I was worried it might interfere with pushing code to dokku.

Afterwards I changed to my new login su - newuser.
Then I added my ssh public key:

mkdir ~/.ssh && touch ~/.ssh/authorized_keys
sudo chmod 700 ~/.ssh && chmod 600 ~/.ssh/authorized_keys
sudo vim ~/.ssh/authorized_keys

You can get your ssh key with cat ~/.ssh/id_rsa.pub locally.

Then you run this to restart ssh:

sudo systemctl restart sshd

Keep this tab open and now try and ssh in from your local computer. ssh newuser@ipaddress. If this fails, keep troubleshooting until you sure you can login... since if you close the other open tab you've lost access to the server forever thanks to you disabling password login.

Once you can ssh into your newuser then you can just swap back to the root user with su - and run dokku commands.

Server monitoring

If you want to monitor your servers with netadata: https://www.vultr.com/docs/installing-netdata-on-linux-multiple-distros

DEV Community: Matthew Platts

Built a community sharing platform in my isolation to help with covid19