Rails CD with Docker, Github Actions and VPS

I want to show you how to deploy your app to production with minimal cost and make the deployment process fully automated. If you have never done it before, this post will show you how to achieve it step by step. Maybe you have already deployed some apps, then you know that there are always some problems, especially when the server is used by multiple applications. This approach isn’t something innovative, there are many blog posts where you can learn how to dockerize apps, how to use GitHub Actions, and how to deploy code to VPS, but this tutorial brings it all together.

Docker

The whole idea is based on Docker's image. So the first thing to do is Docker installation. You can skip that part if you have already installed it.

Install Docker

For more details check the official site.

# Update the apt package index and install packages to allow apt to use a repository over HTTPS:
$ sudo apt-get update && apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common

# Add Docker’s official GPG key
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

# Update the apt package index and install the latest version of Docker Engine and Containerd
$ sudo apt-get update && apt-get install docker-ce docker-ce-cli containerd.io

# Verify that Docker Engine is installed correctly by running the hello-world image
$ sudo docker run hello-world

Manage Docker as a non-root user

For more details check official site

# Create the docker group
$ sudo groupadd docker

# Add your user to the docker group
$ sudo usermod -aG docker $USER

# Activate the changes to groups (only Linux)
$ newgrp docker

# Verify that you can run docker commands without sudo
$ docker run hello-world

Dockerfile

Docker is creating images using Dockerfile - it's a file with all commands that are executed during the build. I will show you the simplest version that will work. Later I will improve it and shorten the build time. Create a file Dockerfile in the main app directory.

#1 This is the official Ruby image (https://hub.docker.com/_/ruby) - a complete Linux system with Ruby installed
FROM ruby:3.0.1

#2 Install applications needed for building Rails app
RUN apt-get update && apt-get install -y \
 build-essential libpq-dev nodejs zlib1g-dev liblzma-dev

#3 The WORKDIR instruction sets the working directory for any RUN, CMD, ENTRYPOINT, COPY and ADD
# If a directory doesn’t exist, it will be created
WORKDIR /app

#4 Copy files from current location to image WORKDIR
COPY . . 

#5 Install gems in the image
RUN bundle install

#6 Command that will be executed when you run the image
CMD bundle exec rails s -p 3000 -b '0.0.0.0'

Now let’s test it and create an image with the name rails_app.

$ docker build -t rails_app .
Sending build context to Docker daemon  86.65MB
Step 1/6 : FROM ruby:3.0.1
3.0.1: Pulling from library/ruby
d960726af2be: Pull complete 
## part ommited
Status: Downloaded newer image for ruby:3.0.1
 ---> 9cba361e78fe
Step 2/6 : RUN apt-get update && apt-get install -y  build-essential libpq-dev nodejs zlib1g-dev liblzma-dev
 ---> Running in fa0bce0b6b81
Get:1 http://deb.debian.org/debian buster InRelease [121 kB]
## part ommited
Removing intermediate container 40b752bd0ef3
 ---> 7d09aa5c9ced
Step 3/6 : WORKDIR /app
 ---> Running in 427dea58acb0
Removing intermediate container 427dea58acb0
 ---> 8ed87d4b0643
Step 4/6 : COPY . .
 ---> 0b3a695a0987
Step 5/6 : RUN bundle install
 ---> Running in 65a2592eca90
Fetching gem metadata from https://rubygems.org/............
Fetching rake 13.0.3
Installing rake 13.0.3
## part ommited
Removing intermediate container 65a2592eca90
 ---> 55d9368c4b98
Step 6/6 : CMD bundle exec rails s -p 3000 -b '0.0.0.0'
 ---> Running in 795356f8553e
Removing intermediate container 795356f8553e
 ---> 2466c41ac676
Successfully built 2466c41ac676
Successfully tagged rails_app:latest

The image is successfully built, to check available images you can use this command

$ docker images

Now it's time to run the container with the application and check if it works.

#-p parm allows to map ports with scheme EXPOSED_PORT:IMAGE_INTERNAL_PORT
$ docker run -p 3001:3000 rails_app

Open the browser and go to http://localhost:3001/ - there is a little success, Rails application is working partially:

This is an error from Rails, so Rails is working. Still, there is a problem with the database. There must be another container with the Postgres application and connection between these containers. To achieve it I will use Docker Compose.

Docker Compose

This is a tool that allows to run multiple containers and create a network between them. The configuration file is stored as YAML.

Install Docker Compose

For more details check official site

# Download the current stable release of Docker Compose
# To install a different version of Compose, substitute 1.29.2 with the version of Compose you want to use.
$ sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

# Apply executable permissions to the binary
$ sudo chmod +x /usr/local/bin/docker-compose

# Test the installation
$ docker-compose --version

Compose config file

Create a file docker-compose.yaml in the main app directory.

version: "3"
services:
  database:
    # Official postgres image available in https://hub.docker.com/
    image: postgres
    # There are many types of volumes, this is a named volume, which will store database in docker directory
    # Named volumes must be listed under the top-level volumes key, as shown at bottom of the file
    volumes:
      - db_data:/var/lib/postgresql/data
    environment:
      - POSTGRES_PASSWORD=password
  web:
    image: rails_app
    # Command will replace CMD from Dockerfile
    command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
    # Path on the host, relative to the Compose file. 'app' is a WORKDIR name from Dockerfile
    # This volume will allow you to run the Rails app with Docker Compose
    # and made live changes without rebuilding the image
    volumes:
      - .:/app
    ports:
      - "3001:3000"
    # 'database' is Postgres service name from the top of the file - it will allow communication between containers
    depends_on:
      - database
    environment:
      - POSTGRES_PASSWORD=password
      - POSTGRES_USERNAME=postgres
      - POSTGRES_HOST=database  # it's Postgres service name from the top of the file
volumes:
  db_data:

Now it's time to run Rails application and Postgres database with Docker Compose, but before you must update Rails database config file, create a database, and run migrations.

#config/database.yaml

default: &default
  adapter: postgresql
  encoding: unicode
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
  username: <%= ENV['POSTGRES_USERNAME'] %>
  password: <%= ENV['POSTGRES_PASSWORD'] %>
  host: <%= ENV['POSTGRES_HOST'] %>

development:
  <<: *default
  database: rails_app_development

test:
  <<: *default
  database: rails_app_test

production:
  <<: *default
  database: rails_app_production

After updating the database file Docker image needs to be rebuild.

$ docker build -t rails_app .

The first 3 steps are cached, but changes in the application directory cause gems installation. I will show you later how to avoid it and use cache.

Now start containers and in another terminal window run a command to create a database.

# Run command and leave it running
$ docker-compose up
# From another terminal window
$ docker-compose run web rake db:create db:migrate

Open the browser and go to http://localhost:3001/ and... You just run the Rails app with Docker.

VPS

The next piece of the puzzle is VPS - a place where you deploy application. You can find many companies that provide cloud services and it's your decision which one you choose. I wanna show you an example based on a server with Ubuntu. Like on your localhost, firstly you install Docker and Docker Compose on VPS. Use steps from the beginning of this post. You will need two additional non-root users: nginx_proxy and rails_app.

$ sudo adduser nginx_proxy
$ sudo adduser rails_app
# Add new users to the docker group
$ sudo usermod -aG docker nginx_proxy
$ sudo usermod -aG docker rails_app

HTTP server

For HTTP server I will use NGINX with this awesome application nginx-proxy and acme-companion for automatic SSL certificate generation. Connect to the server as nginx_proxy user and create two files docker-compose.yaml and nginx_custom.conf.

$ cd ~ && touch docker-compose.yaml nginx_custom.conf

I will show you the basic configuration of these two applications. For more details check the app's documentation from the links above.

# docker-compose.yaml
version: '3.9'

services:
  nginx-proxy:
    restart: always
    image: nginxproxy/nginx-proxy
    container_name: nginx-proxy
    ports:
      - 80:80
      - 443:443
    volumes:
      - conf:/etc/nginx/conf.d
      - vhost:/etc/nginx/vhost.d
      - html:/usr/share/nginx/html
      - dhparam:/etc/nginx/dhparam
      - certs:/etc/nginx/certs:ro
      - /var/run/docker.sock:/tmp/docker.sock:ro
      - ./nginx_custom.conf:/etc/nginx/conf.d/nginx_custom.conf
    networks:
      nginx-proxy-network:

  letsencrypt:
    restart: always
    image: nginxproxy/acme-companion
    container_name: nginx-proxy-acme
    volumes_from:
      - nginx-proxy
    volumes:
      - certs:/etc/nginx/certs:rw
      - acme:/etc/acme.sh
      - /var/run/docker.sock:/var/run/docker.sock:ro
    networks:
      nginx-proxy-network:

volumes:
  conf:
  vhost:
  html:
  dhparam:
  certs:
  acme:

networks:
  nginx-proxy-network:
    name: "nginx-proxy-network"

# nginx_custom.conf
# here you can customize NGINX
server_tokens off;
client_max_body_size 100m;

When these two files are created and filled with content, let's run the NGINX server with deamon (-d param)

$ docker-compose up -d

And that's all you need to do with the HTTP server - this app will handle all new Rails applications on your server with few ENV variables that you will add to the Rails app docker-compose files.

Rails app - Production Docker Compose file

Let's connect to the server as rails_app user. You must create two files docker-compose.yaml and .env on the server and copy the below content to these files.

$ cd ~ && touch docker-compose.yaml .env

In the production version you must pass more ENV variables, so let's create a file to store these variables separately. Also, you must remember that every file created in the Docker image during the app life cycle will be deleted with the new app version release. So e.g. files from ActiveStorage or logs need to be stored outside of the image.

version: "3"
services:
  database:
    # restart docker container when there will be a crash
    restart: always
    image: postgres
    volumes:
      - db_data:/var/lib/postgresql/data
    # instead of environment let's use the env file
    env_file: .env
  web:
    restart: always
    image: rails_app
    command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
    env_file: .env
    environment:
      - VIRTUAL_HOST=your_dns_for_rails_app.com # it will allow nginx-proxy to redirect HTTP request to your Rails app
      # LETSENCRYPT variables are used by acme-companion and it will create SSL certificate for those params
      - LETSENCRYPT_HOST=your_dns_for_rails_app.com
      - LETSENCRYPT_EMAIL=some_user@your_dns_for_rails_app.com
    volumes:
      - ./storage:/app/storage # store ActiveStorage files in `storage` directory
      - ./log:/app/log # store logs in `log` directory
    ports:
      - 3001:3000
    depends_on:
      - database
volumes:
  db_data:
networks:
  default:
    external:
      name: nginx-proxy-network

Example env file:

POSTGRES_PASSWORD=password
POSTGRES_USERNAME=postgres
POSTGRES_HOST=database
RAILS_ENV=production
SECRET_KEY_BASE=some_secret_key
RAILS_LOG_TO_STDOUT=true
RAILS_SERVE_STATIC_FILES=true

Rails app - Production Dockerfile

The main difference is a need to precompile assets to run the production environment. To do it with Rails and Webpacker, then also Yarn is needed. Let’s update Dockerfile to handle it and fix gems caching.

FROM ruby:3.0.1
# add yarn to apt list
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - \
  && echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
# add yarn to installed apps
RUN apt-get update && apt-get install -y \
 build-essential libpq-dev nodejs zlib1g-dev liblzma-dev yarn
WORKDIR /app
# copy Gemfile and Gemfile.lock and install gems before copying rest of the application
# so the steps will be cached until there won't be any changes in Gemfile
COPY Gemfile* ./
RUN bundle install
COPY . .
# precompile assets with temporary secret key base
RUN SECRET_KEY_BASE="assets_compile" RAILS_ENV=production bundle exec rake assets:precompile
CMD bundle exec rails s -p 3000 -b '0.0.0.0'

GitHub Actions

When the production Rails app on Docker image is fully working and VPS is ready, it's time to create an image with GitHub Actions and store it in GitHub Container Registry. Before I show you the config file, there are few things to do in GitHub.

• GitHub Container Registry (GHCR) is in an experimental state, so you must enable that feature with this tutorial.
• Second thing needed is token, which allow to login to GHCR - tutorial (select two scopes: write:packages and delete:packages)
• Create repository secrets. Go to your repository -> Settings -> Secrets and add New repository secret and create two secrets: CR_PAT with GHCR token and VPS_PASSWORD - its password for user rails_app.

Then log in to your server with rails_app user and edit bashrc file. Add a line at the end of the file:

export CR_PAT=<your GHCR token>`

In your project create a file in that path /.github/workflows/deploy.yml

name: Deploy
on:
  push:
    branches:
      # Run deploy job on every push to the master branch
      - master
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      -
        name: Checkout
        uses: actions/checkout@v2
      -
        name: Login to GitHub Container Registry
        run: echo ${{ secrets.CR_PAT }} | docker login ghcr.io -u <YOUR GITHUB LOGIN> --password-stdin
      -
        name: Pull image to use as a cache
        run: docker pull ghcr.io/<YOUR GITHUB LOGIN>/rails_app:latest || exit 0
      -
        name: Build Docker image
        run: docker build . --cache-from ghcr.io/<YOUR GITHUB LOGIN>/rails_app:latest --tag ghcr.io/<YOUR GITHUB LOGIN>/rails_app:latest
      -
        name: Push the image to GitHub Container Registry
        run: docker push ghcr.io/<YOUR GITHUB LOGIN>/rails_app:latest
      -
        name: VPS - pull image and run app containters
        uses: appleboy/ssh-action@master
        with:
          host: <your-server-ip>
          username: rails_app 
          password: ${{ secrets.VPS_PASSWORD }}
          script: |
            echo $CR_PAT | docker login ghcr.io -u <YOUR GITHUB LOGIN> --password-stdin
            docker-compose pull web
            docker-compose up -d --no-deps

After first successful deploy, login to your server as rails_app and create database with command:

$ docker-compose run web rake db:create db:migrate

The last improvement

The final touch to make deployment fully automated is the migration script. Create a file docker-entrypoint.sh in your project main directory and paste the below content.

#!/bin/sh
set -e

if [ -f tmp/pids/server.pid ]; then
  rm tmp/pids/server.pid
fi

bundle exec rails db:migrate 2>/dev/null
exec bundle exec "$@"

And then few changes are needed in Dockerfile:

FROM ruby:3.0.1
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - \
  && echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
RUN apt-get update && apt-get install -y \
 build-essential libpq-dev nodejs zlib1g-dev liblzma-dev yarn
WORKDIR /app
COPY Gemfile* ./
RUN bundle install
COPY . .
RUN SECRET_KEY_BASE="assets_compile" RAILS_ENV=production bundle exec rake assets:precompile
# Add entrypoint script to handle migrations
ENTRYPOINT [ "./docker-entrypoint.sh" ]
EXPOSE 3000
CMD ["rails", "server", "-b", "0.0.0.0"]

You just created a fully working continuous deployment. You don't have to worry about errors on your local machine or some problems with the internet connection anymore. Just write your code and simply push commit and the rest is magic. Below some useful commands that may help you.

# view logs from Postgres
$ docker-compose logs database -f

# view logs from Rails
$ docker-compose logs web -f

# run Rails console inside Docker container
$ docker-compose run web rails c

# list available images
$ docker images

# list running containers
$ docker ps

# stop containers
$ docker-compose down

# remove old images/containers
$ docker system prune

Comments

[nihil-pro]

It doesn't work...

======CMD======
echo $CR_PAT | docker login ghcr.io -u nihilpro --password-stdin
docker-compose pull app
docker-compose up -d --no-deps
======END======
err: Error: Cannot perform an interactive login from a non TTY device
err: No such service: app
err: Pulling redis (redis:alpine)...
out: alpine: Pulling from library/redis
out: Digest: sha256: <..here sha.>
out: Status: Downloaded newer image for redis:alpine
err: Pulling store (app:)...
err: The image for the service you're trying to recreate has been removed. If you continue, volume data could be lost. Consider backing up your data before continuing.
err:
err: pull access denied for app, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
2021/09/28 11:24:33 Process exited with status 1
out: Continue with the new image? [yN]

[nihil-pro]

Hi
Can you help me please. I get an error with github actions:
When try to pull image
"
Run docker pull ghcr.io/nihilpro/app:latest || exit 0
docker pull ghcr.io/nihilpro/app:latest || exit 0
shell: /usr/bin/bash -e {0}
Error response from daemon: manifest unknown
"

And then when push the image to Github Container Registry
"
Run docker push ghcr.io/nihilpro/app:latest
docker push ghcr.io/nihilpro/app:latest
shell: /usr/bin/bash -e {0}
An image does not exist locally with the tag: ghcr.io/nihilpro/app
The push refers to repository [ghcr.io/nihilpro/app]
Error: Process completed with exit code 1.
"

[Marek Stocki]

@nihilpro sorry for that, I wrote a wrong service name, it should be

docker-compose pull web