Self-hosted Continuous Delivery that doesn't cost a fortune 💰 - Part 2

Overview

In the first part we set up Docker and installed Traefik on our VPS (Virtual Private Server).

What's left in this final part of setting up your own Continuous Delivery pipeline is:

• setting up your web app
• installing Drone
• connecting GitHub to Drone
• setting up an auto-update over SSH

Setting up a new web app in Traefik

I think it was worth to install Traefik just for being able to very quickly deploy any Docker image and link a (sub)domain to it with Let's Encrypt's SSL.

It allows you to add a new NodeJS/React/Wordpress web apps or MySQL/PostgreSQL instances in a matter of minutes.

Because once Traefik is set up adding a new piece of software boils down to setting up a new A type in a DNS zone for your domain that will point to your server's IP address and adding another entry in docker-compose.yml file in sites directory. It's quite close to an expierience with using Netlify.

To start, create a new directory inside your home directory called sites:

cd ~
mkdir sites
cd sites

Create a docker-compose.yml file and paste below into it:

version: '3'

services:
  fooapp: # **** EDIT HERE ****
    image: docker.wolnik.co.uk/fooapp # **** EDIT HERE ****
    labels:
      - traefik.backend=fooapp # **** EDIT HERE ****
      - traefik.frontend.rule=Host:fooapp.com # **** EDIT HERE ****
      - traefik.docker.network=web
      - traefik.port=80
    networks:
      - web
  mysql:
    image: mysql:8.0.3
    restart: always
    ports:
      - 3306:3306
    environment:
      MYSQL_ROOT_PASSWORD: apasswordheretochange # **** EDIT HERE ****
      MYSQL_DATABASE: yourdbname # **** EDIT HERE ****
    networks:
      - web
  adminer:
    image: dockette/adminer
    labels:
      - traefik.frontend.rule=Host:mysql.foo.com # **** EDIT HERE ****
      - traefik.docker.network=web
      - traefik.port=80
    restart: always
    ports:
      - 8080:80 # Here I'm mapping adminer's default 8080 port to 80
    networks:
      - web

networks:
  web:
    external: true

There's quite a lot going on above but actually the only new part that deals with Traefik is adding labels so that Traefik can assign a domain for your container and route to the right port.

Run your containers with (you must be in ~/sites directory):

docker-compose up -d

The -d flag means that docker-compose will run in background so you can exit your SSH session and your containers will still run.

Installing Drone

Setting up GitHub OAuth App

Go to GitHub. Open Settings → Developer options → OAuth Apps.

Then fill out the fields:

• Homepage URL: https://drone.yourdomain.com
• Authorization callback URL: https://drone.yourdomain.com/login

Once created copy the GitHub client id and secret:

Drone Server

Create a subfolder in your home folder.

cd ~ && mkdir drone && cd $_

Create a docker-compose.yml file and paste into it below:

version: "3"

networks:
  web:
    external: true
  internal:
    external: false

services:
  drone:
    image: drone/drone:1
    labels:
      - traefik.backend=drone
      - traefik.frontend.rule=Host:drone.yourdomain.com # **** EDIT HERE ****
      - traefik.docker.network=web
      - traefik.port=80
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /var/lib/drone:/home/yourusername/traefik/drone/data # **** EDIT HERE ****
    environment:
      - DRONE_GITHUB_SERVER=https://github.com
      - DRONE_GITHUB_CLIENT_ID=********** # **** EDIT HERE ****
      - DRONE_GITHUB_CLIENT_SECRET=**************************** # **** EDIT HERE ****
      - DRONE_RUNNER_CAPACITY=2
      - DRONE_RPC_SECRET=THIS_RANDOIM_STRING_NEED_TO_BE_THE_SAME # **** EDIT HERE ****
      - DRONE_SERVER_HOST=drone.yourdomain.com # **** EDIT HERE ****
      - DRONE_SERVER_PROTO=https
      - DRONE_TLS_AUTOCERT=true
      - DRONE_AGENTS_ENABLED=true
    networks:
      - internal
      - web
  runner:
    image: drone/drone-runner-docker:1
    restart: always
    ports:
      - "3000:3000"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - DRONE_RPC_PROTO=https
      - DRONE_RPC_HOST=drone.yourdomain.com # **** EDIT HERE ****
      - DRONE_RPC_SECRET=THIS_RANDOM_STRING_NEED_TO_BE_THE_SAME # **** EDIT HERE ****
      - DRONE_RUNNER_CAPACITY=2
      - DRONE_RUNNER_NAME=drone.yourdomain.com THIS_STRING_NEED_TO_BE_THE_SAME # **** EDIT HERE ****
    networks:
      - internal
      - web

Go to drone.yourdomain.com URL and you should see Drone UI running. Asking you to login via your GitHub.

Login to Docker on VPS

Make sure your VPS can connect to your Docker Hub (or your own Docker repo) by running:

docker login -u user

or if you have your own Docker repo at docker.yourdomain.com:

docker login -u user docker.yourdomain.com

You can confirm that you are logged in if there's an auth section in ~/.docker/config.json file. And of course if you are not asked for authentication when pulling image from Docker.

Setting up Continuous Delivery in Drone

First create a script on your VPS that pulls Docker images and restarts docker-compose.yml in ~/sites directory. Name it update-all.sh and paste below into it:

docker pull docker.wolnik.co.uk/fooapp # **** EDIT **** Docker image name
docker pull docker.yourdomain.com/fooapp # **** EDIT/DELETE **** Another web app
docker-compose up -d

Insert there all images that you are intending to update over Drone. So that docker-compose will use the latest Docker images of your web apps.

Drone SSH user

Above update-all.sh script must be invoked by Drone runner after completing building and pushing new Docker images once they are build.

The best way is to create a separate user for this task on your VPS:

adduser drone
sudo usermod -aG docker drone
chown drone: ~/sites/update-all.sh
chmod +x ~/sites/update-all.sh

.drone.yml

In order for Drone to know how to build your web app add .drone.yml file inside root of your Git repo, i.e.

Paste below into .drone.yml file inside your Git repo, add and commit to your master branch.

pipeline:
  build:
    image: plugins/docker
    registry: docker.wolnik.co.uk
    repo: docker.wolnik.co.uk/fooapp
    username:
      from_secret: docker_username
    password:
      from_secret: docker_password
    tags:
      - 1.0.0
      - 1.0
      - latest
  ssh:
    image: appleboy/drone-ssh
    host: fooapp.com
    username:
      from_secret: ssh_username
    password:
      from_secret: ssh_password
    port: 22
    command_timeout: 2m
    script:
      - cd /home/yourdominaname/sites/ && ./update-all.sh

In above file we set up two steps that will build and deploy our web app.

First step - build - will pull the latest Git repo from master and build Docker image based on its Dockerfile. Then it will push the image to your Docker registry.

Second step - ssh - will connect to your server over SSH, go to your sites directory and run update-all.sh script which in turn will pull your Docker images and restart docker-compose so that your web app is using the latest - just built - Docker image.

Important if you are using your own Docker repo!

Remember that both registry URL and repo must match, i.e. repo can't be just fooapp if registry has a domain docker.yourdomain.com. It needs to be then docker.yourdomain.com/fooapp in order to work.

Setting up secrets in Drone

A .drone.yml contains four secret strings that forms two credentials. One for Docker repo (DockerHub or your own Docker repo) and SSH access.

Go to your Drone UI and set up 4 secrets that will replace the placeholders in your Git repo's .drone.yml file.

Final words

Please be patient while fine tuning your .drone.yml and Drone secrets. Sometimes it's tricky to get the Drone to pick up the lastest GitHub's commit.

But it's all worth it once you can see a running gear like below for your own repo:

P.S. Of course this is just a start. Now you can extend your Drone's pipeline with additional step like running tests before deploying anything on a live site.