How to set up Drone CI on Raspberry Pi (and why you shouldn't)

I wanted to put together my home build server using my Raspberry Pi. After looking at the options I picked Drone CI, it has a nice interface, simple to use and has loads of plugins available.

In my last post, I showed you how to set up Traefik as a reverse proxy for the Docker images running on your Raspberry Pi. We are going to use Traefik again so if you haven’t got it set up you can follow that post.

Giving Drone Access to GitHub

Drone CI integrates nicely with GitHub. Once set up it will automatically pick up new commits, usually within seconds. The GitHub UI links back to the build as well.

For Drone CI to work we need to set up a new OAuth App on GitHub, Settings -> Developer Settings -> OAuth Apps.

If you followed my Traefik post, you should already have your domain name set up and pointing to your public IP address. However, Drone CI currently doesn’t support working from a folder (custom base path), so we need to set up a separate subdomain just for Drone CI. These can then be filled out when registering your application:

Once the application is registered you will then be given your Client ID and the option to generate your Client Secret. Store these somewhere safe as you will need them in a bit.

Wildcard SSL certificates on Traefik

When I set up Traefik in my previous post I was only really concerned with running applications in subfolders. However, as Drone CI doesn’t support this, we need to change our previous set up to cope with subdomain as well.

Therefore we need to set up a wildcard SSL certificate for our domain so that we can have SSL on our new subdomains as well.

Our traefik.toml file previously looked liked this.

[entryPoints]
  [entryPoints.web]
    address = ":80"
    [entryPoints.web.http.redirections.entryPoint]
      to = "websecure"
      scheme = "https"

  [entryPoints.websecure]
    address = ":443"

    [entryPoints.websecure.http.tls]
      certResolver = "lets-encrypt"

[api]
  dashboard = true

[certificatesResolvers.lets-encrypt.acme]
  email = "your_email_here"
  storage = "acme.json"
  [certificatesResolvers.lets-encrypt.acme.tlsChallenge]

[providers.docker]
  watch = true
  network = "web"
  exposedByDefault = false

[providers.file]
  filename = "traefik_dynamic.toml"

We were using the tlsChallenge method to validate our certificate. However, for wildcard certificates, we have to use DNS validation. This generally requires a TXT record being either manually or automatically written to your records.

The process will vary depending on what registrar your domain is with. I am with Namecheap, so I will outline that process here.

Your new file needs to resemble the following.

[entryPoints]
  [entryPoints.web]
    address = ":80"
    [entryPoints.web.http.redirections.entryPoint]
      to = "websecure"
      scheme = "https"

  [entryPoints.websecure]
    address = ":443"

    [entryPoints.websecure.http.tls]
      certResolver = "lets-encrypt"
      [[entryPoints.websecure.http.tls.domains]]
        main = "yourdomain.com"
        sans = ["*.yourdomain.com"]

[api]
  dashboard = true

[log]
  level = "DEBUG"

[certificatesResolvers.lets-encrypt.acme]
  email = "your_email_here"
  storage = "acme.json"
  [certificatesResolvers.lets-encrypt.acme.dnsChallenge]
    provider = "namecheap"
    resolvers = ["8.8.8.8:53", "8.8.4.4:53"]
    delayBeforeCheck = 5

[providers.docker]
  watch = true
  network = "web"
  exposedByDefault = false

[providers.file]
  filename = "traefik_dynamic.toml"

You can see we have added in the domain and * wildcard under entryPoints.websecure.http.tls. I have also changed the tlsChallenge to dnsChallenge and set namecheap as the provider. You can find a list of providers on the Traefik website along with the environment variables you will need to add to docker compose. For Namecheap, these are NAMECHEAP_API_USER and NAMECHEAP_API_KEY.

I found I also had to explicitly add DNS addresses (Google DNS in this case but you could use others) otherwise it couldn’t resolve my domain.

On the Namecheap website, we also need to enable API support. This is done by going to Profile -> Tools. At the bottom, you will find Namecheap API Access.

Once turned on you will be given an API Key which you need to add to your Traefik docker compose.

version: '3.4'

services:
  traefik:
    image: 'traefik:2.3'
    container_name: 'traefik'
    restart: 'unless-stopped'
    ports:
      - '80:80'
      - '443:443'
    volumes:
      - '/var/run/docker.sock:/var/run/docker.sock:ro'
      - './config/traefik.toml:/traefik.toml'
      - './config/traefik_dynamic.toml:/traefik_dynamic.toml'
      - './config/acme.json:/acme.json'
   environment:
      - NAMECHEAP_API_USER=your_namecheap_username
      - NAMECHEAP_API_KEY=cg4b2ta63scakr2xdsfxw464vepax577
    networks:
      - pi

networks:
  pi:
    external: true

It might take a little while for the TXT entries to be added to your domain but once it is done you will be issued a domain and wildcard certificate from LetsEncrypt.

Setting up Drone CI Docker Compose

I will cut straight to the chase and share the docker-compose.yml file I use for Drone CI.

version: '2'

services:
  drone-server:
    image: drone/drone:1.10
    restart: always
    volumes:
      - ./drone-data:/var/lib/drone/
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - DRONE_GITHUB_SERVER=https://github.com
      - DRONE_AGENTS_ENABLED=true
      - DRONE_SERVER_HOST=drone.yourdomain.com
      - DRONE_SERVER_PROTO=https
      - DRONE_TLS_AUTOCERT=true
      - DRONE_USER_CREATE=username:yourgithubusername,admin:true
      - DRONE_USER_FILTER=yourgithubusername
      - DRONE_GITHUB_CLIENT_ID=c8e5d02adeed68528987
      - DRONE_GITHUB_CLIENT_SECRET=b3e6390faf15e17d4d0035ea10eaee6cd11eb51c
      - DRONE_RPC_SECRET=g46Myy3pTZtW2kzN
      - DOCKER_API_VERSION=1.37
    labels:
      - 'traefik.enable=true'
      - 'traefik.http.routers.drone.rule=Host(`drone.yourdomain.com`)'
      - 'traefik.http.services.drone.loadbalancer.server.port=80'
    networks:
      - pi

  drone-runner:
    image: drone/drone-runner-docker:1.6.3
    restart: always
    depends_on:
      - drone-server
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - DRONE_RPC_HOST=drone-server
      - DRONE_RPC_PROTO=http
      - DRONE_RPC_SECRET=g46Myy3pTZtW2kzN
      - DRONE_RUNNER_CAPACITY=1
      - DRONE_RUNNER_NAME=drone-runner-docker
      - DOCKER_API_VERSION=1.37
      - PLUGIN_CUSTOM_DNS=8.8.8.8
    networks:
      - pi

networks:
  pi:
    external: true

Drone CI consists of a server and a runner. As the runner is on the same network (the pi network we set up in the previous post) it can communicate with the server using drone-server.

To get this docker-compose working with your set up there are a few bits we need to change. The following environment variables on the server need to be updated:

      - DRONE_SERVER_HOST=drone.yourdomain.com
      - DRONE_USER_CREATE=username:yourgithubusername,admin:true
      - DRONE_USER_FILTER=yourgithubusername
      - DRONE_GITHUB_CLIENT_ID=c8e5d02adeed68528987
      - DRONE_GITHUB_CLIENT_SECRET=b3e6390faf15e17d4d0035ea10eaee6cd11eb51c
      - DRONE_RPC_SECRET=g46Myy3pTZtW2kzN

As this is being installed on your Raspberry Pi I am going to assume it is for your use only. The environment variables DRONE_USER_CREATE and DRONE_USER_FILTER make you the admin and stop other users from being able to create accounts on your Drone CI.

DRONE_GITHUB_CLIENT_ID and DRONE_GITHUB_CLIENT_SECRET need to be updated to use the Client ID and Secret from GitHub we saved earlier.

Lastly, we need to generate a random string which is going to be used to authenticate the runner with the server, this should be stored in DRONE_RPC_SECRET and updated on the server and runner.

Again I had issues with docker images being able to resolve domains and download packages so I had to add PLUGIN_CUSTOM_DNS=8.8.8.8 to the runner to fix this.

Once set up you should be able to access your Drone CI using your subdomain and you can log in with your GitHub account.

After syncing you should see a list of all your repositories. You will need to activate the ones you want Drone CI to monitor.

Adding your .drone.yml file

For Drone CI to be able to build your projects you need to add a .drone.yml file yo your project. As we are running on the Raspberry Pi we need to specify arm as the platform. The platform defaults to amd64 so if we leave it blank our Raspberry Pi won’t pick it up.

A very simple drone file looks like this:

kind: pipeline
type: docker
name: default
platform:
  os: linux
  arch: arm

steps:
  - name: greeting
    image: alpine
    commands:
      - echo hello
      - echo world

You can use different docker images for different steps and they will all share the same workspace. Drone CI have great documentation on steps and what you can do with them.

Why you shouldn’t run Drone CI on a Raspberry Pi

I like Drone CI, it is simple and seems to have everything I need from a home CI tool.

However, there is one flaw with doing this on a Raspberry Pi. The Raspberry Pi has an arm processor whereas your laptop and hosting environment probably doesn’t.

I originally planned to build my Gatsby.js websites on my Raspberry Pi and push the built files up to AWS S3. However, what I didn’t take into account, was all of the dependencies Gatsby and it’s plugins have.

If you have done any frontend work you will know that a staggering amount of packages get downloaded into your node_modules folder. This folder has all of your project dependencies and your dependencies, dependencies etc. For example, the node_modules folder for this website has 1,429 packages installed.

Some of these dependencies rely on binary files which may not be available for the arm processor used in a Raspberry Pi. As such these modules need to be built from source, which requires you to have all of the correct package versions installed to be able to compile these.

I spent a few days trying to get the dependencies together so I could build my websites but once one thing was working, something else would be missing.

I will still keep it set up on my Pi but will limit its use for running unit tests and builds for projects with few dependencies. If you have an x86 based home server then this could be a good option.