I wanted to put together my home build server using my Raspberry Pi. After looking at the options I picked Drone CI, it has a nice interface, simple to use and has loads of plugins available.
In my last post, I showed you how to set up Traefik as a reverse proxy for the Docker images running on your Raspberry Pi. We are going to use Traefik again so if you haven’t got it set up you can follow that post.
Drone CI integrates nicely with GitHub. Once set up it will automatically pick up new commits, usually within seconds. The GitHub UI links back to the build as well.
For Drone CI to work we need to set up a new OAuth App on GitHub, Settings -> Developer Settings -> OAuth Apps.
If you followed my Traefik post, you should already have your domain name set up and pointing to your public IP address. However, Drone CI currently doesn’t support working from a folder (custom base path), so we need to set up a separate subdomain just for Drone CI. These can then be filled out when registering your application:
Once the application is registered you will then be given your Client ID and the option to generate your Client Secret. Store these somewhere safe as you will need them in a bit.
When I set up Traefik in my previous post I was only really concerned with running applications in subfolders. However, as Drone CI doesn’t support this, we need to change our previous set up to cope with subdomain as well.
Therefore we need to set up a wildcard SSL certificate for our domain so that we can have SSL on our new subdomains as well.
traefik.toml file previously looked liked this.
[entryPoints] [entryPoints.web] address = ":80" [entryPoints.web.http.redirections.entryPoint] to = "websecure" scheme = "https" [entryPoints.websecure] address = ":443" [entryPoints.websecure.http.tls] certResolver = "lets-encrypt" [api] dashboard = true [certificatesResolvers.lets-encrypt.acme] email = "your_email_here" storage = "acme.json" [certificatesResolvers.lets-encrypt.acme.tlsChallenge] [providers.docker] watch = true network = "web" exposedByDefault = false [providers.file] filename = "traefik_dynamic.toml"
We were using the tlsChallenge method to validate our certificate. However, for wildcard certificates, we have to use DNS validation. This generally requires a TXT record being either manually or automatically written to your records.
The process will vary depending on what registrar your domain is with. I am with Namecheap, so I will outline that process here.
Your new file needs to resemble the following.
[entryPoints] [entryPoints.web] address = ":80" [entryPoints.web.http.redirections.entryPoint] to = "websecure" scheme = "https" [entryPoints.websecure] address = ":443" [entryPoints.websecure.http.tls] certResolver = "lets-encrypt" [[entryPoints.websecure.http.tls.domains]] main = "yourdomain.com" sans = ["*.yourdomain.com"] [api] dashboard = true [log] level = "DEBUG" [certificatesResolvers.lets-encrypt.acme] email = "your_email_here" storage = "acme.json" [certificatesResolvers.lets-encrypt.acme.dnsChallenge] provider = "namecheap" resolvers = ["220.127.116.11:53", "18.104.22.168:53"] delayBeforeCheck = 5 [providers.docker] watch = true network = "web" exposedByDefault = false [providers.file] filename = "traefik_dynamic.toml"
You can see we have added in the domain and
* wildcard under
entryPoints.websecure.http.tls. I have also changed the
dnsChallenge and set
namecheap as the provider. You can find a list of providers on the Traefik website along with the environment variables you will need to add to docker compose. For Namecheap, these are
I found I also had to explicitly add DNS addresses (Google DNS in this case but you could use others) otherwise it couldn’t resolve my domain.
On the Namecheap website, we also need to enable API support. This is done by going to Profile -> Tools. At the bottom, you will find Namecheap API Access.
Once turned on you will be given an API Key which you need to add to your Traefik docker compose.
version: '3.4' services: traefik: image: 'traefik:2.3' container_name: 'traefik' restart: 'unless-stopped' ports: - '80:80' - '443:443' volumes: - '/var/run/docker.sock:/var/run/docker.sock:ro' - './config/traefik.toml:/traefik.toml' - './config/traefik_dynamic.toml:/traefik_dynamic.toml' - './config/acme.json:/acme.json' environment: - NAMECHEAP_API_USER=your_namecheap_username - NAMECHEAP_API_KEY=cg4b2ta63scakr2xdsfxw464vepax577 networks: - pi networks: pi: external: true
It might take a little while for the TXT entries to be added to your domain but once it is done you will be issued a domain and wildcard certificate from LetsEncrypt.
I will cut straight to the chase and share the
docker-compose.yml file I use for Drone CI.
version: '2' services: drone-server: image: drone/drone:1.10 restart: always volumes: - ./drone-data:/var/lib/drone/ - /var/run/docker.sock:/var/run/docker.sock environment: - DRONE_GITHUB_SERVER=https://github.com - DRONE_AGENTS_ENABLED=true - DRONE_SERVER_HOST=drone.yourdomain.com - DRONE_SERVER_PROTO=https - DRONE_TLS_AUTOCERT=true - DRONE_USER_CREATE=username:yourgithubusername,admin:true - DRONE_USER_FILTER=yourgithubusername - DRONE_GITHUB_CLIENT_ID=c8e5d02adeed68528987 - DRONE_GITHUB_CLIENT_SECRET=b3e6390faf15e17d4d0035ea10eaee6cd11eb51c - DRONE_RPC_SECRET=g46Myy3pTZtW2kzN - DOCKER_API_VERSION=1.37 labels: - 'traefik.enable=true' - 'traefik.http.routers.drone.rule=Host(`drone.yourdomain.com`)' - 'traefik.http.services.drone.loadbalancer.server.port=80' networks: - pi drone-runner: image: drone/drone-runner-docker:1.6.3 restart: always depends_on: - drone-server volumes: - /var/run/docker.sock:/var/run/docker.sock environment: - DRONE_RPC_HOST=drone-server - DRONE_RPC_PROTO=http - DRONE_RPC_SECRET=g46Myy3pTZtW2kzN - DRONE_RUNNER_CAPACITY=1 - DRONE_RUNNER_NAME=drone-runner-docker - DOCKER_API_VERSION=1.37 - PLUGIN_CUSTOM_DNS=22.214.171.124 networks: - pi networks: pi: external: true
Drone CI consists of a server and a runner. As the runner is on the same network (the
pi network we set up in the previous post) it can communicate with the server using
To get this docker-compose working with your set up there are a few bits we need to change. The following environment variables on the server need to be updated:
- DRONE_SERVER_HOST=drone.yourdomain.com - DRONE_USER_CREATE=username:yourgithubusername,admin:true - DRONE_USER_FILTER=yourgithubusername - DRONE_GITHUB_CLIENT_ID=c8e5d02adeed68528987 - DRONE_GITHUB_CLIENT_SECRET=b3e6390faf15e17d4d0035ea10eaee6cd11eb51c - DRONE_RPC_SECRET=g46Myy3pTZtW2kzN
As this is being installed on your Raspberry Pi I am going to assume it is for your use only. The environment variables
DRONE_USER_FILTER make you the admin and stop other users from being able to create accounts on your Drone CI.
DRONE_GITHUB_CLIENT_SECRET need to be updated to use the Client ID and Secret from GitHub we saved earlier.
Lastly, we need to generate a random string which is going to be used to authenticate the runner with the server, this should be stored in
DRONE_RPC_SECRET and updated on the server and runner.
Again I had issues with docker images being able to resolve domains and download packages so I had to add
PLUGIN_CUSTOM_DNS=126.96.36.199 to the runner to fix this.
Once set up you should be able to access your Drone CI using your subdomain and you can log in with your GitHub account.
After syncing you should see a list of all your repositories. You will need to activate the ones you want Drone CI to monitor.
For Drone CI to be able to build your projects you need to add a
.drone.yml file yo your project. As we are running on the Raspberry Pi we need to specify
arm as the platform. The platform defaults to
amd64 so if we leave it blank our Raspberry Pi won’t pick it up.
A very simple drone file looks like this:
kind: pipeline type: docker name: default platform: os: linux arch: arm steps: - name: greeting image: alpine commands: - echo hello - echo world
You can use different docker images for different steps and they will all share the same workspace. Drone CI have great documentation on steps and what you can do with them.
I like Drone CI, it is simple and seems to have everything I need from a home CI tool.
However, there is one flaw with doing this on a Raspberry Pi. The Raspberry Pi has an
arm processor whereas your laptop and hosting environment probably doesn’t.
I originally planned to build my Gatsby.js websites on my Raspberry Pi and push the built files up to AWS S3. However, what I didn’t take into account, was all of the dependencies Gatsby and it’s plugins have.
If you have done any frontend work you will know that a staggering amount of packages get downloaded into your
node_modules folder. This folder has all of your project dependencies and your dependencies, dependencies etc. For example, the
node_modules folder for this website has 1,429 packages installed.
Some of these dependencies rely on binary files which may not be available for the
arm processor used in a Raspberry Pi. As such these modules need to be built from source, which requires you to have all of the correct package versions installed to be able to compile these.
I spent a few days trying to get the dependencies together so I could build my websites but once one thing was working, something else would be missing.
I will still keep it set up on my Pi but will limit its use for running unit tests and builds for projects with few dependencies. If you have an x86 based home server then this could be a good option.