Self-Hosted GitHub Actions Runner in Kubernetes

Self Hosted Runner in K3s Cluster

Hands-on Guide: Self-Hosted GitHub Actions Runner in Kubernetes
GitHub Actions is a powerful tool for automating software workflows, and it can be used to build, test, and deploy code right from GitHub. It provides a way to automate repetitive tasks and can be integrated with many popular tools and platforms.

GitHub Actions can use two types of runners:

1. Hosted and
2. Self-hosted.

Hosted runners are provided by GitHub and run on virtual machines in the cloud.

Self-hosted runners are machines that you set up and manage yourself. They run on your infrastructure, and you can customize them to meet your needs.

Introduction
This guide provides step-by-step instructions for setting up a self-hosted GitHub Actions runner in Kubernetes using Docker-in-Docker (DinD). This setup allows you to run GitHub Actions workflows in your own infrastructure.

Prerequisites
Before starting, ensure you have:

• A working Kubernetes cluster (k3s/kind/etc.)
• kubectl installed and configured
• Docker installed
• A GitHub account and repository
• A GitHub Personal Access Token (PAT)

Step 1: Setting Up the Project Structure
Create a new directory for the project:

mkdir github-runner-k8s
cd github-runner-k8s

Create necessary files:

touch Dockerfile entrypoint.sh kubernetes.yaml

Step 2: Creating the Dockerfile
The Dockerfile creates a container image that serves as our GitHub Actions runner environment. It creates a reproducible environment for the runner and installs necessary tools (Docker CLI, jq, etc.). It forms the base container image for the runner pod in Kubernetes.

docker-integration

Create the Dockerfile with the following content:

FROM debian:bookworm-slim
ARG RUNNER_VERSION="2.302.1"
ENV GITHUB_PERSONAL_TOKEN ""
ENV GITHUB_OWNER ""
ENV GITHUB_REPOSITORY ""

# Install Docker
RUN apt-get update && \
    apt-get install -y ca-certificates curl gnupg
RUN install -m 0755 -d /etc/apt/keyrings
RUN curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg
RUN chmod a+r /etc/apt/keyrings/docker.gpg

# Add Docker repository
RUN echo \
  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian \
  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
  tee /etc/apt/sources.list.d/docker.list > /dev/null
RUN apt-get update

# Install required packages
RUN apt-get install -y docker-ce-cli sudo jq

# Setup github user
RUN useradd -m github && \
    usermod -aG sudo github && \
    echo "%sudo ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers

# Create directories with correct permissions
RUN mkdir -p /actions-runner && \
    chown -R github:github /actions-runner && \
    mkdir -p /work && \
    chown -R github:github /work

USER github
WORKDIR /actions-runner

# Download and install runner
RUN curl -Ls https://github.com/actions/runner/releases/download/v${RUNNER_VERSION}/actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz -o actions-runner.tar.gz && \
    tar xzf actions-runner.tar.gz && \
    rm actions-runner.tar.gz && \
    sudo ./bin/installdependencies.sh

COPY --chown=github:github entrypoint.sh /actions-runner/entrypoint.sh
RUN sudo chmod u+x /actions-runner/entrypoint.sh

ENTRYPOINT ["/actions-runner/entrypoint.sh"]

Step 3: Creating the Entrypoint Script
The entrypoint script handles the runner's lifecycle - registration, execution, and cleanup. It automatically registers the runner with GitHub. It acts as the bridge between container and GitHub.

entrypoint

Create entrypoint.sh with the following content:

#!/bin/sh
registration_url="https://api.github.com/repos/${GITHUB_OWNER}/${GITHUB_REPOSITORY}/actions/runners/registration-token"
echo "Requesting registration URL at '${registration_url}'"
payload=$(curl -sX POST -H "Authorization: token ${GITHUB_PERSONAL_TOKEN}" ${registration_url})
export RUNNER_TOKEN=$(echo $payload | jq .token --raw-output)

./config.sh \
    --name $(hostname) \
    --token ${RUNNER_TOKEN} \
    --labels my-runner \
    --url https://github.com/${GITHUB_OWNER}/${GITHUB_REPOSITORY} \
    --work "/work" \
    --unattended \
    --replace

remove() {
    ./config.sh remove --unattended --token "${RUNNER_TOKEN}"
}

trap 'remove; exit 130' INT
trap 'remove; exit 143' TERM

./run.sh "$*" &
wait $!

Make the script executable:

chmod +x entrypoint.sh

Step 4: Creating the Kubernetes Deployment
This step defines how the runner should be deployed and managed in Kubernetes.

k8s-integration

Create kubernetes.yaml with the following content:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: github-runner
  labels:
    app: github-runner
spec:
  replicas: 1
  selector:
    matchLabels:
      app: github-runner
  template:
    metadata:
      labels:
        app: github-runner
    spec:
      containers:
      - name: github-runner
        imagePullPolicy: Never
        image: github-runner:latest
        env:
        - name: GITHUB_OWNER
          valueFrom:
            secretKeyRef:
              name: github-secret
              key: GITHUB_OWNER
        - name: GITHUB_REPOSITORY
          valueFrom:
            secretKeyRef:
              name: github-secret
              key: GITHUB_REPOSITORY
        - name: GITHUB_PERSONAL_TOKEN
          valueFrom:
            secretKeyRef:
              name: github-secret
              key: GITHUB_PERSONAL_TOKEN
        - name: DOCKER_HOST
          value: tcp://localhost:2375
        volumeMounts:
        - name: data
          mountPath: /work/
      - name: dind
        image: docker:24.0.6-dind
        env:
        - name: DOCKER_TLS_CERTDIR
          value: ""
        resources:
          requests:
            cpu: 20m
            memory: 512Mi
        securityContext:
          privileged: true
        volumeMounts:
          - name: docker-graph-storage
            mountPath: /var/lib/docker
          - name: data
            mountPath: /work/
      volumes:
      - name: docker-graph-storage
        emptyDir: {}
      - name: data
        emptyDir: {}

Step 5: Building and Loading the Image
Build the Docker image:

docker build . -t github-runner:latest

Load the image into your Kubernetes cluster:
For k3s:

# Save the image
docker save github-runner:latest -o github-runner.tar

# Import to k3s
sudo k3s ctr images import github-runner.tar

Step 6: Creating GitHub Token
Go to GitHub Settings → Developer settings → Personal access tokens → Tokens (classic)
Generate new token with following permissions:
repo (full control)
workflow
admin:org (if using organization repository)

Step 7: Creating Kubernetes Secrets
Create namespace:

kubectl create namespace host-runner

Create secrets (replace placeholder values):

kubectl -n host-runner create secret generic github-secret \
  --from-literal=GITHUB_OWNER=<your-github-username> \
  --from-literal=GITHUB_REPOSITORY=<your-repo-name> \
  --from-literal=GITHUB_PERSONAL_TOKEN=<your-github-token>

Step 8: Deploying to Kubernetes
Apply the Kubernetes deployment:

kubectl -n host-runner apply -f kubernetes.yaml

Verify the deployment:

# Check pod status
kubectl -n host-runner get pods

# Check runner logs
kubectl -n host-runner logs -f <pod-name> -c github-runner

Step 9: Verifying the Setup
Go to your GitHub repository
Navigate to Settings → Actions → Runners
You should see your self-hosted runner listed and "Idle"
Troubleshooting Guide
Common Issues and Solutions
Image Pull Error

# Check if image is properly loaded
sudo crictl images | grep github-runner

# If not visible, reload the image
sudo k3s ctr images import github-runner.tar
**Permission Issues**
# Check pod logs
kubectl -n host-runner logs -f <pod-name> -c github-runner

# Verify secrets
kubectl -n host-runner get secrets github-secret -o yaml

Runner Not Registering

# Check if token is valid
kubectl -n host-runner logs <pod-name> -c github-runner | grep "Requesting registration URL"

# Verify network connectivity
kubectl -n host-runner exec <pod-name> -c github-runner -- curl -s https://api.github.com

Cleanup Instructions
To remove the setup:

# Delete the deployment
kubectl -n host-runner delete -f kubernetes.yaml

# Delete the secrets
kubectl -n host-runner delete secret github-secret

# Delete the namespace
kubectl delete namespace host-runner

# Remove local images
docker rmi github-runner:latest

Testing the Runner
Create a simple workflow in your repository:

# .github/workflows/test.yml
name: Test Self-Hosted Runner
on: [push]
jobs:
  test:
    runs-on: self-hosted
    steps:
      - uses: actions/checkout@v2
      - name: Test Runner
        run: |
          echo "Hello from self-hosted runner!"
          docker --version

Commit and push this file to your repository
Check the Actions tab in your repository to see the workflow running
Maintenance Tips
Updating Runner Version:
Update RUNNER_VERSION in Dockerfile

Rebuild and redeploy
Scaling Runners:
Modify replicas in kubernetes.yaml
Apply changes with kubectl

Monitoring:

# Check runner status
kubectl -n host-runner get pods -w

# Check resource usage
kubectl -n host-runner top pod

Remember to:

• Regularly update the runner version
• Monitor resource usage
• Rotate GitHub tokens periodically
• Keep Docker images updated

Benefits of using Self-Hosted Runner:

Improved Performance: By hosting your runners, you can ensure that the build and deployment processes are faster and more reliable, as you have complete control over the hardware and networking resources.

Increased Security: GitHub self-hosted runners can be configured to run on your own servers, which provides an extra layer of security compared to using shared runners. This helps to protect sensitive information and data in your workflows.

Customizable Environments: With self-hosted runners, you can create custom environments that match your exact needs, including specific software versions and configurations.

Cost-Effective: If you have a large number of workflows or use cases that require a lot of resources, self-hosted runners can be more cost-effective than using GitHub’s shared runners or cloud-based solutions.

High Availability: With self-hosted runners, you can set up Horizontal Runner Autoscaler, which provides redundancy and high availability for your workflows.

Greater Control: Self-hosted runners give you complete control over the resources and environment used for your workflows, which can help you optimize performance and ensure that your builds and deployments run smoothly.

Overall, GitHub self-hosted runners offer greater flexibility, control, and customization options than using shared runners or cloud-based solutions. However, setting up and managing self-hosted runners requires additional effort and resources, so it’s essential to weigh the benefits against the costs and resources needed to maintain them.

Conclusion
In conclusion, the Actions Runner Controller with Self-Hosted Runner is a powerful tool that can help you improve your GitHub Actions workflows. It simplifies the management of runners by automating tasks such as the creation, scaling, and deletion of runners. It also provides an easy way to create and manage runners in a Kubernetes cluster. By using Actions Runner Controller with Self-Hosted Runner, you can take full advantage of the power of GitHub Actions while having full control over your infrastructure.