DEV Community

Big Mazzy
Big Mazzy

Posted on • Originally published at serverrental.store

CI/CD Pipeline on Your Own Server: GitHub Actions + VPS

#devops #docker #cloud #webdev

Thinking about the costs and limitations of cloud-based CI/CD services? This article will guide you through setting up your own CI/CD pipeline on your own server, leveraging GitHub Actions and a Virtual Private Server (VPS). You'll learn how to gain more control, reduce expenses, and tailor your build and deployment processes precisely to your needs.

Why Self-Host Your CI/CD Pipeline?

Many developers start with free or low-cost CI/CD solutions offered by platforms like GitHub Actions, GitLab CI, or Bitbucket Pipelines. While these are excellent for getting started, as your projects grow, you might encounter limitations. These can include build minute caps, storage restrictions, or a lack of granular control over the build environment. You might also find that scaling these services can become more expensive than anticipated.

For instance, if you have a large project with frequent builds, you could easily exceed the free tier of build minutes. This means paying for additional usage, which can add up. Furthermore, you might need specific software or configurations on your build agents that aren't readily available or are difficult to set up on shared cloud runners. This is where the power of having your own server comes into play.

Understanding the Core Components

Before we dive into the setup, let's clarify the key technologies we'll be using.

  • CI/CD (Continuous Integration/Continuous Deployment or Delivery): This is a set of practices that automates the software development lifecycle. Continuous Integration (CI) involves merging code changes from multiple developers into a central repository frequently. Continuous Deployment (CD) automatically deploys every change that passes the CI stage to a production environment. Continuous Delivery is similar, but the final deployment to production requires manual approval.
  • GitHub Actions: A CI/CD platform integrated directly into GitHub. It allows you to automate your software development workflows, such as building, testing, and deploying your code, directly from your GitHub repository. You define these workflows using YAML files.
  • Virtual Private Server (VPS): A virtual machine sold as a service by an internet hosting service. A VPS runs its own copy of an operating system, and customers have superuser-level access to that copy of the operating system, so they can install almost any software that runs on that OS. This gives you dedicated resources and control over your server environment. Think of it like renting a small apartment (your VPS) in a larger building (the data center), giving you your own space and keys, rather than staying in a hotel room (a shared cloud runner) with limited access.

Choosing Your Server Provider

Selecting the right VPS provider is crucial for performance and reliability. For this setup, you'll want a provider that offers good performance, reasonable pricing, and reliable uptime. I've had positive experiences with both PowerVPS and Immers Cloud. Both offer competitive pricing for their VPS plans, making them excellent choices for self-hosting CI/CD infrastructure. PowerVPS, for example, provides a range of plans suitable for different workloads, and Immers Cloud offers a user-friendly interface for managing your virtual servers. For those new to server rentals, the Server Rental Guide is an excellent resource to help you compare options.

When choosing a plan, consider the CPU, RAM, and storage requirements for your builds. If you're running complex builds or compiling large projects, you'll need more powerful resources.

Setting Up Your Self-Hosted Runner

GitHub Actions can run on GitHub-hosted runners or on self-hosted runners. For our setup, we'll configure a self-hosted runner on our VPS. This runner will be a small piece of software that listens for jobs from GitHub and executes them on your server.

Step 1: Provision Your VPS

Once you've chosen a provider and a plan, provision your VPS. For this guide, we'll assume you're using a Linux distribution like Ubuntu.

  • Choose an Operating System: Select a stable Linux distribution like Ubuntu LTS.
  • Secure Your Server: Before installing anything else, ensure your server is secure. This includes:
    • Updating all packages: sudo apt update && sudo apt upgrade -y
    • Setting up a firewall (e.g., UFW): sudo ufw enable, sudo ufw allow ssh, sudo ufw allow http, sudo ufw allow https
    • Creating a new user with sudo privileges and disabling root login via SSH.

Step 2: Install Docker

Docker is highly recommended for self-hosted runners. It provides a consistent and isolated environment for your build jobs.

  1. Install Docker:

    curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

  2. Add your user to the docker group:

    sudo usermod -aG docker $USER

    You'll need to log out and log back in for this change to take effect.

Step 3: Register a GitHub Actions Self-Hosted Runner

Now, let's register your VPS as a self-hosted runner with your GitHub repository.

  1. Navigate to your GitHub Repository: Go to your repository on GitHub.
  2. Go to Settings: Click on "Settings" in the repository navigation.
  3. Select Actions: In the left-hand sidebar, click on "Actions," then "Runners."
  4. Click "Set up a self-hosted runner": This will present you with options for different operating systems.
  5. Choose your OS and Architecture: Select the appropriate options for your VPS (e.g., Linux, x64).

  6. Follow the Instructions: GitHub will provide you with a script to download and run. This script will:

    • Download the runner application.
    • Prompt you for your GitHub repository URL and a token for authentication.
    • Ask for a name for your runner.
    • Ask for labels (e.g., self-hosted, linux, docker) to identify your runner. These labels are crucial for targeting specific runners in your workflows.

    The commands will look something like this (but do not copy these directly, use the ones provided by GitHub for your specific repository):

    # Example commands, get your actual commands from GitHub
mkdir actions-runner && cd actions-runner
curl -o actions.zip -L https://github.com/actions/runner/releases/download/v2.315.0/actions-runner-linux-x64-2.315.0.tar.gz
tar xzf actions.zip
./config.sh --url https://github.com/your-username/your-repo --token YOUR_UNIQUE_TOKEN

  7. Run the runner: After configuration, you'll run the runner. It's best to run this as a service so it starts automatically.

    # Example: Run as a service (configure this properly for your system)
sudo ./svc.sh install
sudo ./svc.sh start

Step 4: Use Labels in Your GitHub Actions Workflow

Once your runner is registered and running, you can tell your GitHub Actions workflows to use it. You do this by adding a runs-on directive in your workflow file, specifying the labels you assigned to your runner (e.g., self-hosted).

Here's a simplified example of a .github/workflows/main.yml file:

name: CI/CD Pipeline on Self-Hosted Runner

on: [push]

jobs:
  build-and-deploy:
    runs-on: [self-hosted, linux, docker] # <-- Target your self-hosted runner
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm install

      - name: Build project
        run: npm run build

      - name: Deploy to server (example)
        if: github.ref == 'refs/heads/main' # Only deploy from the main branch
        uses: appleboy/ssh-action@master
        with:
          host: ${{ secrets.SSH_HOST }}
          username: ${{ secrets.SSH_USERNAME }}
          key: ${{ secrets.SSH_PRIVATE_KEY }}
          script: |
            cd /path/to/your/app
            git pull origin main
            npm install --production
            npm run build
            pm2 restart your-app-name # Example for Node.js app managed by PM2
Enter fullscreen mode Exit fullscreen mode

Explanation:

  • runs-on: [self-hosted, linux, docker]: This tells GitHub Actions to find a runner that has all these labels. If you only used self-hosted, it would work, but using more specific labels gives you better control if you have multiple types of self-hosted runners.
  • Steps: Each step represents a task. We check out the code, set up Node.js, install dependencies, build the project, and then deploy.
  • Deployment Step: The example shows a deployment using appleboy/ssh-action. This requires you to set up SSH access from your runner to your deployment server and store your SSH credentials as repository secrets.
    • Repository Secrets: In your GitHub repository's "Settings" -> "Secrets and variables" -> "Actions," you'll need to add:
      • SSH_HOST: The IP address or hostname of your deployment server.
      • SSH_USERNAME: The username for SSH login.
      • SSH_PRIVATE_KEY: Your private SSH key (ensure your public key is in ~/.ssh/authorized_keys on the deployment server).

Managing Your Self-Hosted Runner Environment

Running your CI/CD jobs on your own server gives you immense flexibility.

Dockerizing Your Build Environment

If your project has specific dependencies or requires a particular operating system environment, you can use Docker on your self-hosted runner. Instead of installing Node.js or Python directly on the VPS, you can use a Docker image that already has them.

Modify your workflow to use Docker:

name: CI/CD Pipeline with Docker Runner

on: [push]

jobs:
  build-and-deploy:
    runs-on: [self-hosted, linux, docker]
    container: # <-- Specify the container to run the job in
      image: node:20-alpine # Use an official Node.js Docker image
      options: --user node # Run as a non-root user

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Install dependencies
        run: npm install

      - name: Build project
        run: npm run build

      # ... deployment steps ...
Enter fullscreen mode Exit fullscreen mode

This ensures that your build always runs in a clean, consistent Node.js 20 Alpine Linux environment, regardless of what else is installed on your VPS.

Accessing Other Services

Your self-hosted runner can access other services on your VPS or your local network. This is particularly useful for:

  • Databases: Running database migrations directly on your staging or production database.
  • Private Registries: Pulling private Docker images or packages.
  • Internal APIs: Testing integrations with internal services.

Risk Warning: When your CI/CD pipeline has direct access to sensitive environments like production databases or internal services, the risk of accidental data corruption or unauthorized access increases. Always implement robust access controls, use dedicated service accounts with minimal privileges, and ensure your build scripts are thoroughly tested.

Resource Management and Monitoring

With your own VPS, you are responsible for its resources.

  • CPU and RAM: Monitor your VPS's CPU and RAM usage. If your builds are consistently maxing out resources, you might need to upgrade your VPS plan or optimize your build process. Tools like htop on your server can provide real-time insights.
  • Disk Space: Ensure you have enough disk space for code checkouts, dependencies, build artifacts, and Docker images.
  • Runner Health: Monitor the self-hosted runner process. If it crashes, you'll need to restart it. Running it as a systemd service (as shown earlier with svc.sh) helps with this.

Benefits of This Approach

  • Cost Savings: For significant build volume, a VPS can be more cost-effective than paying for extra build minutes on cloud platforms.
  • Unlimited Build Minutes: You're only limited by your VPS's resources, not a predetermined minute count.
  • Full Control: Customize your build environment, install any software, and configure networking exactly as you need.
  • Security: Keep your build processes within your own infrastructure, which can be advantageous for sensitive projects.
  • Private Networking: Easily connect to internal services that are not exposed to the public internet.

Potential Downsides and Mitigation

  • Maintenance Overhead: You are responsible for server upkeep, security patching, and ensuring the runner is always running.
    • Mitigation: Automate as much as possible (e.g., security updates, runner restarts). Consider managed VPS services that offer some level of support.
  • Single Point of Failure: If your VPS goes down, your CI/CD pipeline stops.

Top comments (0)