Create and configure Your own GitLab Runner automatically on Gitpod

In this blog post, I will explain why and how I installed a GitLab Runner on a Gitpod workspace 😃

🔎 GitLab & Gitpod

I use GitLab all the time for my personal projects. I can save my code, get history and automate tasks like building or deploying my applications with GitLabCI. If this is your first time hearing about GitLab or GitLabCI, you can discover it by reading 📖 my series of Cheatsheets 😀.

Last year, I discovered by chance Gitpod, a Cloud IDE available on GitHub, BitBucket and GitLab. Going to one of my projects on GitLab, I saw an arrow near the “Web IDE” button. After testing this tool, I quickly adopted it!

Since then, I use this Cloud IDE all the time for my personal or open-source projects to develop new features, fix bugs or simply create or update documentation. No more time wasted to install a lot of jdk versions, switch versions of maven, node, or anything else depending on your project.

If you haven’t tried this tool yet, let’s go! You can install the Gitpod chrome extension which allows you to have a Gitpod button when you are on a GitLab or GitHub project!

👷 My work

One month ago, I was working on a personal project and especially on my continuous integration pipeline that needed to be improved. I wanted to create a new job to test new GitLab features. To test my pipeline, I followed this process:

git add .gitlab-ci.yml
git commit -m “feat: create new job to [...]”
git push 

After applying these simple commands, I looked at the pipeline generated in the GitLab interface and my job was validated, great news!

Next, I wanted to complexify my job. I added some elements in the .gitlab-ci.yml file and I repeated the previous Git commands. But this time, damned! There was an error on my pipeline. Not in the structure of the file, but in my commands written in the scripts tag. No worry, I added changes, I repeated the previous git commands and… I always had an error.

Please don’t judge me, I’m sure your pipeline doesn’t work on the first time (or second) 😅.

Finally after some tries, I succeeded in having a pipeline which makes what I want.

But something was wrong. My different attempts, almost all errors, were wasted time. Currently, GitLab offers 400 minutes for your CI/CD. I would like it to be used to build, test, check security and deploy my application, not my tests! It’s a “wasted time”. So how do I resolve this?

💡 A new opportunity?

Let’s take a step back. Pipelines are executed by a GitLab Runner, that’s right. This is an application written in Go and it’s easy to install on different infrastructures. That reminds me of this 📖 blog post I’ve written (in French 🇫🇷) about the installation of a runner on my raspberry PI. To install a runner, you have to follow these steps:

• download binaries
• install them
• register an executor

You can also use the available Docker image with this command :

docker container run -d --name gitlab-runner --restart always \
    -v /srv/gitlab-runner/config:/etc/gitlab-runner \
    -v /var/run/docker.sock:/var/run/docker.sock \
    gitlab/gitlab-runner:latest

You can find more information in the 📖 GitLab documentation.

On my Gitpod workspace, I am in a container context. So I can install my GitLab Runner in my workspace 💡! Let’s try it!

🦊 My new GitLab Runner

First, I get my registration token in the Setting > CI/CD > Runners on the specific part.

After opening a Gitpod workspace on my project, I can install a GitLab Runner with this Docker command. The default image of a workspace is gitpod/workspace-full which contains many developer tools.

docker container run -d --name gitlab-runner --restart always \
    -v /srv/gitlab-runner/config:/etc/gitlab-runner \
    -v /var/run/docker.sock:/var/run/docker.sock \
    gitlab/gitlab-runner:latest

I registered a runner with this command. (I replaced the registration-token with mine).

docker container run --rm -it -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner register \
    --non-interactive \
    --executor "docker" \
    --docker-image ruby:2.7 \
    --url "https://gitlab.com/" \
    --registration-token "<gitlab runner token" \
    --description "gitpod-runner" \
    --tag-list "gitpod" \
    --access-level="not_protected"

At this step, if I write a docker container ls command, I see one container gitlab-runner.

For the token, to avoid getting this one in the project, it can be exported on the Gitpod variables. First, go to my profile on gitpod.io and in the Settingsmenu.

To create this variable, I added a name, a value and a scope representing my project.

Now, my variable is created. In my .gitpod.yml file, I can get the value of this variable by writing $MY_CICD_REGISTRATION_TOKEN.

🎲 The .gitpod.yml file

The .gitpod.ymlfile contains all specifications to customize a workspace. For this experiment, all commands can be written in the tasks section. We can assign a name to task:

tasks:
  - name: install Gitlab runner

In Gitpod, there are three types of tasks: before, init and command. For Docker instructions, heavy commands, I use the inittask. To write a sequence of commands on multiple lines, the | operator can be used:

tasks:
  - name: install Gitlab runner
    init: | 
      echo '🦊 Installation GitLab Runner'

And then I just had to add Docker commands:

tasks:
  - name: install Gitlab runner
    init: |
      echo "🦊 Installation GitLab Runner"
      docker container run -d --name gitlab-runner --restart always \
        -v /srv/gitlab-runner/config:/etc/gitlab-runner \
        -v /var/run/docker.sock:/var/run/docker.sock \
        gitlab/gitlab-runner:latest

      docker container run --rm -it -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner register \
        --non-interactive \
        --executor "docker" \
        --docker-image ruby:2.7 \
        --url "https://gitlab.com/" \
        --registration-token $MY_CICD_REGISTRATION_TOKEN \
        --description "gitpod-runner" \
        --tag-list "gitpod" \
        --access-level="not_protected"

After committing and pushing this file on my project, when I opened a new Gitpod workspace, tasks were executed and this good message appear:

The GitLab Runner is installed and registered! 🎉

As I registered my GitLab Runner with a tag-list property containing the value gitpod, jobs have to require this tag to be executed by this runner.

stages:
    - build
    - test
    - deploy

.gitpod:
  **tags:**
    - **gitpod**

my-job-1:
  stage: build 
  extends: [".gitpod"]
  script:
    - echo "my job 1"

To verify that my pipeline was working, I ran a new one and checked the details:

On my Gitpod workspace, I checked the Docker logs of the gitlab-runner container:

We could see the runner id “z8crb8” visible on the job and the Docker logs: my pipeline was executed in my Gitpod workspace !

🏁 The result

To resume, my .gitpod.yml file looks like:

tasks:
  - name: install Gitlab runner
    init: | 
      echo '🦊 Installation GitLab Runner'

      docker container run -d --name gitlab-runner --restart always \
        -v /srv/gitlab-runner/config:/etc/gitlab-runner \
        -v /var/run/docker.sock:/var/run/docker.sock \
        gitlab/gitlab-runner:latest

      docker container run --rm -it -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner register \
        --non-interactive \
        --executor "docker" \
        --docker-image ruby:2.7 \
        --url "https://gitlab.com/" \
        --registration-token $MY_CICD_REGISTRATION_TOKEN \
        --description "gitpod-runner" \
        --tag-list "gitpod" \
        --access-level="not_protected"

And all my jobs contain a new tag :

tags:
    - gitpod

All sources are available on this GitLab repository.

💭This is the beginning

This test is a beginning. I had this idea and I tested it quickly. I think this approach is interesting, because each person working on a project can use a dedicated and temporary GitLab runner and can take advantage of Gitpod infrastructure to execute pipelines. We can imagine, for example, executing a pipeline at the opening of a Gitpod workspace to launch some operations such as checking security or other tools to help developers.

In this test, the next thing to review is tags tag in the .gitlab-ci.yml file. Currently the tag is fixed but this should only be present for development mode. Maybe a script in the Gitpod tasks to add this to workspace initialization?

The good thing is that attempts of pipeline modification don’t reduce the GitLab CI/CD time limit. We can imagine this can attract novices in GitLabCI or developers who hesitate, fearing of breaking something, to add changes in pipelines. For my personal case, I create a new runner on each my new Gitpod workspace when I want to test some features on GitLabCI 😁

If you have any comment or suggestion about this experience, don’t hesitate to send me a direct message on Twitter 🤘

Comments

[Devarshi Shimpi]

I was trying this since a long time!! Great Article!!