DEV Community

Cover image for 🦊 GitLab Pages per Branch: The No-Compromise Hack to Serve Preview Pages
Benoit COUETIL 💫 for Zenika

Posted on • Updated on

🦊 GitLab Pages per Branch: The No-Compromise Hack to Serve Preview Pages

Initial thoughts

With GitLab Pages, you can publish static websites directly from a repository in GitLab. By default, we cannot have preview pages: if a job deploys the pages, this overwrites previous content, which disallows preview mode.

Some article on the internet show how to get around that with artifacts, knowing that GitLab can display artifacts. But this trick has disadvantages, mainly highly technical links, that have to be shared again after changes on the branch.

Some Github gist points into the right direction, but in a complex way and with too few side features.

In this article, we will get around the limitation by taking advantage of the cache mechanism, and be able to display per-branch content, with the side benefit of obfuscating the path to ephemeral branches content, if desired.

The solution can be broken down to these steps:

  • Generate files for current branch
  • Get previous branches generation from GitLab cache
  • Merge and update cache
  • Auto delete obsolete cache on branch deletion, using GitLab environments

Prerequisites

We assume you already have a way of generating your HTML static content, and just want to serve the files using GitLab Pages.

For the code to work, your cache must be centralized, either by using gitlab.com runners, by having a single runner, or by sharing caches between multiple private runners.

You need to accept a global cache by deactivating the GiLab option Use separate caches for protected branches in Settings -> CICD -> General Pipelines.

((orange PiratePunkAI fox reading a book in a tree)), (piles of books), <lora:PiratePunkAIV4-000004:1>, <lora:blindbox_v1_mix:1>, caribbean sea background

The GitLab pipeline code

workflow:
  rules: # disable tag pipelines and duplicate MR pipelines
    - if: $CI_COMMIT_BRANCH

variables:
  MAIN_BRANCH_PATH: "."
  EPHEMERAL_BRANCHES_PATH: preview # subpath to ephemeral branches content for preview

pages:
  stage: build
  image: alpine:3.18
  cache:
    key: gitlab-pages
    paths: [public]
  before_script:
    # default available 'tree' app in alpine image does not work as intended
    - apk add tree
    # CURRENT_CONTENT_PATH is defined in rules, different between main branch and ephemeral branches
    - mkdir -p public/$CURRENT_CONTENT_PATH && ls public/$CURRENT_CONTENT_PATH/..
    - | # avoid deleting main branch content when cache has been erased
      if [ "$CI_COMMIT_BRANCH" != "$CI_DEFAULT_BRANCH" ] && [ ! -f public/$MAIN_BRANCH_PATH/index.html ]; then
        echo -e "💥\e[91;1m Unable to retrieve $CI_DEFAULT_BRANCH generated files from cache ; please regenerate $CI_DEFAULT_BRANCH files first\e[0m"
        exit 1
      fi
    - rm -rf public/$CURRENT_CONTENT_PATH || true # remove last version of current branch
  script:
    - ./generate-my-html.sh --output public/$CURRENT_CONTENT_PATH || true # insert here your code that generates documentation
    - cd public/$EPHEMERAL_BRANCHES_PATH
    - tree -d -H '.' -L 1 --noreport --charset utf-8 -T "Versions" -o index.html # generate a root HTML listing all previews for easier access
  environment:
    name: pages/$CI_COMMIT_BRANCH
    action: start
    url: $CI_PAGES_URL/$CURRENT_CONTENT_PATH
    on_stop: pages-clean-preview
  rules:
    # default branch is exposed at GitLab Pages root
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      variables:
        CURRENT_CONTENT_PATH: $MAIN_BRANCH_PATH
    # other (short-lived) branches generation are exposed in 'EPHEMERAL_BRANCHES_PATH/branch-name-sanitized' sub path
    - variables:
        CURRENT_CONTENT_PATH: $EPHEMERAL_BRANCHES_PATH/$CI_COMMIT_REF_SLUG
  artifacts:
    paths: [public]
    expire_in: 1h

pages-clean-preview:
  stage: build
  image: alpine:3.18
  cache:
    key: gitlab-pages
    paths: [public]
  variables:
    GIT_STRATEGY: none # git files not available after branch deletion
    FOLDER_TO_DELETE: $EPHEMERAL_BRANCHES_PATH/$CI_COMMIT_REF_SLUG # an indirection to allow arbitrary deletion when launching this job
  script:
    - rm -rf public/$FOLDER_TO_DELETE
  environment:
    name: pages/$CI_COMMIT_BRANCH
    action: stop
  rules:
    - if: $CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH
      when: manual
      allow_failure: true
Enter fullscreen mode Exit fullscreen mode

((orange PiratePunkAI fox reading a book in a tree)), (piles of books), <lora:PiratePunkAIV4-000004:1>, <lora:blindbox_v1_mix:1>, caribbean sea background

Integrated features

The above code has below features:

  • main content is exposed on $CI_PAGES_URL and the path is configurable with $CURRENT_CONTENT_PATH
  • Per-branch preview content is exposed on $CI_PAGES_URL/preview, with a homepage to easily navigate to branches content
  • Path to root preview folder is configurable with $EPHEMERAL_BRANCHES_PATH variable to hide preview content by obfuscation
  • Generated pages are associated with environments to take advantage of auto-cleaning on branch deletion
  • To avoid disturbing already existing environments, pages environment are placed under a pages folder
  • If main content has not been generated in current cache, or if the cache has been deleted, an error is triggered, to avoid accidental deletion
  • Deletion job can be triggered manually with any cache path as input, to clean outdated data
  • Code can safely be added to an existing project pipeline without causing trouble with already existing jobs
  • The workflow:rules can be deleted if you already have your own, or updated to match your flow
  • The job must be named pages and the artifact must be a public folder to be deployed to GitLab Pages (or you can use the pages:publish keyword)

Remove obsolete folders after concurrency problems

If you experience concurrency problems leading to obsolete folders from deleted branches, you have multiple choices:

  • Manually launch pages-clean-preview with a folder name
  • Manually clear the GitLab cache, regenerate content for default branch, and let other branches be updated on their future pipelines
  • Use the resource_group keyword
    • This may slow down your pipelines, waiting to acquire the lock
  pages:
    [...]
    resource_group: avoid-cache-racing-conditions

  pages-clean-preview:
    [...]
    resource_group: avoid-cache-racing-conditions
Enter fullscreen mode Exit fullscreen mode
  • Delete obsolete folders using git branch at the end of pages jobs:

    variables:
      GIT_DEPTH: 0 # all branches are fetched (and known)
    script:
      [...]
      - apk add git
      - git branch -r
      - cd $EPHEMERAL_BRANCHES_PATH
      - |
      for folder in *; do
          exists=$(git rev-parse -q --verify "origin/$folder") || true
          if [ -z "$exists" ]; then
          rm -rf $folder
          echo "removed folder $folder, the branch does not exist anymore"
          fi
      done
      [...]
    

Wrapping up

Given the piece of yaml provided, using it in your pipeline, you should be able to share your GitLab Pages on a per-branch basis, on the path you want, while serving the stable content from the root context.

For any question or remark, please use below comment section 🤓.

If you need information to choose better runner architecture, you can read GitLab Runners topologies: pros and cons.

((orange PiratePunkAI fox reading a book in a tree)), (piles of books), <lora:PiratePunkAIV4-000004:1>, <lora:blindbox_v1_mix:1>, caribbean sea background

Illustrations generated locally by Automatic1111 using RevAnimated model with PiratePunkAI and Blindbox LoRA

Further reading

Top comments (4)

Collapse
 
edupton profile image
Edward Upton

In pages-clean-preview job, should it be:

FOLDER_TO_DELETE: preview/$CI_COMMIT_REF_SLUG
Enter fullscreen mode Exit fullscreen mode
Collapse
 
bcouetil profile image
Benoit COUETIL 💫

Thanks, nice catch ! I only tested when the commit branch name is kebab-case.

There should also be another fix, 'preview' is parameterized as 'EPHEMERAL_BRANCHES_PATH' :

FOLDER_TO_DELETE: $EPHEMERAL_BRANCHES_PATH/$CI_COMMIT_REF_SLUG
Enter fullscreen mode Exit fullscreen mode

I'm updating right away 🔧

Collapse
 
lohoj41531 profile image
loho • Edited

In before_script we should be checking for ! -d public instead of ! -d public/$CI_DEFAULT_BRANCH right? Since we are putting the content of default branch in root of public directory.

Collapse
 
bcouetil profile image
Benoit COUETIL 💫 • Edited

You are right, another difference with my usual setup (I usually have the main branch generated files in a the common subfolders).

I'm even replacing -d public/$CI_DEFAULT_BRANCH with -f public/$MAIN_BRANCH_PATH/index.html, more solid proof that something has been generated !

What do you think ?