Mastering Monorepos in GitLab CI/CD: Dynamic Child Pipelines & Path-Based Triggers

When you consolidate dozens of microservices, frontend packages, AI agents, and shared libraries into a single Monorepo, code sharing and atomic refactoring become effortless.

However, your CI/CD pipeline often turns into a bottleneck:

1. The 3,000-Line YAML Monster: A single root .gitlab-ci.yml becomes impossible to read or maintain.
2. Wasted Compute: Changing a single line of CSS in services/web-portal accidentally triggers a 20-minute C++/CUDA compilation in services/inference-engine.
3. Job Collisions: Different teams step on each other's global variables and stage definitions.

GitLab CI/CD solves the monorepo scaling problem natively with two powerhouse features:

1. Parent-Child Pipelines with rules:changes: Isolates each sub-project’s CI configuration into its own directory and triggers it only when files in that path are modified.
2. Dynamic Child Pipeline Generation: Uses a lightweight script to inspect the git diff at runtime and programmatically generate the exact CI jobs needed for that commit.

In this architectural guide, I will show you how to implement both patterns to keep your monorepo builds blazingly fast.

---

1. The Architecture: Monolithic vs. Parent-Child Pipelines

+-------------------------------------------------------------+
|                   Root Monorepo (.gitlab-ci.yml)            |
|          Inspects git diff across 50+ sub-projects          |
+-------------------------------------------------------------+
         |                         |                         |
  (Changed: /ai-agent)     (Unchanged: /web-ui)     (Changed: /cuda-core)
         |                         |                         |
         v                         v                         v
+-------------------+     +-----------------+     +--------------------+
| Trigger Child CI  |     |   SKIPPED (0s)  |     |  Trigger Child CI  |
| ai-agent/.ci.yml  |     |  Zero resources |     | cuda-core/.ci.yml  |
+-------------------+     +-----------------+     +--------------------+

---

2. Pattern 1: Modular Parent-Child Pipelines with rules:changes

Instead of stuffing every job into the root .gitlab-ci.yml, place a dedicated .gitlab-ci.yml inside each sub-project folder:

monorepo/
├── .gitlab-ci.yml                 # Root orchestrator (50 lines)
├── packages/
│   ├── ai-gateway/
│   │   ├── .gitlab-ci.yml         # Python / FastAPI pipeline
│   │   └── pyproject.toml
│   ├── web-dashboard/
│   │   ├── .gitlab-ci.yml         # Next.js / TypeScript pipeline
│   │   └── package.json
│   └── gpu-runtime/
│       ├── .gitlab-ci.yml         # C++20 / CUDA pipeline
│       └── CMakeLists.txt

The Root .gitlab-ci.yml:

stages:
  - triggers

trigger_ai_gateway:
  stage: triggers
  trigger:
    include: packages/ai-gateway/.gitlab-ci.yml
    strategy: depend # Root pipeline reflects child success/failure status
  rules:
    - changes:
        - packages/ai-gateway/**/*

trigger_web_dashboard:
  stage: triggers
  trigger:
    include: packages/web-dashboard/.gitlab-ci.yml
    strategy: depend
  rules:
    - changes:
        - packages/web-dashboard/**/*

trigger_gpu_runtime:
  stage: triggers
  trigger:
    include: packages/gpu-runtime/.gitlab-ci.yml
    strategy: depend
  rules:
    - changes:
        - packages/gpu-runtime/**/*

Why strategy: depend is Crucial:

By default, GitLab marks a trigger job as successful the moment the child pipeline starts. Adding strategy: depend forces the parent pipeline to wait for the child pipeline to finish and mirrors its exact pass/fail status on your Merge Request!

---

3. Pattern 2: Dynamic Child Pipelines (For 50+ Projects)

When your monorepo grows to 50+ projects, even writing 50 trigger_* blocks in the root .gitlab-ci.yml becomes repetitive.

With Dynamic Child Pipelines, a tiny Python or shell script inspects the modified directories and generates a dynamic-child-pipeline.yml artifact on the fly!

Step A: Write the Generator Script (scripts/generate_ci.py)

import os
import subprocess
import yaml

def get_changed_projects():
    # Compare current commit against target branch or previous commit
    target = os.environ.get("CI_MERGE_REQUEST_TARGET_BRANCH_NAME", "HEAD~1")
    cmd = ["git", "diff", "--name-only", f"origin/{target}...HEAD"]

    try:
        files = subprocess.check_output(cmd, text=True).splitlines()
    except Exception:
        files = subprocess.check_output(["git", "diff", "--name-only", "HEAD~1"], text=True).splitlines()

    projects = set()
    for f in files:
        parts = f.split("/")
        if len(parts) >= 2 and parts[0] == "projects":
            projects.add(parts[1])
    return sorted(projects)

def build_pipeline(projects):
    if not projects:
        # Fallback no-op job if only root README changed
        return {
            "stages": ["noop"],
            "nothing_to_build": {
                "stage": "noop",
                "script": ["echo 'No sub-projects modified. Skipping builds.'"]
            }
        }

    pipeline = {"stages": ["test", "build"]}
    for proj in projects:
        pipeline[f"test_{proj}"] = {
            "stage": "test",
            "image": "alpine:latest",
            "script": [
                f"echo 'Running test suite for projects/{proj}...'",
                f"cd projects/{proj} && ./test.sh"
            ]
        }
    return pipeline

if __name__ == "__main__":
    changed = get_changed_projects()
    config = build_pipeline(changed)
    with open("generated-config.yml", "w") as f:
        yaml.dump(config, f, sort_keys=False)

---

Step B: Execute & Trigger in Root .gitlab-ci.yml

stages:
  - setup
  - dynamic_execution

generate_pipeline:
  stage: setup
  image: python:3.12-slim
  before_script:
    - apt-get update && apt-get install -y git
    - pip install pyyaml
  script:
    - python scripts/generate_ci.py
    - cat generated-config.yml
  artifacts:
    paths:
      - generated-config.yml
    expire_in: 1 hour

run_dynamic_child_pipeline:
  stage: dynamic_execution
  needs: [generate_pipeline]
  trigger:
    include:
      - artifact: generated-config.yml
        job: generate_pipeline
    strategy: depend

---

4. What Happens in the GitLab UI?

When you open a Merge Request that touches 2 out of 52 projects:

1. generate_pipeline runs in 4 seconds, outputting a tailored YAML containing only those 2 projects.
2. run_dynamic_child_pipeline spawns a visual downstream child graph in the GitLab Pipeline UI.
3. Only the 2 modified projects are built and tested. The remaining 50 projects consume zero runner minutes.

---

5. Key Takeaways for Monorepo Architects

1. Decouple Ownership: Each sub-project folder owns its .gitlab-ci.yml, Docker image, and variables without polluting global state.
2. Use changes: with Care: Combine rules:changes with if: $CI_PIPELINE_SOURCE == "merge_request_event" for deterministic git diff evaluation.
3. Generate Dynamically at Scale: When managing dozens of packages, generate the child YAML programmatically via artifact: includes.

---

Resources & Links

• GitLab Profile: gitlab.com/hrlpavan
• GitLab Contributor Profile: contributors.gitlab.com/users/hrlpavan
• GitHub: @hrlpavan

Do you prefer multi-repo setups or monorepos for your engineering projects? Share your architecture thoughts in the comments below!