Dynamic Config in CircleCI: Skipping Untouched Services in a Monorepo Pipeline

Most monorepo teams on CircleCI start with one workflow that runs every job on every push: lint, test, and build for every service, every time. It's simple and it works — until the repo grows past four or five services and a one-line README fix in services/docs triggers a full test run of services/billing, services/auth, and services/notifications anyway. Multiply that by every PR and every push-to-fix-the-push, and you're paying compute for work that provably can't have changed.

The usual first fix is a shell script that greps git diff and exits early inside each job. It's a trap. Skipping inside a job still spins up the executor, checks out the repo, and restores caches before the skip check runs — you've paid most of the cost already, and CircleCI still bills and reports it as a run. What you actually want is for jobs to never be scheduled in the first place. That's what CircleCI's dynamic config feature is for, and it's underused because the setup has a few sharp edges that aren't obvious from the docs.

The shape of the solution

Dynamic config splits your pipeline into two stages. A small setup workflow runs first, figures out what changed, and then generates (or selects) the real config for this run — including deciding which service workflows even exist. CircleCI calls the second stage via the continuation orb, passing it a config file and a parameter payload built by the first stage.

Concretely, for a monorepo with services/api, services/web, and services/worker, plus a shared packages/core library, the setup stage should determine:

1. Did packages/core change? If yes, treat it as "everything changed" — a shared dependency bump can silently break any service.
2. Otherwise, which of services/* changed, based on a diff against the correct base revision (more on this below).
3. Emit a small JSON/YAML parameter set describing which service pipelines to include, and hand it to continuation/continue.

The setup config

# .circleci/config.yml
version: 2.1

setup: true

orbs:
  path-filtering: circleci/path-filtering@1.1.0
  continuation: circleci/continuation@1.0.0

workflows:
  setup-workflow:
    jobs:
      - path-filtering/filter:
          base-revision: main
          config-path: .circleci/continue-config.yml
          mapping: |
            packages/core/.* run-api true
            packages/core/.* run-web true
            packages/core/.* run-worker true
            services/api/.* run-api true
            services/web/.* run-web true
            services/worker/.* run-worker true

The path-filtering/filter job does three things in one step: diffs the branch against base-revision, matches changed paths against the mapping regexes, and calls continuation/continue with the resulting booleans as pipeline parameters — run-api, run-web, run-worker. Notice packages/core/.* maps to all three flags. That's the shared-dependency safety net: a change anywhere under packages/core forces every service's flag to true, so you never get a false-negative skip on a library bump.

The continued config

.circleci/continue-config.yml is a normal CircleCI config, except each service workflow is gated behind the parameter the setup stage computed:

version: 2.1

parameters:
  run-api:
    type: boolean
    default: false
  run-web:
    type: boolean
    default: false
  run-worker:
    type: boolean
    default: false

jobs:
  test-api:
    docker:
      - image: cimg/node:20.11
    steps:
      - checkout
      - run: cd services/api && npm ci && npm test

workflows:
  api:
    when: << pipeline.parameters.run-api >>
    jobs:
      - test-api
  web:
    when: << pipeline.parameters.run-web >>
    jobs:
      - test-web
  worker:
    when: << pipeline.parameters.run-worker >>
    jobs:
      - test-worker

A when: << pipeline.parameters.run-api >> on the workflow, not the job, is the detail that actually saves money — CircleCI never schedules the workflow at all when the parameter is false, so there's no executor spin-up, no checkout, nothing billed for that service on this run.

The base-revision trap that breaks this on merge

The most common failure mode isn't in the YAML — it's base-revision resolving to the wrong commit. path-filtering/filter diffs your branch against base-revision (default main) to find changed files. On a normal feature branch this works fine. It breaks in two specific situations:

• Squash-merge workflows, where main and your feature branch share no common ancestor after a rewrite — the diff falls back to comparing against the branch's first commit, which can pick up unrelated files.
• Pipelines triggered directly on main (post-merge builds), where diffing main against itself returns an empty changeset and every service flag comes back false — silently skipping a build that should run everything.

The fix for the second case is to special-case builds on the default branch: set all run-* flags to true unconditionally when pipeline.git.branch == "main", bypassing path-filtering entirely for post-merge runs. This is a few lines added to the mapping logic or, more reliably, a small shell step before path-filtering/filter that short-circuits with a fixed parameter file when on main.

What this actually saves

On a five-service repo where a typical PR touches one or two services, this pattern took a team's median pipeline duration from roughly 14 minutes (full fan-out, five services in parallel but each paying checkout/install/cache-restore) down to about 5 minutes for single-service PRs, with credit consumption dropping proportionally — most PRs stopped paying for services they never touched. The setup workflow itself adds one short job (checkout plus a diff, typically under 20 seconds) to every run, which is the fixed cost of the approach.

Where it stops paying off

Dynamic config adds a layer of indirection: debugging "why didn't my job run" now means checking the setup workflow's output before you even look at the service job. For repos with two or three services where most PRs touch all of them anyway, the added complexity isn't worth it — path-filtering earns its keep specifically when the repo has enough independent services that most changes are narrow, and enough shared code that you still need the "anything shared changed → run everything" fallback to keep it safe.