Cache Key Fallback Chains: Fixing All-or-Nothing Caching in Monorepo CircleCI Pipelines

Most CircleCI monorepo setups cache dependencies with a single key built from one lockfile checksum:

- restore_cache:
    keys:
      - v1-deps-{{ checksum "pnpm-lock.yaml" }}

This works fine for a single-package repo. In a monorepo it quietly becomes one of the biggest sources of wasted CI minutes on the account, because a root pnpm-lock.yaml changes on every PR that touches any package's dependencies — even a docs-only bump in packages/docs. The cache key for packages/api's test job goes stale even though nothing packages/api depends on actually changed. You get a full cold install on jobs that didn't need one, multiplied across every affected job in the workflow.

The fix isn't a bigger cache — it's a better key. This walkthrough builds a layered cache-key strategy for a pnpm workspace monorepo on CircleCI: per-package content hashes instead of a single root hash, and a restore_cache fallback chain that degrades gracefully instead of missing entirely.

Why the single-key approach fails

checksum "pnpm-lock.yaml" hashes the entire lockfile. pnpm's workspace lockfile format interleaves every package's resolved dependency tree into one file, so touching packages/web/package.json rewrites pnpm-lock.yaml in a way that changes the checksum for the whole repo — including sections that describe packages/api, which didn't change at all. CircleCI's cache key has no concept of "this part of the file didn't change"; it's a flat string comparison.

The result: cache hit rate scales inversely with the number of packages you have. A 20-package monorepo where PRs typically touch 1-2 packages will see full cache misses on 90%+ of CI runs, because some package's dependency changed almost every time.

Step 1: generate a per-package dependency manifest

Instead of hashing the whole lockfile, extract just the resolved dependency subset for the package a given job actually builds, using pnpm list scoped with --filter:

#!/usr/bin/env bash
# scripts/ci/package-manifest.sh <package-dir>
set -euo pipefail
PKG_DIR="$1"
pnpm --filter "./${PKG_DIR}..." list --depth=Infinity --json > /tmp/manifest.json
sha256sum /tmp/manifest.json | awk '{print $1}'

The ... suffix tells pnpm to include the package's transitive workspace dependencies too — important, because if packages/api depends on packages/shared-utils, a change to shared-utils should still invalidate api's cache. This is the part teams get wrong when they hash package.json directly: it misses transitive workspace deps entirely.

Commit this script and call it from each job rather than duplicating the logic inline in YAML.

Step 2: build a fallback chain, not a single key

CircleCI's restore_cache already supports multiple keys, checked in order, with prefix matching on partial keys. Most teams only ever supply one exact key and never use the fallback behavior:

jobs:
  test-api:
    docker:
      - image: cimg/node:20.11-browsers
    steps:
      - checkout
      - run:
          name: Compute package cache key
          command: |
            echo "$(./scripts/ci/package-manifest.sh packages/api)" > /tmp/pkg-hash.txt
      - restore_cache:
          keys:
            - v2-api-deps-{{ checksum "/tmp/pkg-hash.txt" }}
            - v2-api-deps-
            - v2-deps-
      - run: pnpm --filter ./packages/api... install --frozen-lockfile
      - save_cache:
          key: v2-api-deps-{{ checksum "/tmp/pkg-hash.txt" }}
          paths:
            - node_modules
            - packages/api/node_modules

The three keys form a deliberate degradation path:

1. v2-api-deps-{hash} — exact match, no install work needed beyond pnpm install's own integrity check.
2. v2-api-deps- — prefix match against the most recent cache for this package, even if the hash changed. pnpm's install is incremental against an existing node_modules, so restoring a near-miss cache and letting pnpm install --frozen-lockfile reconcile the diff is dramatically faster than installing from zero.
3. v2-deps- — a repo-wide fallback populated by whichever job runs first in the workflow, useful for shared low-level packages that rarely diverge.

Without the fallback tiers, a single hash miss means a full cold install identical to having no cache at all. With them, you're paying for a diff-sized install in the common case.

Step 3: only run jobs for packages that changed

Layered cache keys reduce the cost of a job running; you still don't want to run test-api on a PR that never touched packages/api or its dependencies. Pair the caching change with CircleCI's dynamic config and the circleci/path-filtering orb so the workflow itself is generated based on git diff:

# .circleci/config.yml (setup config)
version: 2.1
setup: true
orbs:
  path-filtering: circleci/path-filtering@1.1.0
workflows:
  determine-scope:
    jobs:
      - path-filtering/filter:
          base-revision: main
          config-path: .circleci/continue-config.yml
          mapping: |
            packages/api/.* run-api true
            packages/web/.* run-web true
            packages/shared-utils/.* run-api true
            packages/shared-utils/.* run-web true

Note that packages/shared-utils maps to both downstream jobs — this is where the transitive-dependency awareness from Step 1 and the path-filtering mapping need to agree. If they drift out of sync (someone adds a new internal package dependency but forgets to update the mapping), you'll get stale test results that silently pass because the job never ran. Treat the mapping file and the workspace dependency graph as one artifact — a small script that reads pnpm -r list --json and generates the mapping automatically is worth writing once you have more than five or six internal packages, rather than hand-maintaining the regex list.

What this actually buys you

On a 24-package pnpm/Turborepo monorepo we migrated with this pattern, average install time per CI job dropped from roughly 90 seconds (cold install) to 12-18 seconds (fallback-tier restore plus incremental reconcile) on typical single-package PRs, and jobs unrelated to the changed packages stopped running at all. The v2- key prefix versioning also matters in practice: when you change what a job installs (a new build tool, a Node version bump), bump the prefix once so you don't restore a cache shaped for the old toolchain and get confusing partial-match failures.

The underlying idea generalizes past pnpm: any monorepo tool with a workspace-aware dependency lister (Cargo workspaces, Go modules with go list, Gradle's dependencies task) can feed the same manifest-hash-plus-fallback-chain pattern. The lockfile checksum trick from CircleCI's own docs examples is a fine starting point for a single-package repo; it's the wrong tool once you have more than a handful of independently deployable packages sharing one CI config.