DEV Community

Daniel Pertu
Daniel Pertu

Posted on

fs.cpSync copied the framework and left every symlink pointing at the cache

Notifio is a desktop app that watches rental search pages and tells you the moment something new appears. It drives a Playwright-managed Chromium in the background, and that browser is bundled into the installer. I wrote about getting it in there before: Shipping Playwright's Chromium inside a packaged Electron app.

This is the part of that story I did not know about yet, because it takes a few weeks to show up.

The failure

The prebuild step copies Chromium out of the local ms-playwright cache into app/playwright-browsers/ so electron-builder can bundle it. That worked. Builds shipped. Then one day, on a machine where nothing about the build had changed, packaging failed several minutes in:

ENOENT: no such file or directory,
  lstat '/Users/daniel/Library/Caches/ms-playwright/chromium-1228/chrome-mac-arm64/...'
Enter fullscreen mode Exit fullscreen mode

A path inside the cache. During a step that reads from the staged copy. The only thing that had happened in between was a routine clear-out of ~/Library/Caches.

What a .framework actually is

On macOS a framework is not a directory of files, it is a directory of symlinks over one real version:

Chromium Framework.framework/
  Versions/
    A/                      <- the real payload
    Current -> A
  Resources  -> Versions/Current/Resources
  Libraries  -> Versions/Current/Libraries
  Helpers    -> Versions/Current/Helpers
  Chromium Framework -> Versions/Current/Chromium Framework
Enter fullscreen mode Exit fullscreen mode

In Playwright's own cache, those links are stored as absolute paths. And fs.cpSync does not follow symlinks, it copies the link itself, verbatim. So a staged copy made with:

fs.cpSync(src, dst, { recursive: true });
Enter fullscreen mode Exit fullscreen mode

contains a perfect copy of the payload plus a handful of links that still say /Users/you/Library/Caches/ms-playwright/....

Every one of those links resolves, right up until the cache is cleaned. Then the staged copy is a broken framework, and the build that consumes it fails at packaging time, which is the most expensive minute to fail in.

Why not just dereference the copy

fs.cpSync takes dereference: true, and my first instinct was to use it. It is the wrong answer here for two reasons. The whole point of Versions/Current is that the payload exists once, so following the links copies the framework body several times over, and this is a browser, not a small library. More importantly the symlink layout is the structure macOS expects: code signing and the dynamic loader both walk it, and a "framework" that is really four independent copies of the same binary is not a framework any more.

The links are not the problem. Their being absolute is.

The fix is four lines in the middle of a walk

Every target already exists inside the staged copy. So rewrite each absolute link as the relative path a normal framework uses:

const links = findSymlinks(ROOT);

for (const link of links) {
  const target = fs.readlinkSync(link);
  if (!path.isAbsolute(target)) continue;

  const staged = toStagedPath(target);
  if (!staged) {
    console.warn(`[fix-symlinks] Cannot place target, leaving alone: ${link}`);
    continue;
  }
  if (!fs.existsSync(staged)) {
    console.error(`[fix-symlinks] Target missing from the staged copy: ${staged}`);
    broken++;
    continue;
  }

  const relative = path.relative(path.dirname(link), staged);
  fs.unlinkSync(link);
  fs.symlinkSync(relative, link);
  fixed++;
}
Enter fullscreen mode Exit fullscreen mode

The interesting function is the one that decides where a cache path lands in the staged tree:

/**
 * Keyed on the "chromium-<revision>" segment rather than on the cache location,
 * so it works whatever the cache directory is called on this machine.
 */
function toStagedPath(target) {
  const parts = target.split(path.sep);
  const index = parts.findIndex((p) => /^chromium[-_]/.test(p));
  if (index === -1) return null;
  return path.join(ROOT, ...parts.slice(index));
}
Enter fullscreen mode Exit fullscreen mode

It would have been shorter to string-replace the cache directory with the staging directory. That version works on my machine and on nobody else's: the cache lives under ~/Library/Caches on macOS, ~/.cache on Linux and %LOCALAPPDATA% on Windows, and it can be moved with an environment variable. Finding the revision segment and keeping everything after it does not care where the link came from, only where it is going.

Three smaller decisions in that script I would repeat:

Idempotent by construction. Links that are already relative are skipped, so the script is safe to run at any point, and it runs as part of prebuild rather than as something you remember to do before a release.

A missing target is a hard failure with the command to fix it. If a link points at something genuinely not staged, exiting 0 would hand a broken framework to the packager. So it exits 1 and says Re-download the browser: npx playwright install chromium. The error message is the documentation for this script.

It reports the boring case. All 10 link(s) already relative. tells you the step ran and found nothing to do, which is a different log line from the step not running.

The second bug the first one was hiding

While I was in there: copy-browsers.js had a comment at the top claiming it was safe to re-run because it skips folders that already exist. It did skip them, and then it checked for the cache anyway and exited 1 if it was missing. On a machine with both browsers already staged and a cleaned cache, a step that had nothing to do failed the build.

The fix is to work out what is missing before deciding what is required:

const staged = subFolders.filter((sub) => fs.existsSync(path.join(dest, sub)));
const needed = subFolders.filter((sub) => !staged.includes(sub));

if (needed.length === 0) {
  console.log('[copy-browsers] All browsers already staged, nothing to copy.');
  process.exit(0);
}

if (!fs.existsSync(srcBase)) {
  console.error(`[copy-browsers] Missing from ${dest}: ${needed.join(', ')}`);
  console.error(`[copy-browsers] Source not found: ${srcBase}`);
  console.error('[copy-browsers] Run: npx playwright install chromium');
  process.exit(1);
}
Enter fullscreen mode Exit fullscreen mode

Both bugs are the same bug wearing different clothes: the build had an undeclared dependency on a cache directory, and nothing noticed because the cache is always there until it is not.

The check worth stealing

If your build stages anything by copying it, the staged copy is not self-contained until nothing inside it points outside itself. That is one command:

find app/playwright-browsers -type l -lname '/*'
Enter fullscreen mode Exit fullscreen mode

Any output is a file that will work on your machine and break on a build agent, or next month, whichever comes first. Run it in CI and you find out at the copy step instead of at the signing step.

See what it is all in service of

The bundled browser exists so that checks run on your own machine against the real search results page, with no email pipeline in the middle. The live demos of that are on the Notifio home page, the installers those per-arch DMGs turn into are at notifio.app/download, and if you want the reasoning about why a locally polled page beats a portal's own alert email, that is notifio.app/guides/how-to-be-first-to-a-rental-listing.

Top comments (0)