DEV Community

hyuga
hyuga

Posted on

572 tests green, and three packages were one command from shipping broken

#node #javascript #testing #npm

I publish seven packages out of one monorepo. Last week, with 572 tests passing, typecheck clean, lint clean and the tarball-contents check green, three of them were one command away from being published in a state that would fail on install.

The bug is boring. The reason nothing caught it is not.

What happened

I added one subpath export to the shared package, @hyuga/spar:

{
  "exports": {
    ".": "./src/spar.mjs",
    "./cli": "./src/cli.mjs"
  }
}
Enter fullscreen mode Exit fullscreen mode

Then I changed the three packages that depend on it to use that subpath:

import { runDirectly, readStdin } from '@hyuga/spar/cli';
Enter fullscreen mode Exit fullscreen mode

And I forgot to raise the dependency floor:

"dependencies": { "@hyuga/spar": "^0.1.0" }
Enter fullscreen mode Exit fullscreen mode

^0.1.0 resolves to the 0.1.0 already on the registry, whose exports are:

$ npm view @hyuga/spar@0.1.0 exports
{ '.': './src/spar.mjs' }
Enter fullscreen mode Exit fullscreen mode

No ./cli. Three lines to reproduce:

mkdir t && cd t && npm init -y
npm i @hyuga/spar@0.1.0
node -e "import('@hyuga/spar/cli')"
# ERR_PACKAGE_PATH_NOT_EXPORTED - Package subpath './cli' is not defined by "exports"
Enter fullscreen mode Exit fullscreen mode

It cannot reproduce in the workspace. npm install symlinks all seven packages to each other, so @hyuga/spar always resolves to the working tree. Of course the tests passed.

Why nothing surfaced it

These three run as hooks. Throwing inside a hook takes the user's session with it, so they are all written like this:

try {
  if (sub === 'pre') emit('PreToolUse', check(payload));
} catch {
  // a limiter that breaks the session is worse than no limiter
}
Enter fullscreen mode Exit fullscreen mode

I still think that is the right call. It also swallows ERR_PACKAGE_PATH_NOT_EXPORTED.

From outside: no error, no warning, exit 0, a counter that stays at zero forever, and a part meant to keep a copy of your draft before it is overwritten that keeps nothing. It reads as "a quiet day." You find out when you go looking for a draft that was never saved.

What was green

check result
node --test, 7 packages, 572 tests pass
tsc --noEmit pass
biome lint pass
Node 18 / 20 / 22 / 24 matrix pass
integration against real MySQL 8.4 and PostgreSQL 16 pass
npm pack --dry-run contents (no config / dump / env) pass
tag vs package.json vs src/version.ts pass

Every one of them runs inside the working tree. That is the whole problem. It was not that I had too few checks — it was that all of them were checking the wrong place.

The check I added

scripts/smoke-install.mjs, and it is exactly what it sounds like:

  1. npm pack
  2. make an empty directory
  3. npm i ./the-tarball.tgz there — letting npm resolve dependencies from the registry
  4. import() every subpath the package declares in exports
  5. run every command in bin with --help

Step 3's parenthesis is the entire point: no workspace links.

It failed immediately:

@hyuga/redline@0.2.0
  ✗ npm install from the tarball
      npm error code ETARGET
      npm error notarget No matching version found for @hyuga/spar@^0.2.0.
Enter fullscreen mode Exit fullscreen mode

A side effect worth having: the publish order is now enforced rather than documented. Publishing redline before spar reaches the registry fails with ETARGET. It used to be a sentence in RELEASING.md.

Then the check was wrong. Twice.

It stopped a release and explained nothing

First run in CI:

✗ npm pack
    undefined
Enter fullscreen mode Exit fullscreen mode

I had written the destination path already-quoted, because npm is a .cmd on Windows — and since Node 20, spawning a .cmd without a shell fails with EINVAL, so it needs shell: true, which needs the quotes. Linux runs it with shell: false, where those quote marks are just characters in a directory name.

So: a check whose entire purpose is "what works in here is not what happens out there" failed for exactly that reason. Written on Windows, first run on Linux.

The fix moves quoting inside the runner, which is the only place that knows whether a shell is involved:

const WINDOWS = process.platform === 'win32';
const quote = (a) => (/[\s"]/.test(a) ? `"${a.replace(/"/g, '\\"')}"` : a);
const runNpm = (args, opts = {}) => (WINDOWS
  ? run('npm.cmd', args.map(quote), { shell: true, ...opts })
  : run('npm', args, { shell: false, ...opts }));
Enter fullscreen mode Exit fullscreen mode

I also fixed the undefined. A gate that halts a release and says nothing is worse than the bug it caught — the next person's first move is not to read it, it is to delete it.

It called correct behaviour a defect

Last package:

✗ import '@hyuga/llm-safe-sql/mysql'
    Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'mysql2'
Enter fullscreen mode Exit fullscreen mode

Nothing wrong with the package. mysql2 is an optional peer: you install it if you want the MySQL adapter, and you never touch that subpath if you don't.

"peerDependencies": { "mysql2": ">=3.9.0", "pg": ">=8.11.0" },
"peerDependenciesMeta": { "mysql2": { "optional": true }, "pg": { "optional": true } }
Enter fullscreen mode Exit fullscreen mode

My check imported every declared subpath against a bare install, so it was calling documented behaviour a bug. Peers now get installed alongside:

const peers = Object.keys(manifest.peerDependencies ?? {});
runNpm(['i', `./${tarball}`, ...peers, ...]);
Enter fullscreen mode Exit fullscreen mode

This is the failure mode that actually matters. A gate that cries wolf gets switched off, and after it is switched off, the one call that should have stopped you looks exactly like the nineteen that should not have.

Where it ended up

Twelve subpaths and eight bins, all checked against a registry install:

@hyuga/llm-safe-sql@0.10.1
  · peers installed: mysql2, pg
  ✓ import @hyuga/llm-safe-sql
  ✓ import @hyuga/llm-safe-sql/mysql
  ✓ import @hyuga/llm-safe-sql/postgres
  ✓ import @hyuga/llm-safe-sql/sqlite
  ✓ import @hyuga/llm-safe-sql/mcp
  ✓ llm-safe-sql --help
  ✓ llm-safe-sql-mcp --help
Enter fullscreen mode Exit fullscreen mode

All seven went out through it.

Takeaways

  • Green means "it worked under the conditions I built," not "it works." If you have workspaces, a monorepo, or path mapping, those conditions differ from your users' by construction.
  • A catch you added for a good reason still makes everything that dies inside it invisible. Being right about the catch does not help. Keep a list of the places your system is designed to stay quiet.
  • Anything that can stop a release must say why it stopped. A check that does not is a check somebody deletes.
  • A false positive can cost more than a miss. A miss loses one case. A false positive loses the check, and therefore every case after it.

Who this is likely to hit

  • npm / pnpm / yarn workspaces monorepos
  • packages with subpath exports consumed by a sibling package in the same repo
  • TypeScript paths or project references doing the resolving
  • anything with optional peer dependencies

If you are not reinstalling the tarball in CI, you are probably not checking this. I wasn't.

The script is under 100 lines: smoke-install.mjs. MIT — take it.

Top comments (0)