DEV Community

Cover image for I filed it as a flaky test. It was a crash that try/catch can't reach.
אחיה כהן
אחיה כהן

Posted on

I filed it as a flaky test. It was a crash that try/catch can't reach.

Safari MCP, the open-source server I maintain, runs one process that owns a local bridge port. Any other instance becomes a secondary: it proxies through that primary, and if the primary goes away, it takes the port over. One test covers exactly that handoff. Start a primary, start a secondary, stop the primary, wait for the secondary to say it's listening.

On October 1 that test failed once on the Node 24 macOS runner, in a pull request run. The rerun passed, Node 20 and 22 passed, and I filed issue #140 as a CI flake. The last line I wrote there was: "Filed to track the flake and the Node version it needs; no code change in this issue yet."

Two days later the same test failed again, this time on main, a few minutes after the pull request with the same code had gone green on the same Node version.

What the secondary died of

The secondary never printed its listening line. Its stderr ended like this:

Uncaught: Error: setTypeOfService EINVAL
    at Socket.setTypeOfService (node:net:911:13)
    at writeH1 (node:internal/deps/undici/undici:8022:16)
    at Object.write (node:internal/deps/undici/undici:7758:18)
    at _resume (node:internal/deps/undici/undici:9656:54)
Enter fullscreen mode Exit fullscreen mode

The code that was supposed to handle a missing primary looks like every health check you have written:

try {
  const res = await fetch(`http://127.0.0.1:${port}/proxy-check`, {
    signal: AbortSignal.timeout(2000),
  });
  // ...
} catch {
  // Nothing answers on the bridge port: the primary is gone. Take it over.
  takeOverThePort();
}
Enter fullscreen mode Exit fullscreen mode

When the primary is gone, fetch() should reject and the catch should run. Instead the process ended.

Why the catch never ran

The undici that ships inside Node 24 calls socket.setTypeOfService() on every HTTP/1.1 request it writes. It's a quality-of-service hint and harmless almost everywhere. On macOS, though, the kernel refuses that option on a socket whose peer has just reset the connection. The call fails with EINVAL, and Node turns that into a thrown error.

The throw happens inside undici's write path, which runs from a socket I/O callback. Nothing there routes the error into the promise that fetch() returned, so the try/catch around await fetch() never sees it. It becomes an uncaught exception, and an uncaught exception ends the process.

So the crash needs three things at once: Node 24, macOS, and a peer that resets at the wrong moment. The test stops the primary a moment after the secondary comes up, close to the secondary's first check, which is about as close to the wrong moment as a test gets. In real use the race is rarer. But a secondary that dies exactly when it should take over is the failure the whole handoff exists to prevent.

Fixed upstream, just not in Node 24

undici fixed it in nodejs/undici#5547, "fix(h1): ignore type of service errors", released in undici 8.8.0. It skips the call when nobody asked for a type of service, and it ignores the error when the call fails. Node 24 didn't get that change.

I measured the difference by counting calls to setTypeOfService during one plain fetch() to a local server:

Node calls per fetch
24.21.0 (newest 24.x today) 1
26.10.0 0

I'm not the only one who hit this. get-bb/bb, telomi and t3code each have a pull request for the same uncaught EINVAL.

You can't schedule a race, so fail at the boundary instead

My first instinct was to reproduce the real thing: a keep-alive server in a child process, one request to pool the socket, then a reset or a kill, then a second request onto the dead socket. Two harnesses and six timing variants on Node 24.21.0 produced zero EINVALs. The kernel race simply didn't happen on my machine, which is also why the test passed nearly every time in CI.

What worked was failing at the exact boundary instead of waiting for the race. Replace the native method with one that throws what macOS throws, then let undici's own write path call it:

import { Socket } from "node:net";
import http from "node:http";

Socket.prototype.setTypeOfService = function () {
  throw Object.assign(new Error("setTypeOfService EINVAL"), { code: "EINVAL" });
};

const server = http.createServer((req, res) => res.end("ok")).listen(0, "127.0.0.1");
await new Promise((resolve) => server.on("listening", resolve));
try {
  const res = await fetch(`http://127.0.0.1:${server.address().port}/`);
  console.log("resolved:", await res.text());
} catch (err) {
  console.log("rejected:", err.message);
}
Enter fullscreen mode Exit fullscreen mode

On Node 24.21.0 that script prints neither line. It dies on the uncaught EINVAL with exit code 1, try/catch and all. On Node 26.10.0 it prints resolved: ok, because undici there never makes the call.

The fix

Upgrading Node is the real fix, but I don't pick my users' Node version, and the package supports 20 and up. So the server wraps the method once at startup and ignores EINVAL only:

export function ignoreTypeOfServiceEinval(proto) {
  const original = proto.setTypeOfService;
  if (typeof original !== "function" || original.ignoresEinval) return;
  const wrapped = function (...args) {
    try {
      return original.apply(this, args);
    } catch (err) {
      if (err?.code === "EINVAL") return this;
      throw err;
    }
  };
  wrapped.ignoresEinval = true;
  proto.setTypeOfService = wrapped;
}
Enter fullscreen mode Exit fullscreen mode

With the error swallowed, undici carries on writing to a socket that's gone, the write fails the normal way, and fetch() rejects into the catch that was there all along. Any other error from the method still throws. On Node 20 and 22 the method doesn't exist, so the wrapper does nothing.

The test suite now runs the script above in a child process with the wrapper applied and asserts that the fetch resolves. I checked that the test can fail: with the wrapper replaced by a no-op, the child exits 1 on Node 24.21.0 and the test goes red. With the wrapper it passes, on my machine and on Node 24.20.0 in CI. The fix shipped in v2.22.10.

What I got wrong

The label. A test that fails because the process under test died of an uncaught exception isn't flaky in any sense that matters. The code crashed, and the test just couldn't make it crash often. "Flaky" told me to wait and see, and waiting meant the second report came from main instead of from me.

If your project runs on Node 24 on macOS and calls fetch() against something that can go away under you (a local daemon, a sidecar, a dev server you restart), search your crash logs for setTypeOfService EINVAL. It won't show up as a rejected request. It shows up as a process that isn't running anymore.

Top comments (0)