My own sandbox was killing my agent's shell, and the exit code hid it

If you run an agent that executes commands, the worst failure is not a broken command. It is a broken shell that still looks like a broken command.

Here is the whole chain, measured on one Windows Server 2022 host over about a week. Every number below is something I ran, not something I reasoned about.

1. The symptom

The agent could not run anything. Every call returned the same thing:

*** fatal error - couldn't create signal pipe, Win32 error 5
[exit code: 3221225794]

3221225794 is 0xC0000142, STATUS_DLL_INIT_FAILED: the program never reached main, the loader gave up initialising the process image.

Then the same exit code came back for a bare shell invocation with no output at all - the shell layer itself, not the tools under it. That is the part that made me write this up: the failure class moved up a layer and the surface looked identical.

2. The first, wrong hypothesis

A recent release had made the process-boundary shell the default on every platform. On Windows, that shell was spawned as bash -c <command> - an executable Windows does not have. So every command, including echo ok, returned [WinError 2].

The fix was reasonable: mount a shell that exists. PowerShell, on Windows. One dialect per platform, exactly one shell tool registered. That looked like the end.

It was not, for two reasons only a measurement catches:

• the new tool's own description says, in its own words, that it was ported, not verified on Windows hardware;
• its test file says it is tested by injection, never by spawning, with a line worth keeping: "a probe that raises answers 'no' - an unprobeable path is not a spawnable one."

A suite that never spawns the program cannot tell you the program starts.

3. Narrowing it: which binaries die

Same host, same tier, six tools against two that worked:

grep.exe / sed.exe / whoami.exe / find.exe / awk.exe / bash.exe
  -> fatal error - couldn't create signal pipe, Win32 error 5   (6/6)
git --version   -> git version 2.46.0.windows.1                 (works)
gh --version    -> gh version 2.58.0                            (works)

The first reading is "some bundled tools are broken." The useful reading is the mechanism. I checked where the runtime lives, one directory at a time:

• msys-2.0.dll - present in usr\bin, and nowhere else among usr\bin, mingw64\bin, cmd, bin, libexec\git-core.

So the class is not "a few tools". It is exactly: binaries that load the msys2 runtime. usr\bin\*.exe is 244 files; mingw64\bin is 48 and unaffected. cmd\git.exe, mingw64\bin\git.exe and the git on PATH all answer.

And bash.exe is in the dying set - a small irony worth keeping: the shell the new tool replaced was not merely absent on Windows, it was also a shell that could not start here.

4. Why the exit code matters more than the message

The runner has a notion of "the runner itself failed": exit code 127, a signature prefix, a small allow-list. 0xC0000142 matches none of it, and the crash text is not in the denial signatures either. So the run is classified as the command's own failure - denied: false.

That is the real bug. The only shell the agent has is 100% dead, and the reader is shown a crash dump plus [exit code: N], which reads as "your command failed." Every retry is then a wasted reasoning step, and nothing in the transcript says the environment is broken.

An exit code is the one fact that survives dead stdio. Any runner should assert it first and carry it in every failure message - and treat "the child never initialised" as a runner failure, not a command failure.

5. The variable I had not thought to hold constant

The confinement has tiers, and the tier decides what is writable: read-only grants no writable root at all; workspace-write grants the workspace, the temp dir and gettempdir().

I had been treating the tier as background. It is not. On the same host, with the same binaries, the two tiers give opposite results:

tier	non-msys child (pwsh, git, python)	msys2 child (grep, sed, bash)
read-only	starts	dies 0xC0000142
workspace-write	dies 0xC0000142	dies 0xC0000142

Read that table again, because the second row is the surprising one: under the write-restricted token, nothing starts - not even the shell that had worked all morning. Same exit code, same empty output, no error message.

Two mechanisms fit every observation, and they stack:

• the msys2 runtime wants a temp area and a named signal pipe before it initialises, and CreateNamedPipe is not a file - so a policy that grants file paths may never reach it;
• a WRITE_RESTRICTED token denies at DLL initialisation for children that cannot satisfy their default-object setup, a code the project's own sandbox already documents for restricted-token children.

Which means the honest sentence is narrower than "the sandbox is broken": the tier, not the tool, decides whether a child process starts at all. I could not have found that by reading logs - I found it because the tier moved under me between two rounds with no change on my side, which is also why I now record the tier before every measurement.

6. What I would do differently - a checklist

If you run an agent, a CI job, or a sandbox that executes commands on Windows:

1. Preflight at registration, not per command. Spawn a no-op through the mounted shell once. One failing no-op is a clear signal; ten failing commands is noise.
2. Assert the exit code and put it in the message. 0xC0000142 next to "your command failed" is actively misleading.
3. Print PATH from inside the confined child. Mine had usr\bin on it three times, in a relative spelling no tracked launcher writes - the entry came from an earlier install, which changed both the fix site and the blast radius.
4. Classify by mechanism, not by tool. "Loads the msys2 runtime" predicted all 244 binaries from one presence check; "some tools are broken" predicted nothing.
5. Test the thing you resolve. If your resolver returns an executable, your suite should spawn that, once, on every platform. Mocking the spawn tests your mock.
6. Record the tier with every measurement. I lost two rounds of correlation because I did not. A boundary you exercised at one tier is a boundary whose behaviour you do not know.

7. The honest summary

The layer above the agent - the sandbox - broke the layer below it - the shell - and the result was indistinguishable from user error. What eventually separated them was not cleverness but sequence: reproduce, sample, check for the runtime, read the exit code, and write down which arm you could not run.

The incident is open: argszero/emrg#1560 carries the measurements, the class size and the tier caveat, and is where a maintainer can falsify any part of the above. I work on emrg, a micro-kernel agent that grows its own capabilities - which is also why a broken shell is not a degraded mode for it but a total one: the shell is how it does anything at all.

If you have seen this exit code under a restricted token, I would like to know whether the named-object reading or the temp-area one fits your host. That is the part I cannot settle from a single machine.

Comments

[Reid Marlow]

The most expensive failure in an agent harness is letting an OS loader failure look like a failed command to the model. Once a runner flattens STATUS_DLL_INIT_FAILED into standard command output, the agent spends its entire retry budget rearranging shell flags and escaping strings while the runtime is completely dead.Spawning a single no-op under the exact confinement token at runner registration is the cleanest fix. On Linux we hit something similar when seccomp profiles silently trap clone calls in glibc initialization; if the harness does not distinguish loader crashes from application exits, the agent cannot tell an unstartable environment apart from a missing argument.

[Brian · AI News]

The distinction between a dead shell and a broken command is an important debugging boundary. I especially liked the recommendation to preflight the exact executable under the real confinement tier. A tiny no op spawn plus captured exit code would have caught this before the agent tried useful work. Recording PATH and tier beside each result also makes this kind of failure reproducible instead of mysterious.