<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Space App</title>
    <description>The latest articles on DEV Community by Space App (@space_app_fc7f6872808dc84).</description>
    <link>https://dev.to/space_app_fc7f6872808dc84</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4186545%2Fed2da106-0e27-4d3c-83b1-91de11522f4a.png</url>
      <title>DEV Community: Space App</title>
      <link>https://dev.to/space_app_fc7f6872808dc84</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/space_app_fc7f6872808dc84"/>
    <language>en</language>
    <item>
      <title>Building a multi-pane workspace for coding agents: PTYs, replay, and the backpressure we have not solved</title>
      <dc:creator>Space App</dc:creator>
      <pubDate>Fri, 06 Nov 2026 16:06:15 +0000</pubDate>
      <link>https://dev.to/space_app_fc7f6872808dc84/building-a-multi-pane-workspace-for-coding-agents-ptys-replay-and-the-backpressure-we-have-not-4ak5</link>
      <guid>https://dev.to/space_app_fc7f6872808dc84/building-a-multi-pane-workspace-for-coding-agents-ptys-replay-and-the-backpressure-we-have-not-4ak5</guid>
      <description>&lt;p&gt;SpaceApp is a self-hosted workspace that runs AI coding CLIs side by side in the browser, each in its own pane. I build it, it's MIT-licensed, and this post is the terminal side of it as it stands in the public release (v1.1.9): how a pane gets a real PTY, how output gets from that PTY to xterm.js and back after a reconnect, what broke on the way, and what we still haven't solved. Every number below comes from the code; file names are in the public repo so you can check them.&lt;/p&gt;

&lt;h2&gt;
  
  
  Two containers, one socket between them
&lt;/h2&gt;

&lt;p&gt;A Docker install made with &lt;code&gt;npx --yes run-spaceapp@latest install&lt;/code&gt; runs a small compose project. Two services matter here:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;spaceapp-core&lt;/code&gt; runs the web app, the API and the WebSocket. It listens on &lt;code&gt;127.0.0.1:4911&lt;/code&gt; by default and mounts no provider keys or login volume.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;spaceapp-cli&lt;/code&gt; owns every PTY. It runs the pane host (&lt;code&gt;packages/cli-host&lt;/code&gt;) as an unprivileged user (uid 10001).&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;They talk over a Unix socket on a shared volume (&lt;code&gt;/run/spaceapp-cli/host.sock&lt;/code&gt;, mode 0660). PostgreSQL holds the rest, and a browser container is optional.&lt;/p&gt;

&lt;p&gt;The split pays off on the first restart: restarting the core doesn't kill your CLIs, because they are processes of another container.&lt;/p&gt;

&lt;h2&gt;
  
  
  One PTY per pane
&lt;/h2&gt;

&lt;p&gt;Each pane is one &lt;code&gt;node-pty&lt;/code&gt; process (&lt;code&gt;packages/cli-host/src/node-pty-spawn.ts&lt;/code&gt;):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="k"&gt;export&lt;/span&gt; &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;spawnNodePty&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;CliHostSpawn&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;spec&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt;
  &lt;span class="nf"&gt;spawn&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;/bin/sh&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;-c&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;/usr/bin/stty -isig; exec "$@"&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;space-cli-host&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;spec&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;command&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;...&lt;/span&gt;&lt;span class="nx"&gt;spec&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;args&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;xterm-256color&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;cols&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;spec&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;cols&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;rows&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;spec&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;rows&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;cwd&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;spec&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;cwd&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;env&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;spec&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;env&lt;/span&gt; &lt;span class="p"&gt;});&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That &lt;code&gt;stty -isig&lt;/code&gt; is there because of a bug. Without it, Ctrl-C in a pane became SIGINT and killed the whole CLI process tree. With signals off on the PTY, Ctrl-C reaches the CLI as the byte 0x03, which is the CLI's own cancel key.&lt;/p&gt;

&lt;p&gt;The rest of the lifecycle lives in &lt;code&gt;session-registry.ts&lt;/code&gt;:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Sessions are keyed by a CLI session id. Each spawn gets a generation id and a monotonically increasing output sequence number, which is what makes replay possible later.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;kill()&lt;/code&gt; in node-pty only sends SIGHUP and swallows errors, and we had process trees that survived it. The registry now escalates: SIGHUP, then SIGTERM, then SIGKILL on the whole process group (the PTY child is a session leader, so &lt;code&gt;kill(-pid, SIGKILL)&lt;/code&gt; reaches everything under it).&lt;/li&gt;
&lt;li&gt;Some exit events never arrive. A sweep checks &lt;code&gt;process.kill(pid, 0)&lt;/code&gt; and reconciles them.&lt;/li&gt;
&lt;li&gt;A session with no attachment is reaped after 30 minutes by default (swept every 60 s). When you come back, the pane host recreates it and resumes. So, honestly: an agent does not keep running forever after the last browser tab on it closes. Warm rooms, below, change when that happens.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  The pane host protocol
&lt;/h2&gt;

&lt;p&gt;The API and the pane host speak length-prefixed JSON over the socket (&lt;code&gt;framing.ts&lt;/code&gt;): a 4-byte big-endian length, then UTF-8 JSON, 8 MiB at most per frame. The methods are &lt;code&gt;inspect&lt;/code&gt;, &lt;code&gt;attach&lt;/code&gt;, &lt;code&gt;input&lt;/code&gt;, &lt;code&gt;resize&lt;/code&gt;, &lt;code&gt;detach&lt;/code&gt;, &lt;code&gt;terminate&lt;/code&gt;, &lt;code&gt;reapDetached&lt;/code&gt; and &lt;code&gt;ping&lt;/code&gt;, and requests on one connection run strictly in order through a promise queue.&lt;/p&gt;

&lt;p&gt;Two details turned out to matter.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Replay from a ring buffer.&lt;/strong&gt; Output is chunked to 8,000 characters and kept per session in an 8 MiB ring buffer. An attach says "everything after sequence N". If those events already left the ring, the host says so instead of pretending:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;earliestSequence&lt;/span&gt; &lt;span class="o"&gt;!==&lt;/span&gt; &lt;span class="kc"&gt;undefined&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nx"&gt;afterSequence&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="nx"&gt;earliestSequence&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;throw&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;CliHostError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;CLI_HOST_REPLAY_GAP&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;`CLI host output before sequence &lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;earliestSequence&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt; is no longer buffered …`&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Lanes.&lt;/strong&gt; Strict ordering per connection means one slow session could delay every other session on that connection. So the API opens 32 connections to the host and pins each session to one with an FNV-1a hash of its id. One session's queue never sits in front of another's.&lt;/p&gt;

&lt;h2&gt;
  
  
  The API: two paths for the same output
&lt;/h2&gt;

&lt;p&gt;When output arrives from the host, the API does two things with it (&lt;code&gt;apps/api/src/cli-terminal.ts&lt;/code&gt;):&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;It sends it straight to the connected browsers.&lt;/li&gt;
&lt;li&gt;Separately, it batches it for PostgreSQL, every 20 ms or 32 KiB, into a transcript table. Before anything is stored, it goes through pattern redaction (API keys of the common shapes, Bearer and Basic headers, session cookies).&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;So there are two stores: the host's ring buffer, which is fast and gone if the host restarts, and the transcript, which survives. A reconnect replays the stored transcript and yields to the event loop every 16 frames, so a long replay doesn't freeze the API for everyone else. Redaction applies to what is stored; the live stream your own browser receives is the raw output.&lt;/p&gt;

&lt;h2&gt;
  
  
  The WebSocket
&lt;/h2&gt;

&lt;p&gt;Each pane's terminal is one WebSocket at &lt;code&gt;/api/panes/:id/cli/terminal&lt;/code&gt;:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;It opens with a one-time ticket (32 random bytes, base64url, with a TTL).&lt;/li&gt;
&lt;li&gt;Every frame is a JSON text frame, validated with zod on both sides. Clients send &lt;code&gt;input&lt;/code&gt; (up to 1,000,000 characters), &lt;code&gt;resize&lt;/code&gt; (2-400 columns, 2-200 rows), &lt;code&gt;interrupt&lt;/code&gt;, &lt;code&gt;ping&lt;/code&gt; and control messages. The server sends &lt;code&gt;output&lt;/code&gt;, &lt;code&gt;status&lt;/code&gt;, &lt;code&gt;ready&lt;/code&gt;, &lt;code&gt;error&lt;/code&gt;, &lt;code&gt;pong&lt;/code&gt; and &lt;code&gt;session_replaced&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;Output is split into frames of at most 8 KiB, cut at code-point boundaries so a split never breaks a multi-byte character.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;permessage-deflate&lt;/code&gt; is on above 1 KiB, and &lt;code&gt;maxPayload&lt;/code&gt; is 16 MiB.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Several tabs can watch the same pane, but only one can type: the holder of a control lease. The lease lasts 30 s. We first renewed it at 30 s, which raced the TTL and dropped keystrokes in a live session (20 September). The heartbeat is now capped at 15 s.&lt;/p&gt;

&lt;h2&gt;
  
  
  xterm.js in the browser
&lt;/h2&gt;

&lt;p&gt;Each pane is one &lt;code&gt;@xterm/xterm&lt;/code&gt; 6 terminal with 2,000 lines of scrollback. Two settings come with stories:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;The terminal starts at the PTY's stored size, not xterm's default, so a replay isn't reflowed through the wrong geometry first.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;convertEol&lt;/code&gt; stays off. The PTY's termios already handles line endings, and turning it on corrupted the cursor positioning of full-screen TUIs.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Writes are paced in the browser. Only one &lt;code&gt;terminal.write(chunk, callback)&lt;/code&gt; is in flight at a time, and the next chunk goes after xterm's callback. Chunks are at most 1 MiB, with a yield between them.&lt;/p&gt;

&lt;p&gt;Hidden and minimized panes don't parse anything. Their output is queued in JavaScript until the pane becomes visible. Past 128 KiB for one pane, or 1 MiB in total, the app records a pressure diagnostic. The queue itself isn't dropped at those sizes, so it is a warning, not a cap.&lt;/p&gt;

&lt;p&gt;Then there are warm rooms. Up to 6 recently used rooms stay mounted with live sockets, up to 96 connected panes (24 on mobile). We used to park hidden sockets after 30 s to save resources. Every return to a room then needed a fresh ticket, an xterm rebuild and a full replay. Parking is off now, and leaving a room no longer rebuilds its terminals. A room that falls out of the six disposes its terminals and every addon properly.&lt;/p&gt;

&lt;p&gt;One more fix that you only find by running real CLIs: OpenCode's TUI renderer sometimes drops the &lt;code&gt;ESC[&lt;/code&gt; introducer between colour runs, and xterm then prints the raw parameters as text. A small repair parser in the output coordinator puts the introducer back.&lt;/p&gt;

&lt;h2&gt;
  
  
  What we haven't solved: backpressure
&lt;/h2&gt;

&lt;p&gt;This is the part I'd most like opinions on. There is no end-to-end backpressure:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;The server doesn't check &lt;code&gt;bufferedAmount&lt;/code&gt; and doesn't pause the PTY when a browser is slow. It just sends.&lt;/li&gt;
&lt;li&gt;On the host socket, &lt;code&gt;socket.write&lt;/code&gt; is called without checking its return value.&lt;/li&gt;
&lt;li&gt;What we do instead: 8 KiB frames, one xterm write in flight, 1 MiB chunks with yields, hidden panes that queue instead of parsing, and a replay that yields every 16 frames.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;That holds up for normal agent output. A CLI that dumps a huge log while a browser on a slow link is attached is exactly the case it doesn't cover. Pausing the PTY when a socket's buffer grows would be the honest fix; the open question is how it should behave when several tabs watch the same pane at different speeds.&lt;/p&gt;

&lt;h2&gt;
  
  
  Isolation, precisely
&lt;/h2&gt;

&lt;p&gt;People ask whether panes are sandboxed. They are not, and the README says so: rooms organize sessions; they are not a security boundary. Precisely:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Core, CLI, browser and PostgreSQL are separate containers on one compose network. All run with &lt;code&gt;no-new-privileges&lt;/code&gt;, and none gets the Docker socket.&lt;/li&gt;
&lt;li&gt;Inside the CLI container, every pane is a process of the same user. Each provider gets its own &lt;code&gt;HOME&lt;/code&gt;, &lt;code&gt;TMPDIR&lt;/code&gt; and config root with &lt;code&gt;umask 077&lt;/code&gt;. That separates configuration, not security: one CLI could read another CLI's files.&lt;/li&gt;
&lt;li&gt;Host folders are mounted only after you add them as workspaces (&lt;code&gt;run-spaceapp workspace add&lt;/code&gt;). An opt-in host-root mode on Linux mounts &lt;code&gt;/&lt;/code&gt; read-write into the CLI container. It does what it says, so read the docs before you turn it on.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Caps and sizes
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;16 panes per room, enforced by a database check (&lt;code&gt;pane_cap BETWEEN 1 AND 16&lt;/code&gt;) and the schema.&lt;/li&gt;
&lt;li&gt;6 rooms by default, configurable up to 16.&lt;/li&gt;
&lt;li&gt;All panes share one CLI container with a memory and CPU cap: 2 GB / 1.5 CPUs on the small profile, up to 4 GB / 3 CPUs on the large one. The host should have at least 4 CPUs and about 8 GB of RAM. It isn't lightweight, and I'd rather say that up front.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Things that broke, in one list
&lt;/h2&gt;

&lt;p&gt;Each of these is written in a comment next to its fix in the public code:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Two OpenCode 2 &lt;code&gt;--standalone&lt;/code&gt; servers on one store failed to start. Now there is one shared &lt;code&gt;opencode serve&lt;/code&gt; per CLI container, with a lock and a health check, and panes attach to it.&lt;/li&gt;
&lt;li&gt;OpenCode's update dialog opened over a fresh pane and swallowed the user's first prompt. Auto-update is off inside the container.&lt;/li&gt;
&lt;li&gt;Ctrl-C killed the whole CLI tree. Fixed with &lt;code&gt;stty -isig&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;node-pty &lt;code&gt;kill()&lt;/code&gt; left process trees alive. Fixed with process-group SIGKILL escalation.&lt;/li&gt;
&lt;li&gt;Parking hidden sockets forced a rebuild and a replay on every room switch. Parking is off.&lt;/li&gt;
&lt;li&gt;Renewing the control lease at its TTL dropped keystrokes. The heartbeat is 15 s.&lt;/li&gt;
&lt;li&gt;A TUI renderer dropped CSI introducers. A repair parser restores them.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  Try it, or tell me how you'd do backpressure
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx &lt;span class="nt"&gt;--yes&lt;/span&gt; run-spaceapp@latest &lt;span class="nb"&gt;install&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Node.js 20.11 or newer; Linux, macOS or Windows 11. The code is at &lt;a href="https://github.com/oll4com/spaceapp" rel="noopener noreferrer"&gt;https://github.com/oll4com/spaceapp&lt;/a&gt; and the paths above are where to start reading.&lt;/p&gt;

&lt;p&gt;If you've built a multi-viewer terminal over WebSockets: how do you pause a PTY when one watcher is slow and another isn't?&lt;/p&gt;

</description>
      <category>ai</category>
      <category>opensource</category>
      <category>architecture</category>
      <category>webdev</category>
    </item>
    <item>
      <title>We gave 6 AI coding CLIs the same concurrency bug and 6 hidden tests. All six passed. The diffs did not agree.</title>
      <dc:creator>Space App</dc:creator>
      <pubDate>Thu, 05 Nov 2026 13:35:22 +0000</pubDate>
      <link>https://dev.to/space_app_fc7f6872808dc84/we-gave-6-ai-coding-clis-the-same-concurrency-bug-and-6-hidden-tests-all-six-passed-the-diffs-did-pl5</link>
      <guid>https://dev.to/space_app_fc7f6872808dc84/we-gave-6-ai-coding-clis-the-same-concurrency-bug-and-6-hidden-tests-all-six-passed-the-diffs-did-pl5</guid>
      <description>&lt;p&gt;Arguments about which AI coding CLI is "best" usually rest on vibes or on synthetic leaderboards. So we took one small, real-shaped concurrency bug, gave it to six CLIs with the same prompt, and graded each fix with tests the CLIs never saw.&lt;/p&gt;

&lt;p&gt;All six fixed it. That's the first result, and it limits what this post can claim: the task didn't separate them on correctness. What it did show is how long each took, what each spent, how big the diff was, and that one fix that passes every test still changes the queue's behaviour in a way the tests didn't check.&lt;/p&gt;

&lt;p&gt;I build SpaceApp, a self-hosted, MIT-licensed workspace that runs these CLIs side by side. That's why we have them all installed. The runs below were headless, from a script, not inside SpaceApp, so this post is about the CLIs.&lt;/p&gt;

&lt;h2&gt;
  
  
  The bug
&lt;/h2&gt;

&lt;p&gt;The repo is a throwaway Node package, &lt;code&gt;jobq&lt;/code&gt;: an in-process job queue with &lt;code&gt;concurrency&lt;/code&gt;, &lt;code&gt;retries&lt;/code&gt;, exponential &lt;code&gt;backoffMs&lt;/code&gt; and an &lt;code&gt;onIdle()&lt;/code&gt; promise. It has no dependencies, about 90 lines in &lt;code&gt;src/queue.js&lt;/code&gt;, and 5 tests on &lt;code&gt;node:test&lt;/code&gt;. The bug is in the retry path:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="nf"&gt;_run&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;job&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_running&lt;/span&gt;&lt;span class="o"&gt;++&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nx"&gt;job&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;attempt&lt;/span&gt;&lt;span class="o"&gt;++&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;value&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;job&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;fn&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt; &lt;span class="na"&gt;id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;job&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;attempt&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;job&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;attempt&lt;/span&gt; &lt;span class="p"&gt;});&lt;/span&gt;
    &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_completed&lt;/span&gt;&lt;span class="o"&gt;++&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nx"&gt;job&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;resolve&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;value&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;catch &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;error&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;job&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;attempt&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;retries&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_running&lt;/span&gt;&lt;span class="o"&gt;--&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;                       &lt;span class="c1"&gt;// first decrement&lt;/span&gt;
      &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;delay&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;backoffMs&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt; &lt;span class="o"&gt;**&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;job&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;attempt&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
      &lt;span class="nf"&gt;setTimeout&lt;/span&gt;&lt;span class="p"&gt;(()&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_pending&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;push&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;job&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
        &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;_next&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
      &lt;span class="p"&gt;},&lt;/span&gt; &lt;span class="nx"&gt;delay&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
      &lt;span class="k"&gt;return&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_failed&lt;/span&gt;&lt;span class="o"&gt;++&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;onError&lt;/span&gt;&lt;span class="p"&gt;?.(&lt;/span&gt;&lt;span class="nx"&gt;error&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;job&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;id&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="nx"&gt;job&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;reject&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;error&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;finally&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_running&lt;/span&gt;&lt;span class="o"&gt;--&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;                         &lt;span class="c1"&gt;// runs on the retry path too&lt;/span&gt;
    &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;_next&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Two things are wrong:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;A failed attempt that will be retried decrements &lt;code&gt;_running&lt;/code&gt; twice, once in &lt;code&gt;catch&lt;/code&gt; and again in &lt;code&gt;finally&lt;/code&gt; (a &lt;code&gt;return&lt;/code&gt; inside &lt;code&gt;catch&lt;/code&gt; doesn't skip &lt;code&gt;finally&lt;/code&gt;). So the queue starts more jobs than &lt;code&gt;concurrency&lt;/code&gt; allows.&lt;/li&gt;
&lt;li&gt;A job waiting out its backoff is counted nowhere: not in &lt;code&gt;_pending&lt;/code&gt;, not in &lt;code&gt;_running&lt;/code&gt;. So &lt;code&gt;onIdle()&lt;/code&gt; resolves while a retry is still due.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The 5th visible test fails on the baseline: "onIdle resolved before the retried job finished". Deleting just the extra decrement makes that test pass while &lt;code&gt;onIdle()&lt;/code&gt; stays wrong. So a shallow fix is possible, and the hidden tests exist to catch it.&lt;/p&gt;

&lt;h2&gt;
  
  
  The prompt
&lt;/h2&gt;

&lt;p&gt;Every CLI got this, verbatim:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;The test suite of this repository fails (&lt;code&gt;npm test&lt;/code&gt;). Find the root cause of the bug in the job queue and fix it. Do not modify the tests. Run &lt;code&gt;npm test&lt;/code&gt; to check your fix. When you are done, reply with a short summary of the cause and the fix.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  Method
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;One run per CLI, one after another, each on a fresh copy of the repo.&lt;/li&gt;
&lt;li&gt;The same 1,200 s timeout for all. No run came near it.&lt;/li&gt;
&lt;li&gt;Headless, auto-approve, stdin closed, and a minimal environment (&lt;code&gt;env -i&lt;/code&gt;): no MCP servers, no global agent instructions, throwaway config or home directories. Sign-ins were linked in, never copied.&lt;/li&gt;
&lt;li&gt;Wall time is the whole CLI process from start to exit, so it includes start-up.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Grading ran after each CLI had exited:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;the diff against the baseline, the files touched, and whether the CLI committed;&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;npm test&lt;/code&gt; as the CLI left the repo;&lt;/li&gt;
&lt;li&gt;the visible tests again on a copy with &lt;code&gt;test/&lt;/code&gt; and &lt;code&gt;package.json&lt;/code&gt; restored from the baseline, so a CLI that edited the tests couldn't pass that way;&lt;/li&gt;
&lt;li&gt;6 hidden acceptance tests, never shown to any CLI, run 3 times each.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The hidden tests cover:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;onIdle()&lt;/code&gt; while only a retry is waiting;&lt;/li&gt;
&lt;li&gt;a retry waiting for a free slot;&lt;/li&gt;
&lt;li&gt;30 flaky jobs at concurrency 3 with exact counters;&lt;/li&gt;
&lt;li&gt;backoff doubling;&lt;/li&gt;
&lt;li&gt;three &lt;code&gt;onIdle()&lt;/code&gt; waiters;&lt;/li&gt;
&lt;li&gt;a synchronous throw.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Before trusting them, we ran them against three known states:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Version of &lt;code&gt;queue.js&lt;/code&gt;
&lt;/th&gt;
&lt;th&gt;Hidden tests passed&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Baseline (the bug)&lt;/td&gt;
&lt;td&gt;1 of 6 (2 fail, 3 time out)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Shallow fix (only drop the extra decrement)&lt;/td&gt;
&lt;td&gt;4 of 6&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Reference fix&lt;/td&gt;
&lt;td&gt;6 of 6&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;So the hidden tests do tell a shallow fix from a real one.&lt;/p&gt;

&lt;h2&gt;
  
  
  Results
&lt;/h2&gt;

&lt;p&gt;All six:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;changed only &lt;code&gt;src/queue.js&lt;/code&gt;;&lt;/li&gt;
&lt;li&gt;made no commit;&lt;/li&gt;
&lt;li&gt;left the visible tests passing;&lt;/li&gt;
&lt;li&gt;passed 6/6 hidden tests on all three runs.&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;CLI (version)&lt;/th&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;Wall time&lt;/th&gt;
&lt;th&gt;Hidden tests (3 runs)&lt;/th&gt;
&lt;th&gt;Diff&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Claude Code 2.1.316&lt;/td&gt;
&lt;td&gt;claude-opus-5-5&lt;/td&gt;
&lt;td&gt;16.3 s&lt;/td&gt;
&lt;td&gt;6/6, 6/6, 6/6&lt;/td&gt;
&lt;td&gt;+4 −2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;OpenCode 2.2.5&lt;/td&gt;
&lt;td&gt;claude-opus-5-5 (see below)&lt;/td&gt;
&lt;td&gt;16.9 s&lt;/td&gt;
&lt;td&gt;6/6, 6/6, 6/6&lt;/td&gt;
&lt;td&gt;+4 −2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;GitHub Copilot CLI 1.0.105&lt;/td&gt;
&lt;td&gt;auto: "GPT-6 Luna"&lt;/td&gt;
&lt;td&gt;25.9 s&lt;/td&gt;
&lt;td&gt;6/6, 6/6, 6/6&lt;/td&gt;
&lt;td&gt;+4 −2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Cursor CLI 2026.10.22&lt;/td&gt;
&lt;td&gt;"Auto" (not named)&lt;/td&gt;
&lt;td&gt;83.2 s&lt;/td&gt;
&lt;td&gt;6/6, 6/6, 6/6&lt;/td&gt;
&lt;td&gt;+5 −2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Grok CLI 1.0.70&lt;/td&gt;
&lt;td&gt;grok-4.7&lt;/td&gt;
&lt;td&gt;88.3 s&lt;/td&gt;
&lt;td&gt;6/6, 6/6, 6/6&lt;/td&gt;
&lt;td&gt;+18 −10&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Gemini (Antigravity CLI &lt;code&gt;agy&lt;/code&gt;) 1.4.12&lt;/td&gt;
&lt;td&gt;Gemini 3.8 Flash (High)&lt;/td&gt;
&lt;td&gt;149.4 s&lt;/td&gt;
&lt;td&gt;6/6, 6/6, 6/6&lt;/td&gt;
&lt;td&gt;+4 −2&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h3&gt;
  
  
  Tokens and cost
&lt;/h3&gt;

&lt;p&gt;These are the figures each CLI reported, or our API gateway's list-price estimate where the CLI printed none. They come from four different kinds of source, so don't read them as one ranking. Tokenizers differ, cache accounting differs, and Copilot rounds.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;CLI&lt;/th&gt;
&lt;th&gt;Turns / tool calls&lt;/th&gt;
&lt;th&gt;Tokens&lt;/th&gt;
&lt;th&gt;Cost&lt;/th&gt;
&lt;th&gt;Source&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Claude Code&lt;/td&gt;
&lt;td&gt;5 turns&lt;/td&gt;
&lt;td&gt;input 10, cache write 18,939, cache read 112,804, output 1,367&lt;/td&gt;
&lt;td&gt;$0.1446&lt;/td&gt;
&lt;td&gt;Claude Code's own estimate at API list prices. The run used a subscription, so this is not money spent; our gateway's estimate for its 5 requests was the same $0.1446&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;OpenCode&lt;/td&gt;
&lt;td&gt;4 steps, 3 tool calls&lt;/td&gt;
&lt;td&gt;input 10, cache write 6,274, cache read 33,592, output 851, reasoning 130&lt;/td&gt;
&lt;td&gt;$0.0586&lt;/td&gt;
&lt;td&gt;Our gateway's list-price estimate for its 5 requests, which includes OpenCode's session-title call. OpenCode printed cost 0 (no price table in its provider config); its token counts match the gateway's&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Copilot&lt;/td&gt;
&lt;td&gt;—&lt;/td&gt;
&lt;td&gt;↑ 74.4k (57.7k cached, 16.7k written), ↓ 1.2k (636 reasoning)&lt;/td&gt;
&lt;td&gt;0.33 AI Credits&lt;/td&gt;
&lt;td&gt;Copilot's own summary&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Cursor&lt;/td&gt;
&lt;td&gt;10 tool calls&lt;/td&gt;
&lt;td&gt;input 18,275, cache read 70,784, output 5,023&lt;/td&gt;
&lt;td&gt;not reported&lt;/td&gt;
&lt;td&gt;Cursor's stream-json &lt;code&gt;result&lt;/code&gt; event&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Grok&lt;/td&gt;
&lt;td&gt;7 turns&lt;/td&gt;
&lt;td&gt;input 37,288, cache read 184,064, output 9,390 (reasoning 7,969)&lt;/td&gt;
&lt;td&gt;$0.2229&lt;/td&gt;
&lt;td&gt;Grok CLI's own figure&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Gemini (agy)&lt;/td&gt;
&lt;td&gt;agy reports 1 turn&lt;/td&gt;
&lt;td&gt;input 132,071, cache read 231,732, output 13,509 (thinking 11,803)&lt;/td&gt;
&lt;td&gt;$0.1672&lt;/td&gt;
&lt;td&gt;Tokens from agy's JSON; the cost is our gateway's estimate for the run's 19 requests. agy prints no cost&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  What the fixes look like
&lt;/h2&gt;

&lt;p&gt;Three CLIs (Claude Code, OpenCode, Gemini) wrote the same fix as our reference. A counter for jobs waiting out their backoff replaces the extra decrement, the timer decrements it, and &lt;code&gt;_isIdle()&lt;/code&gt; checks it. This is Claude Code's:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight diff"&gt;&lt;code&gt;&lt;span class="p"&gt;@@ -16,6 +16,7 @@&lt;/span&gt; export class JobQueue {
     this._failed = 0;
     this._idleWaiters = [];
     this._seq = 0;
&lt;span class="gi"&gt;+    this._retrying = 0; // jobs waiting out their backoff: not pending, not running, but not done
&lt;/span&gt;   }
&lt;span class="p"&gt;@@ -41,7 +42,7 @@&lt;/span&gt; export class JobQueue {
   _isIdle() {
&lt;span class="gd"&gt;-    return this._pending.length === 0 &amp;amp;&amp;amp; this._running === 0;
&lt;/span&gt;&lt;span class="gi"&gt;+    return this._pending.length === 0 &amp;amp;&amp;amp; this._running === 0 &amp;amp;&amp;amp; this._retrying === 0;
&lt;/span&gt;   }
&lt;span class="p"&gt;@@ -65,9 +66,10 @@&lt;/span&gt; export class JobQueue {
     } catch (error) {
       if (job.attempt &amp;lt;= this.retries) {
&lt;span class="gd"&gt;-        this._running--;
&lt;/span&gt;&lt;span class="gi"&gt;+        this._retrying++;
&lt;/span&gt;         const delay = this.backoffMs * 2 ** (job.attempt - 1);
         setTimeout(() =&amp;gt; {
&lt;span class="gi"&gt;+          this._retrying--;
&lt;/span&gt;           this._pending.push(job);
           this._next();
         }, delay);
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The others:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Cursor:&lt;/strong&gt; the same idea with a &lt;code&gt;_delayed&lt;/code&gt; counter, decremented after the job is queued again.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Grok:&lt;/strong&gt; the same counter, plus a restructure that moves all retry scheduling into &lt;code&gt;finally&lt;/code&gt;. It's correct, and it's the biggest diff (+18 −10). Whether that's a cleanup or churn depends on who reviews it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Copilot:&lt;/strong&gt; a different design. The failed job keeps its running slot through the backoff, and the timer releases it:
&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight diff"&gt;&lt;code&gt;&lt;span class="p"&gt;@@ -59,16 +59,18 @@&lt;/span&gt; export class JobQueue {
   async _run(job) {
     this._running++;
     job.attempt++;
&lt;span class="gi"&gt;+    let retryScheduled = false;
&lt;/span&gt;     try {
&lt;span class="p"&gt;@@
       if (job.attempt &amp;lt;= this.retries) {
-        this._running--;
+        retryScheduled = true;
         const delay = this.backoffMs * 2 ** (job.attempt - 1);
         setTimeout(() =&amp;gt; {
           this._pending.push(job);
+          this._running--;
           this._next();
         }, delay);
         return;
@@&lt;/span&gt; -77,7 +79,7 @@ export class JobQueue {
     } finally {
&lt;span class="gd"&gt;-      this._running--;
&lt;/span&gt;&lt;span class="gi"&gt;+      if (!retryScheduled) this._running--;
&lt;/span&gt;       this._next();
     }
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It passes every visible and hidden test. &lt;code&gt;onIdle()&lt;/code&gt; is right, because a waiting job still counts as running, and the concurrency limit holds. So by the grading above, it's a full pass.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the hidden tests didn't catch
&lt;/h2&gt;

&lt;p&gt;This part came after the results, so it isn't part of the pass/fail above. It's a finding, not a score.&lt;/p&gt;

&lt;p&gt;Reading the diffs, Copilot's design looked like it changed what "running" means. The package README says &lt;code&gt;concurrency&lt;/code&gt; is "the most jobs running at the same time", and a job sleeping through its backoff isn't running. So we wrote two more checks from that sentence:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;P1:&lt;/strong&gt; at concurrency 1, does another job run while one job waits out a 200 ms backoff?&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;P2:&lt;/strong&gt; does &lt;code&gt;stats().running&lt;/code&gt; leave a waiting job out?&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Five fixes and the reference pass both. Copilot's fails both: job b waited 190 ms behind the other job's backoff, and &lt;code&gt;stats().running&lt;/code&gt; read 1 while nothing was running. Its fix is correct for the limit and for &lt;code&gt;onIdle()&lt;/code&gt;, but now a backoff blocks a slot. With long backoffs and low concurrency, that's real throughput you lose without any test going red.&lt;/p&gt;

&lt;p&gt;Two lessons for anyone grading agent output, ourselves included:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;"Passes the hidden tests" is only as strong as the tests.&lt;/strong&gt; We wrote six that caught the shallow fix, and they still missed a design change. We only saw it because we read the diffs.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The prompt said "fix the bug", not "keep the semantics".&lt;/strong&gt; Copilot's version is a defensible reading of an underspecified task. If your spec lives in a README sentence, say so in the prompt or put it in a test.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  Who didn't run, and why
&lt;/h2&gt;

&lt;p&gt;N is 6 because that's how many really ran. We have more CLIs installed:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Codex and DeepSeek:&lt;/strong&gt; skipped on purpose. There was no quota or balance on those accounts at the time of the run.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Kimi Code:&lt;/strong&gt; installed and signed in, but the smoke run answered &lt;code&gt;403 Your current subscription does not have access to Kimi Code right now&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Qwen Code:&lt;/strong&gt; installed, but its credential check reported the quota as exhausted before the batch, so it wasn't run.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;OpenCode on its free default model:&lt;/strong&gt; the first OpenCode attempt used the free Big Pickle model and stopped after 3.6 s with a 429 &lt;code&gt;Free usage limit reached&lt;/code&gt;. OpenCode was then run on Claude Opus 5.5 with an Anthropic API key. So &lt;strong&gt;the OpenCode row measures the OpenCode harness with Opus, not OpenCode's free tier.&lt;/strong&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Honest limits
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;n = 1 per CLI.&lt;/strong&gt; Wall times and token counts would move on another run. One run tells you what happened once; it doesn't rank CLIs.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;One small bug in one ~90-line file.&lt;/strong&gt; Every CLI solved it, so this task doesn't separate them on correctness. It shows speed, cost, diff size and fix design on an easy-to-medium concurrency bug.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Each CLI ran a different model&lt;/strong&gt;, so harness and model can't be separated. Claude Code and OpenCode both ran Opus 5.5 (16.3 s vs 16.9 s, same fix), which is the closest thing here to a same-model comparison.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Copilot and Cursor pick their own model&lt;/strong&gt; ("auto"). Copilot's smoke run 7 minutes earlier picked a different one (MAI-Code-1.1-Flash). agy's model came from its saved settings.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Costs come from four kinds of source&lt;/strong&gt; (the CLI's own figure, our gateway's list-price estimate, AI Credits, none), so they aren't comparable. Claude Code's figure is a list-price estimate for a subscription run.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;One shared Linux server under load&lt;/strong&gt; (24 cores, load average about 16 during the batch, Node v26.13.0). The Claude and Gemini requests went through our gateway, and wall time includes that hop and CLI start-up.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Environments were clean but not identical.&lt;/strong&gt; Grok ran with its usual home directory, which holds its skills and memory, because its sign-in lives there. The others ran with throwaway homes and linked sign-ins.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  If you want to repeat it
&lt;/h2&gt;

&lt;p&gt;The ingredients are all in this post: the buggy &lt;code&gt;_run&lt;/code&gt; above, the prompt, the shallow-fix trap, and the list of hidden tests. The rest of the queue is the obvious code around them (&lt;code&gt;add&lt;/code&gt;, &lt;code&gt;stats&lt;/code&gt;, &lt;code&gt;onIdle&lt;/code&gt;, a &lt;code&gt;_next&lt;/code&gt; loop). Write the hidden tests before you run anything, and check them against the baseline, a shallow fix and a reference fix, as we did. That step is what makes the pass column mean something.&lt;/p&gt;

&lt;p&gt;What would you want in the next run: a harder bug that actually separates them, or the same CLIs pinned to the same model?&lt;/p&gt;

</description>
      <category>ai</category>
      <category>programming</category>
      <category>opensource</category>
      <category>productivity</category>
    </item>
    <item>
      <title>Why terminal CLI agents beat web chat on real codebases (and where web chat still wins)</title>
      <dc:creator>Space App</dc:creator>
      <pubDate>Wed, 04 Nov 2026 15:12:08 +0000</pubDate>
      <link>https://dev.to/space_app_fc7f6872808dc84/why-terminal-cli-agents-beat-web-chat-on-real-codebases-and-where-web-chat-still-wins-14k0</link>
      <guid>https://dev.to/space_app_fc7f6872808dc84/why-terminal-cli-agents-beat-web-chat-on-real-codebases-and-where-web-chat-still-wins-14k0</guid>
      <description>&lt;p&gt;I build SpaceApp, a self-hosted workspace for running AI coding CLIs side by side. This post is about the workflow, not the tool; the tool gets two lines at the end.&lt;/p&gt;

&lt;p&gt;If your agent can't run &lt;code&gt;npm test&lt;/code&gt; and read the failure in the same shell, you end up as the relay between the model and your codebase: copy the file in, copy the answer out, run it, paste the error back, repeat. That loop is fine for a snippet. On a real codebase it breaks down, and the reasons are worth spelling out.&lt;/p&gt;

&lt;h2&gt;
  
  
  What a web chat can't see
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;The rest of the repo.&lt;/strong&gt; You paste the file you think matters. The bug is often in the one you didn't paste: the caller, the fixture, the config.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The real output.&lt;/strong&gt; You paste the part of the error you think matters. The useful line is often 40 lines up, or in a warning you skipped.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Whether the fix worked.&lt;/strong&gt; The chat never runs it. "This should fix it" is a guess until a test says so.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;What changed.&lt;/strong&gt; After five rounds of copy-paste, nobody knows exactly which version of which file is on disk.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What a terminal agent does instead
&lt;/h2&gt;

&lt;p&gt;A CLI agent works in the working tree. It can grep for every caller before it changes a signature, run the test it just broke, read the compiler's diagnostics in full, edit several files in one change, and leave the result as a diff on a branch you can review. The feedback loop runs in the shell, not through your clipboard.&lt;/p&gt;

&lt;p&gt;A small real example from our own repo: one of our contract tests passed on its own in about 1.5 seconds and timed out only when the whole suite ran in parallel. Nothing about that is visible in a pasted file or a pasted error. It took something that could run the suite, see the timing and change the test, and that was an agent working in a shell.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where web chat still wins
&lt;/h2&gt;

&lt;p&gt;To be fair to it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Exploring an idea before there is any code.&lt;/li&gt;
&lt;li&gt;Explaining a concept, an API or an error message you don't understand.&lt;/li&gt;
&lt;li&gt;A self-contained snippet: a regex, a SQL query, a one-off script.&lt;/li&gt;
&lt;li&gt;Zero setup, and no access to your machine, which is sometimes exactly what you want.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What terminal agents cost you
&lt;/h2&gt;

&lt;p&gt;They are more powerful because they touch real things, and that is also the risk.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Give them a branch, not your main checkout.&lt;/strong&gt; One git worktree per agent keeps their work apart, and you review the diff before it lands.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Give them only what they need.&lt;/strong&gt; The folders for the task, read-only where reading is enough. No production credentials in the environment.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Make tests the referee.&lt;/strong&gt; An agent that can run tests will also happily "fix" a test to make it pass. Review test changes as carefully as code changes.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Watch for parallel surprises.&lt;/strong&gt; If you run several agents at once, run your suite in parallel too. Tests that pass alone and fail together are common, and some of them point at real bugs.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A worktree per agent is one command:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git worktree add ../myrepo-agent-a &lt;span class="nt"&gt;-b&lt;/span&gt; agent-a
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Running several side by side
&lt;/h2&gt;

&lt;p&gt;Once one terminal agent works, the next step is usually two or three: one per repo, or one planning and one executing. That works in tmux, in separate terminal windows or in a workspace that keeps the sessions for you. The rules stay the same: a worktree each, one way into main that builds and tests, and you as the reviewer rather than the clipboard.&lt;/p&gt;

&lt;h2&gt;
  
  
  Over to you
&lt;/h2&gt;

&lt;p&gt;Where do you still use web chat for coding, and where did you switch to a terminal agent for good? And for those running more than one agent: what was the first rule you had to add?&lt;/p&gt;




&lt;p&gt;The two lines about the tool: SpaceApp keeps CLI sessions side by side in persistent rooms on your own machine and your own provider accounts. It is MIT-licensed and installs on Linux, macOS or Windows 11 with Node.js 20.11 or newer: &lt;code&gt;npx --yes run-spaceapp@latest install&lt;/code&gt;. Code and docs: &lt;a href="https://github.com/oll4com/spaceapp" rel="noopener noreferrer"&gt;https://github.com/oll4com/spaceapp&lt;/a&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>devtools</category>
      <category>cli</category>
      <category>productivity</category>
    </item>
    <item>
      <title>Nous: one mind over every AI coding CLI in a self-hosted workspace</title>
      <dc:creator>Space App</dc:creator>
      <pubDate>Wed, 04 Nov 2026 14:18:03 +0000</pubDate>
      <link>https://dev.to/space_app_fc7f6872808dc84/nous-one-mind-over-every-ai-coding-cli-in-a-self-hosted-workspace-1cm2</link>
      <guid>https://dev.to/space_app_fc7f6872808dc84/nous-one-mind-over-every-ai-coding-cli-in-a-self-hosted-workspace-1cm2</guid>
      <description>&lt;p&gt;I build SpaceApp, a self-hosted workspace where AI coding CLIs (OpenCode, Codex, Claude Code, Gemini and others) run side by side in persistent rooms. Each pane in a room is its own CLI or app session, on your own provider accounts.&lt;/p&gt;

&lt;p&gt;Running several agents at once moves the bottleneck to the person. Any pane can stop on a question, a sign-in or a choice, and you end up going from pane to pane to find out which one is waiting and for what. Nous is the part of SpaceApp built for that. This post covers what it does, taken from the public docs, and where its limits are.&lt;/p&gt;

&lt;h2&gt;
  
  
  One mind, fed by progress reports
&lt;/h2&gt;

&lt;p&gt;Nous (Greek νους, "mind") sees every room, pane, copilot, plan and release train. It does not watch screens: it reads each pane's progress reports, so it knows what every agent works on and what it waits for. You talk to Nous instead of to every agent.&lt;/p&gt;

&lt;h2&gt;
  
  
  The routine work it does itself
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;It opens, steers and closes panes, and sends each one its next step.&lt;/li&gt;
&lt;li&gt;It runs up to five helpers side by side.&lt;/li&gt;
&lt;li&gt;It saves finished work to Space Memory and closes panes whose work is done.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Standing orders&lt;/strong&gt; run around the clock: finished panes are saved and closed, stalled plans get a nudge, and a daily self-review looks back at its own turns.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Goals.&lt;/strong&gt; Type &lt;code&gt;/goal&lt;/code&gt; and your objective in your own words. Nous breaks it into plans, panes and helpers, and keeps working on it in turns of its own until it is done.&lt;/p&gt;

&lt;h2&gt;
  
  
  The four kinds of decisions it brings to you
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;Releases.&lt;/li&gt;
&lt;li&gt;Deletes.&lt;/li&gt;
&lt;li&gt;Anything that reaches third parties or money.&lt;/li&gt;
&lt;li&gt;The direction of the work.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Each decision is a card with Approve and Decline; a plain yes or no in words decides it too. "Always approve" lets a routine kind of decision run on its own next time, but never a release or a delete.&lt;/p&gt;

&lt;p&gt;When an agent needs your hands or an answer, you get one &lt;strong&gt;Needs you&lt;/strong&gt; request: one question, its options and a recommendation. You reach Nous in its dock, on your phone or in a paired Telegram chat. While you are away it keeps a log for your return, and at night, requests for your hands wait for the morning.&lt;/p&gt;

&lt;p&gt;The &lt;strong&gt;Plan map&lt;/strong&gt; shows every open plan: what each one waits on (you, a date, a promote, a third party, Nous or the server), the plans it depends on and what to unblock first. Nous reads the same map before it proposes or starts work.&lt;/p&gt;

&lt;h2&gt;
  
  
  Logins it never sees
&lt;/h2&gt;

&lt;p&gt;Your browser encrypts every login in the Vault with keys that live only in the open tab; the database keeps ciphertext. When an agent or Nous needs a login or a one-time code, a broker on your host fills it into the pane's browser, and the agent gets back a status and an audit id, not the value. After a fill, the page stays protected: the agent can click and type but cannot read the field.&lt;/p&gt;

&lt;h2&gt;
  
  
  Quotas
&lt;/h2&gt;

&lt;p&gt;Nous watches your AI quotas, spends less while they run low and uses a cheaper model for its background turns.&lt;/p&gt;

&lt;h2&gt;
  
  
  A small, real example
&lt;/h2&gt;

&lt;p&gt;This article started as a brief from Nous to the Marketing Copilot pane of my own SpaceApp: draft an article, a Show HN text and an X thread about Nous, for review only. The copilot drafted them and asked for a review; nothing goes out before it is approved.&lt;/p&gt;

&lt;h2&gt;
  
  
  Limits worth knowing
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Nous is off on a new install. You turn it on with &lt;strong&gt;Work on my behalf&lt;/strong&gt; in its dock.&lt;/li&gt;
&lt;li&gt;By default SpaceApp runs for one owner on one machine; team accounts, with a CLI container per user, are opt-in.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Try it
&lt;/h2&gt;

&lt;p&gt;You need Linux, macOS or Windows 11, Node.js 20.11 or newer, and at least 4 CPUs, 8 GB RAM and 15 GiB of free disk:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx &lt;span class="nt"&gt;--yes&lt;/span&gt; run-spaceapp@latest &lt;span class="nb"&gt;install&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;SpaceApp is MIT-licensed. Code and docs: &lt;a href="https://github.com/oll4com/spaceapp" rel="noopener noreferrer"&gt;https://github.com/oll4com/spaceapp&lt;/a&gt;. Nous in the docs: &lt;a href="https://spaceapp.dev/docs/nous/" rel="noopener noreferrer"&gt;https://spaceapp.dev/docs/nous/&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;I'd like to hear where you would draw the line between "routine" and "ask me" for your own agents.&lt;/p&gt;

</description>
      <category>selfhosted</category>
      <category>ai</category>
      <category>devtools</category>
      <category>opensource</category>
    </item>
    <item>
      <title>One tool layer for eleven AI coding CLIs: what we learned building SpaceApp</title>
      <dc:creator>Space App</dc:creator>
      <pubDate>Wed, 28 Oct 2026 13:40:43 +0000</pubDate>
      <link>https://dev.to/space_app_fc7f6872808dc84/one-tool-layer-for-eleven-ai-coding-clis-what-we-learned-building-spaceapp-cg</link>
      <guid>https://dev.to/space_app_fc7f6872808dc84/one-tool-layer-for-eleven-ai-coding-clis-what-we-learned-building-spaceapp-cg</guid>
      <description>&lt;p&gt;Why I built SpaceApp, in my own words:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;I built it to be my main work tool. I'm a developer! And it became my main tool and work environment. I no longer need to open email, social media or any development program: absolutely nothing. I'm inside SpaceApp all day; I even watch YouTube videos and listen to music through SpaceApp.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;SpaceApp is a self-hosted workspace where AI coding CLIs (Claude Code, Codex, OpenCode, Gemini CLI and others) run side by side in persistent rooms. Each of them comes with its own config format, its own way of registering MCP servers, its own sign-in, its own "resume the last session" flag and its own idea of what Esc means. This post is about the two parts that turned out to matter most: where the agents' tools come from, and where the agents' sessions live.&lt;/p&gt;

&lt;h2&gt;
  
  
  The N×M problem
&lt;/h2&gt;

&lt;p&gt;Every CLI wants its tools declared in its own file: one JSON shape for one, a TOML table for another, a command-line flag for a third. Multiply that by the tools you want every agent to have (a managed browser, shared memory, tasks, files, deployment checks) and you get N×M configurations that drift apart. Worse, a CLI usually starts every declared MCP server when the session starts, whether or not the agent ever calls it. Twelve panes × five servers is a lot of idle processes.&lt;/p&gt;

&lt;h2&gt;
  
  
  Capability first, MCP on demand
&lt;/h2&gt;

&lt;p&gt;The first rule we settled on: &lt;strong&gt;use what the runtime already has.&lt;/strong&gt; If the CLI has a shell, file editing and image input, the agent uses them directly. A tool layer that wraps &lt;code&gt;cat&lt;/code&gt; in an MCP call only adds latency and tokens.&lt;/p&gt;

&lt;p&gt;For SpaceApp's own operations (status, logs, the managed browser, memory), the route depends on the runtime:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Codex and Claude Code&lt;/strong&gt; panes register &lt;em&gt;no&lt;/em&gt; MCP server at startup. When the agent needs a SpaceApp operation it calls one on-demand entrypoint, &lt;code&gt;space-capability&lt;/code&gt;. It answers "what can I call and with which arguments" from cached, compact schemas without starting anything. The first real call starts a small worker on a pane-private Unix socket, and that worker starts only the backend that was asked for: at most three per pane, each stopped after an idle timeout.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Other runtimes&lt;/strong&gt; keep a single dispatcher (&lt;code&gt;space-mcp-dispatcher&lt;/code&gt;) declared once from a shared registry. The dispatcher also spawns registry servers only when called and stops them when idle.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Two details turned out to matter more than the transport:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;A declared tool grants nothing.&lt;/strong&gt; Every operation still passes its own schema, role, approval and broker gates, whichever path it came through. MCP is a transport, not an authorization model.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;No replays after ambiguity.&lt;/strong&gt; If a call may have executed and the transport broke, the call is not retried automatically. A duplicated deploy check is annoying; a duplicated "send" is worse.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The operator can override routing per runtime, model and tool (Automatic, Native only, Allow fallback, Disabled), and the settings page explains which route a call will take.&lt;/p&gt;

&lt;p&gt;This path is still marked &lt;strong&gt;experimental&lt;/strong&gt; in the architecture doc: it is enabled for Codex and Claude Code only, because each runtime's capabilities have to be verified before it gets the on-demand path.&lt;/p&gt;

&lt;h2&gt;
  
  
  Sessions that outlive the web server
&lt;/h2&gt;

&lt;p&gt;The second lesson: &lt;strong&gt;the process that serves the UI must not own the agents.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;In SpaceApp each CLI runs in a terminal session owned by a separate pane host process, not by the API. The API attaches to that session over a socket. When the API restarts during a deploy, or the browser drops its connection, the agent keeps working, and the pane re-attaches to the same session. The pane host tracks session generations and output sequence numbers, so a reconnecting view knows where it left off.&lt;/p&gt;

&lt;p&gt;That separation is also why we never resume a provider conversation by copying a transcript. If a CLI has a native resume (a session id, a thread id), we use it; if it does not, we say so instead of pretending.&lt;/p&gt;

&lt;h2&gt;
  
  
  Eleven TUIs are eleven TUIs
&lt;/h2&gt;

&lt;p&gt;Things that broke along the way, in roughly the order they hurt:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Stop means different keys.&lt;/strong&gt; Esc interrupts a turn in some CLIs, Ctrl-C in others, and one is stopped through its own server API. Stop now sends each TUI its own interrupt.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Model pickers are UI, not API.&lt;/strong&gt; Switching a model in one CLI means driving its &lt;code&gt;/model&lt;/code&gt; menu. In a short pane the menu shows a scrolled window of rows and clips its hints, so the picker has to read scroll markers and page through the menu. It also has to pick "this session only", never the saved default every other session shares.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;"Ready" screens lie.&lt;/strong&gt; An old, partial MCP startup message once kept a newer, ready Codex screen from being detected. Now a newer header supersedes the earlier partial marker, while a screen that is still loading keeps blocking.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Contracts are not end-to-end proof.&lt;/strong&gt; A contract suite checks every declared start, resume and login command line of every declared runtime against the wrapper filter. That catches regressions in our adapters; it does not prove that the next upstream release of a CLI behaves the same. We track that gap openly.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What it does not solve
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Isolation between users.&lt;/strong&gt; Rooms organise work; they are not a security boundary between mutually untrusted users. The Unix user is the outer trust boundary, so one install should serve one trust domain.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Your provider's view.&lt;/strong&gt; Self-hosting keeps the workspace and its data on your host. The model providers you configure still receive what the agents send them.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Try it
&lt;/h2&gt;

&lt;p&gt;SpaceApp is open source (MIT) and installs with:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx &lt;span class="nt"&gt;--yes&lt;/span&gt; run-spaceapp@latest &lt;span class="nb"&gt;install&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Architecture: &lt;a href="https://github.com/oll4com/spaceapp/blob/main/docs/architecture.md" rel="noopener noreferrer"&gt;https://github.com/oll4com/spaceapp/blob/main/docs/architecture.md&lt;/a&gt; · Known limitations: &lt;code&gt;docs/quality/KNOWN_LIMITATIONS.md&lt;/code&gt; in the repository.&lt;/p&gt;

&lt;p&gt;If you run several coding agents on one codebase, I would like to hear how you keep their tools and sessions in order.&lt;/p&gt;

</description>
      <category>selfhosted</category>
      <category>ai</category>
      <category>devtools</category>
      <category>productivity</category>
    </item>
    <item>
      <title>Six AI models played chess through their CLIs. None of them delivered checkmate.</title>
      <dc:creator>Space App</dc:creator>
      <pubDate>Mon, 19 Oct 2026 17:05:14 +0000</pubDate>
      <link>https://dev.to/space_app_fc7f6872808dc84/six-ai-models-played-chess-through-their-clis-none-of-them-delivered-checkmate-4fhp</link>
      <guid>https://dev.to/space_app_fc7f6872808dc84/six-ai-models-played-chess-through-their-clis-none-of-them-delivered-checkmate-4fhp</guid>
      <description>&lt;p&gt;I build SpaceApp, a self-hosted workspace for running AI coding CLIs side by side. On 18 October I let six models play chess against each other there, each through its own CLI. Nobody won by checkmate. The way they lost is the interesting part.&lt;/p&gt;

&lt;h2&gt;
  
  
  The setup
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Players:&lt;/strong&gt; Opus 5.5 and Fable 5.1 (through Claude Code), GPT-6-Astra and GPT-6.1-Sol (through Codex), all at xhigh reasoning; Gemini 4 Argon (High) and Gemini 3.8 Flash (High) (through the Gemini CLI).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Format:&lt;/strong&gt; a 4-round Swiss tournament, 12 games.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Clock:&lt;/strong&gt; 120 seconds per move. Each move is one one-shot turn of the model's CLI: it gets the position and answers with a move.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Strikes:&lt;/strong&gt; a timeout, a provider error or a second illegal answer plays a random legal move for that side and counts a strike. The third strike forfeits the game.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Length:&lt;/strong&gt; after 60 plies (30 moves each) the game is adjudicated on material.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  The results
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;#&lt;/th&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;Points&lt;/th&gt;
&lt;th&gt;W / D / L&lt;/th&gt;
&lt;th&gt;Elo&lt;/th&gt;
&lt;th&gt;Avg per move&lt;/th&gt;
&lt;th&gt;Forfeits&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;Opus 5.5 (xhigh)&lt;/td&gt;
&lt;td&gt;3.5&lt;/td&gt;
&lt;td&gt;3 / 1 / 0&lt;/td&gt;
&lt;td&gt;1535&lt;/td&gt;
&lt;td&gt;13 s&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;Gemini 4 Argon (High)&lt;/td&gt;
&lt;td&gt;3.5&lt;/td&gt;
&lt;td&gt;3 / 1 / 0&lt;/td&gt;
&lt;td&gt;1535&lt;/td&gt;
&lt;td&gt;23 s&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;GPT-6-Astra (xhigh)&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;2 / 0 / 2&lt;/td&gt;
&lt;td&gt;1500&lt;/td&gt;
&lt;td&gt;37 s&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;Fable 5.1 (xhigh)&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;1 / 0 / 3&lt;/td&gt;
&lt;td&gt;1478&lt;/td&gt;
&lt;td&gt;26 s&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;td&gt;Gemini 3.8 Flash (High)&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;1 / 0 / 3&lt;/td&gt;
&lt;td&gt;1477&lt;/td&gt;
&lt;td&gt;40 s&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;6&lt;/td&gt;
&lt;td&gt;GPT-6.1-Sol (xhigh)&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;1 / 0 / 3&lt;/td&gt;
&lt;td&gt;1475&lt;/td&gt;
&lt;td&gt;34 s&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Zero checkmates.&lt;/strong&gt; 11 of the 12 games ended on a third strike.&lt;/li&gt;
&lt;li&gt;The 12th game, Gemini 4 Argon against Opus 5.5, ran the full 60 plies and was drawn on material.&lt;/li&gt;
&lt;li&gt;Opus 5.5 and Gemini 4 Argon were the only models that never needed a fallback move. Opus 5.5 was also the fastest, at 13 seconds per move on average.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What it measures, and what it does not
&lt;/h2&gt;

&lt;p&gt;Twelve games are a small sample, and a tournament decided by forfeits says very little about chess strength. What it does show is how reliably a model answers through its CLI when every answer has a deadline: no timeouts, no provider errors, no illegal moves, game after game. That is the property I care about most when agents run unattended for hours, and it is not the same ranking you get from a single-prompt leaderboard.&lt;/p&gt;

&lt;h2&gt;
  
  
  Try SpaceApp
&lt;/h2&gt;

&lt;p&gt;SpaceApp runs on Linux, macOS and Windows 11 with Node.js 20.11 or newer:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx &lt;span class="nt"&gt;--yes&lt;/span&gt; run-spaceapp@latest &lt;span class="nb"&gt;install&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It is source-available under the Business Source License 1.1. Code, docs and known limitations: &lt;a href="https://github.com/oll4com/spaceapp" rel="noopener noreferrer"&gt;https://github.com/oll4com/spaceapp&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;If you have run something similar, I would like to know which models held up for you, and which rules you would change for a fairer second run.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>llm</category>
      <category>selfhosted</category>
      <category>benchmark</category>
    </item>
    <item>
      <title>From one command to a first page: what changed in SpaceApp's setup</title>
      <dc:creator>Space App</dc:creator>
      <pubDate>Sat, 17 Oct 2026 14:10:50 +0000</pubDate>
      <link>https://dev.to/space_app_fc7f6872808dc84/from-one-command-to-a-first-page-what-changed-in-spaceapps-setup-o4a</link>
      <guid>https://dev.to/space_app_fc7f6872808dc84/from-one-command-to-a-first-page-what-changed-in-spaceapps-setup-o4a</guid>
      <description>&lt;p&gt;I build SpaceApp, a self-hosted workspace for running AI coding CLIs in isolated rooms. A room organizes work; each pane in it is an independent CLI or app session. The runtimes on offer are OpenCode, Codex, Gemini, Qwen Code, Kimi Code, Grok Build and an experimental DeepSeek CLI; Claude Code has its own owner-initiated installation flow.&lt;/p&gt;

&lt;p&gt;The latest setup releases were about one stretch of the experience: the time between running the installer and getting a first useful answer. Here is that path step by step, and what changed on it.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. Install
&lt;/h2&gt;

&lt;p&gt;You need Linux, macOS or Windows 11, Node.js 20.11 or newer, and at least 4 CPUs, 8 GB RAM and 15 GiB of free disk. The standard profile, which adds a managed browser, recommends 8 CPUs, 16 GB RAM and 25 GiB.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx &lt;span class="nt"&gt;--yes&lt;/span&gt; run-spaceapp@latest &lt;span class="nb"&gt;install&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The launcher manages the core API and web app, the CLI runtimes, PostgreSQL and Temporal. The default light profile leaves out the managed Chromium; the standard profile starts a browser container as well.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. Create the owner account
&lt;/h2&gt;

&lt;p&gt;Once the installation passes its readiness checks, you use a one-time setup token to create the first account, register the folders you want agents to work in, and connect providers through their official login flows or masked credential input.&lt;/p&gt;

&lt;p&gt;Host folders are visible only after you register them as workspaces, and read-only mounts are supported.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;New:&lt;/strong&gt; the setup wizard's Ready step offers three first prompts you can copy.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. The first model
&lt;/h2&gt;

&lt;p&gt;The first OpenCode pane starts on a free model, Big Pickle. A model counts as working only after a real tool call: it has to read a small file with its read tool and answer with the random token inside it. When Big Pickle fails that check, the newest free tool-capable models are tried instead.&lt;/p&gt;

&lt;p&gt;Free models can change upstream at any time. If the one you are on stops responding, pick another in OpenCode.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. The first prompt
&lt;/h2&gt;

&lt;p&gt;Two things used to get in the way here.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;A room without a project had nowhere obvious to write.&lt;/strong&gt; Now a CLI pane in such a room starts in &lt;code&gt;/workspaces/&amp;lt;room&amp;gt;&lt;/code&gt;, so a first prompt like "build me a page" writes &lt;code&gt;index.html&lt;/code&gt; there.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The first task could stop at a permission prompt.&lt;/strong&gt; On macOS, a first OpenCode task could wait at "Access external directory /tmp" until someone answered. Now OpenCode's first run authorizes only its own private scratch directory, and the bootstrap instructions send temporary files there. Other paths outside the workspace still ask.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. Staying current
&lt;/h2&gt;

&lt;p&gt;The app now shows "Update available" when a newer release exists and opens the update command with a Copy button:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx &lt;span class="nt"&gt;--yes&lt;/span&gt; run-spaceapp@latest update
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  What to know before you try it
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;SpaceApp is designed for one trusted owner on one self-hosted instance.&lt;/li&gt;
&lt;li&gt;Backups are portable and checksummed and cover application data, PostgreSQL and owner memory. Provider credentials, login state, workspace contents and browser profiles are left out.&lt;/li&gt;
&lt;li&gt;Telemetry is opt-in. Answer No and nothing is sent. Answer Yes and one anonymous event is sent the first time an agent finishes an answer (event, random id, version, CPU architecture, date).&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Feedback welcome
&lt;/h2&gt;

&lt;p&gt;I would like to hear where the first five minutes still break for you: your OS, the profile the installer picked, and the step where it stopped. The full release notes are here: &lt;a href="https://github.com/oll4com/spaceapp/releases" rel="noopener noreferrer"&gt;https://github.com/oll4com/spaceapp/releases&lt;/a&gt;&lt;/p&gt;

</description>
      <category>selfhosted</category>
      <category>ai</category>
      <category>devtools</category>
    </item>
  </channel>
</rss>
