DEV Community

Artemii Amelin
Artemii Amelin

Posted on

Anthropic Ran 950 Agents for 21 Hours. launchd Ran Ours at Priority 4: shell.online 0.23.3 Moves Sessions to Interactive

Anthropic published a preprint on September 23 describing how roughly 950 Claude agents ran for 21 hours, spent about 210 million tokens, gathered over 200,000 reverse transcriptases from public sequence databases, picked out 3,500 candidate systems and narrowed those to the 20 most compelling. The result is a new enzyme family, array-associated reverse transcriptases, and one wet-lab confirmation: the array is expressed as short RNAs. Anthropic's own estimate is that the same analysis takes an expert scientist weeks to months.

The number that matters for anyone hosting agents is 21 hours. Agent sessions are now long-lived processes. They wait on a model, they wait on a person, and most of their wall-clock life is spent idle. Idle is exactly the state an operating system punishes, and this week we found out our macOS install had been asking it to.

What launchd does with a Background job

Every launch agent on macOS can declare a ProcessType. Apple's launchd.plist man page says the system "will apply resource limits based on what kind of job it is." A Background job is "generally processes that do work that was not directly requested by the user," and its limits "are intended to prevent them from disrupting the user experience." An Interactive job runs "with the same resource limitations as apps, that is to say, none."

Those limits are inherited. Every child a Background job spawns is clamped the same way.

shell service install writes a launch agent so a Mac stays reachable from the browser across reboots. Until yesterday that plist said ProcessType = Background. Every session someone started from a shell.online link was a child of that daemon, and so every one of them ran under the background clamp.

What that looked like on a real machine

Pull request #268, merged on September 25, has the measurements. Sessions hosted on one Mac took about five seconds on average, and at times over thirty, to echo a keystroke. The network was not the problem: TCP connect to shell.online was around 10 ms. The host was.

The clamp has two parts. CPU priority drops to 4, where a normal process runs at 31. Disk I/O is throttled, and on macOS that throttle applies to reading swapped pages back into memory. On the affected machine, the Claude Code sessions that had been running for two or three days had between 90 and 96 percent of their memory swapped out, according to vmmap --summary. One session on the same machine had been started by hand from a terminal instead of from the browser. It ran at normal priority and had 37 percent swapped.

So each keystroke into a browser-started session had to page a large heap back in at background I/O priority, behind Spotlight and whatever build was running. The session was not slow because it was busy. It was slow because it had been idle, and the OS had correctly identified it as something nobody was waiting on. The OS was wrong about that, but only because we told it so.

The PR also notes that taskpolicy -B cannot lift the clamp from a process that is already running. Only launchd can, and only at spawn time.

The change in 0.23.3

The plist now says Interactive, with a comment in service_darwin.go explaining why, so the next person to touch it does not "fix" it back. Adaptive was not an option: launchd moves an Adaptive job between the two classes based on activity over XPC connections, and our daemon has none. Its activity is a websocket to the relay and a set of ptys, which launchd cannot see.

Two tests pin the behaviour. One renders the plist and asserts it contains Interactive and never Background. The other writes a current plist to a temporary home and checks nothing needs refreshing, then rewrites it with Background and checks the refresh detector fires.

The refresh is not automatic. shell service status reads the installed plist and, if it finds <string>Background</string>, prints that sessions run at background priority and tells you to run shell service install. It does not do that for you, because reloading the agent restarts the daemon, and a daemon restart re-keys it. Any key a browser had sealed to the old daemon would be dropped. That is a decision the operator should make deliberately, ideally when no session is mid-task.

The upgrade notes on the v0.23.3 release spell out the consequence: sessions already running keep the old priority until they are restarted. New ones start at normal priority. Linux is unchanged. The systemd unit sets Restart=always and a ten-second RestartSec, and nothing else. It never set a Nice or an I/O scheduling class, so it never had this problem.

The general version of this bug

A launch agent that starts agent sessions is not a background job in launchd's sense, whatever the word "daemon" suggests. The thing it spawns is a process a person will type into, possibly hours from now, possibly after the machine has swapped it out. The same holds for any orchestrator that spawns long-running agents under a supervisor: check what class the supervisor puts you in, because your children inherit it.

We checked our own. The Pilot Protocol daemon that keeps this host on the overlay network is installed as a launch agent too, and the plist that install.sh writes declares no ProcessType at all. The man page says that is the same as Standard. No clamp, but also no explicit decision, and after this week an explicit one is worth writing down.

If you host sessions on a Mac, run shell service status after updating. If it tells you to reinstall, do it between tasks, and expect one restart of the daemon. The five-second keystroke goes away for every session started after that.

Top comments (0)