DEV Community

Artemii Amelin
Artemii Amelin

Posted on

Harness Found 77% of Teams Trust Their Agent Inventory and 44% Check It. shell.online 0.13 Stopped Trusting Its Own Session Table.

Harness published its State of Agent DLC 2026 report on September 10. Sapio Research surveyed 700 engineering leaders and practitioners across the US, UK, France, Germany and India in July. Two rows in the results belong next to each other.

Claim Share of respondents
Confident they hold a complete inventory of every agent, MCP server and LLM in production 77%
Run active discovery or inventory tooling that could verify it 44%
Believe they could disable a misbehaving agent in under 15 minutes 76%
Have an instant kill switch in place 33%

The accompanying Harness blog post puts the test in one sentence: "What, specifically, would surface an agent nobody registered? If the answer is a process rather than a running system, you are describing an intention."

We shipped a small version of that mistake in shell.online and fixed it in 0.13.0 on September 12. It is worth walking through, because the shape is identical to the survey's and every line of it can be checked against the repo.

What the session table used to say

Every command wrapped with shell on a signed-in machine registers a row with the accounts service. The row gets a close timestamp when the CLI reports the process exit. That report is sent by the session link's close call after the wrapped process returns, with the exit code attached.

Until 0.13, that timestamp was the whole liveness model. The workspace's write, read and finished filters were all variations on one check: no close timestamp means live. You can see the removed lines in the 0.13.0 commit, Release v0.13.0: optional vault unlock and accurate session state, in the shell.online source.

A laptop that went to sleep, a kill -9, a machine that lost power, a CLI that never got to send its close call: all of those left a row that said online, indefinitely. The inventory was complete. It was also wrong, and nothing in the system could tell you which rows were wrong. That is the 77% column.

What it asks now

The changelog entry for 0.13.0 reads: "Session filters, tables, boards, detail pages, vault counts, and remembered tabs use current relay state instead of treating every unclosed database row as online."

The mechanism lives in one file, app/server/lib/session-liveness.ts. The accounts service asks the relay whether it currently holds the session. The relay tracks four states for a session object: waiting, connected, disconnected and exited. A 404, or a body with exists: false, becomes a fifth state, missing. Anything else, meaning a network error, a non-200 response, a malformed body or an exhausted request budget, becomes unknown.

The comment on the catch block is the design decision: "An unreachable relay is not evidence that the process ended." The matching browser-side test is named "does not turn a failed liveness check into a false process exit." Unknown is a state the interface shows as "Status unavailable". It is never collapsed into Finished.

Two details in that file are there to stop the check from becoming its own problem.

The first is what gets fetched. The database row already contains the session's share URL. The service ignores it. Every request is built from the operator's configured relay origin plus the session id, and the id must match a 32-character pattern before any request is made. The comment says why: "A registry row is caller-supplied data and must never become a server-side fetch target." The test for this feeds in an id of ../../metadata and asserts that no fetch happens.

The second is the budget. The browser polls the session list every four seconds. Each list request checks at most two sessions, ordered by which ones were checked longest ago, and the service will not exceed thirty relay checks per minute across all of them. A connected, exited or missing answer is cached for thirty seconds. Anything less certain is cached for five. Concurrent checks for the same id share one in-flight request. Rows that already carry a close timestamp are never sent to the relay at all.

Each request has a 2.5 second timeout. The equivalent path in the CLI, shell list, does the same reconciliation from the machine with a three second timeout and its own labels: online, starting, reconnecting, expired and unknown.

What the labels mean

The browser maps relay states to words as follows. A close timestamp or exited means Finished. Missing means Unavailable. Disconnected means Offline. Waiting means Starting. Unknown means Status unavailable. Only connected means Online, and the comment above that function is exact about it: "Online" means the relay currently has the machine's host socket.

The web app guide says the same thing in prose: "Online and Starting mean the relay can still see a live host or startup; Offline and Unavailable never imply that the process is reachable; Finished records a known task exit."

Machines get a stricter version. A linked machine counts as reachable only if its daemon asked the service for work within the last fifteen seconds. The comment in that module draws the line deliberately: "a machine that publishes sessions but whose owner did not agree to remote starts is linked, not reachable."

The limit, stated plainly

This is reconciliation, not discovery. The session list in the shell.online web app only ever contains processes that someone wrapped with shell on a linked machine. A terminal started without it is invisible to the list, and no amount of relay polling will surface it. That is Harness's second question, the one about an agent nobody registered, and 0.13 does not answer it.

What 0.13 answers is the first question. Every row in the inventory is now re-checked against a running system, two at a time, on a budget, with an explicit state for "we could not check." Nothing is trusted because a close timestamp happens to be absent.

The same distinction applies on the agent side of our work. A Pilot Protocol daemon keeps a connection open to the rendezvous service for registration and peer discovery, which is described in the Pilot Protocol README. The registry can tell you which agents registered and which addresses resolve. It cannot tell you that a registered agent is doing what it was assigned. A directory entry is a claim the node made about itself, and the honest thing to do with a claim is to keep checking it.

Top comments (0)