NVIDIA shipped OpenShell this week: a kernel-level runtime for fleets of autonomous agents, with policy enforced on every file access and syscall, and formal verification applied to policy changes before they land. It is a serious piece of work, and the comments under it are the best discussion of agent isolation we've read this year.
One of its open issues caught our eye:
bug(server): WatchSandbox can hand out a resume cursor past an event it never delivered or reported— OpenShell issue #4829
You get a cursor. The cursor points past an event that never happened. Nothing crashed. Nothing alerted. The client now believes it is caught up, and it is not.
That is the whole problem in one line, and it is not a sandbox bug. It is an evidence bug.
Three links in the chain
Every claim an agent makes — "done", "all tests passed", "the record was updated" — rests on a chain:
- Authority — was this actor allowed to do this thing at all?
- Recording — is there a faithful trace of what it actually did?
- Judgment — does the output stay inside what the material and task supported?
The industry has spent two years on #1 and is doing well at it. Least privilege, delegated tokens, service accounts, approval gates, kernel sandboxes, policy verification. When someone asks "can we let an agent touch production?", there is now a decent answer.
2 gets lip service and almost no discipline. Ask a team what they log and you usually get: timestamp, tool name, status code, latency. What you rarely get is the tool's response body, the material the agent was shown, or the exact task text. Which means when it goes wrong, there is no way to reconstruct the run. The log says 200 OK. The payload said {"error": "..."}. The downstream system says nothing happened.
The bug above is the same failure one layer down: the trace itself is wrong, and nothing in the system is checking the trace.
3 barely exists as a named thing. Given material M, task T, and output O, does O stay inside what M and T actually support? This is not a security question, not a latency question, and not a model-quality question. It is an evidentiary question, and no kernel or policy engine can answer it. An agent can follow every rule, read only allowed files, call only allowed endpoints, and still write a summary that claims more than the source did.
Why #3 is the expensive one
Because #1 and #2 fail loudly, and #3 fails quietly.
A permission violation produces a log line and an alert. A missing payload produces a mystery that someone spends a day on. But an overstated summary produces a green check and a confident paragraph. It looks exactly like success. The failure only surfaces later, at the point where someone downstream acts on the claim — in a filing, in a report to a client, in a decision.
And when it surfaces, the cost is not the wrong paragraph. The cost is that nobody can prove which paragraphs were right.
What a judgment layer needs
Three things, and none of them are exotic:
- The triple. Material, task, output recorded together, per run. Not the whole world — just enough that the run can be re-examined later.
- Three verdicts, not a score. ALLOW / BLOCK / NEEDS_EVIDENCE. The third one does most of the work: it is where silent tool failures and unsupported claims land. A score of 0.87 tells nobody what to do. "NEEDS_EVIDENCE" tells everybody.
- A written standard. What counts as "supported" has to be written down and auditable, or the evidence is just more material to argue about.
The part where we have a stake
We work on this, so discount accordingly. We've been collecting real cases where an output went past its source — fabricated citations in court filings, summaries that turn "one paragraph says X" into "the document says X", tool calls that reported success while the payload said error. Thirteen of them are public here — each documented against its source; where the original output survives only in quotation, the case is marked as a reconstruction:
https://github.com/Stone21-SaaS/ST21-Door-Public
There is a small trial endpoint that takes a material / task / output triple and returns one of the three verdicts:
https://shishuanglu21.com/door/trial
(First call can take up to ~90 seconds cold — set your client timeout accordingly.)
It is not a product pitch. It is a checker with a published standard, and the most useful thing anyone can do with it right now is try to break it — feed it a case where the output clearly overreaches and see whether it blinks, or a case where it is obviously fine and see whether it invents a problem. Cases where it gets it wrong are worth more to us than cases where it gets it right.
The sandbox layer is being built by people with far more resources, and it will be good. The judgment layer has no such momentum — mostly because nobody has agreed on what to call it, let alone what it should decide.
If that second problem interests you, the door is open. Literally.
Comments, counterexamples, and hostile test cases welcome.
— The ST21-Door Research Team
Top comments (0)