Every agent I've ever trusted has eventually done something I didn't ask for. Not maliciously — just confidently wrong. It wrote the right SQL against the wrong table. It "fixed" a config file and took down staging. It retried a failed tool call fourteen times because nobody told it to stop.
The usual answer is a better model. I don't buy it. The model isn't the failure point — the trust is. I was treating the LLM's output as if it were already correct, and then I was surprised when it wasn't.
The fix that actually worked: stop trusting the output, and verify it instead. Every agent output passes through a proof kernel before it's accepted. The model proposes. The kernel disposes.
There's a paper out that formalizes this better than I ever did — "AI with Authority, from Application to Silicon" (https://arxiv.org/abs/2608.21356v1). The Salt method, as they call it, runs verification at every layer, from the application down to the silicon. The headline idea is that verification isn't a final QA step — it's the thing that makes autonomy possible in the first place. An agent can only be trusted to work unattended if every output it produces is checked against a contract it can't argue with.
That matches what I've been doing by hand for the last year, so let me give you the practical version.
The kernel is not another LLM call.
This is the part people get wrong. They build a "verifier" that's just a second prompt — "check if this output is correct" — and then they're surprised when the second model agrees with the first one. That's not verification, that's two people nodding at each other.
A proof kernel is deterministic. It's a piece of code that takes the agent's output and checks it against a formal contract. No judgment, no vibes. It either passes or it doesn't.
Concretely, here's what I mean. I run an agent that writes SQL against our warehouse. The kernel:
- parses the SQL and confirms it's syntactically valid
- checks every table and column against the schema manifest
- confirms the query is read-only — no UPDATE, no DELETE, no DDL
- verifies it only touches tables the agent is allowed to see
If any check fails, the output is rejected. Not fixed, not nudged — rejected. The agent gets the rejection reason and tries again. That's the loop.
Another one: an agent that edits code files. The kernel diffs the proposed change against a manifest of files the agent is allowed to touch, and rejects anything outside it. It also runs the linter and the test suite before the change is accepted. The agent doesn't get to merge anything that doesn't pass.
The real work is the contract, not the model.
Here's the honest part. Writing the kernel is easy. Writing the contract — the spec that says what "correct" means — is the actual job. It's boring, it's detailed, and it's where ninety percent of the value lives.
The model is the part that's cheap and fast to swap. The contract is the part you'll still be maintaining in two years. If you're not willing to write the contract, you don't get the autonomy. That's the trade, and it's not a bad one.
What it costs you.
It costs latency. Every output has to be checked, and the check isn't free. For my SQL agent, the kernel adds maybe a hundred milliseconds. For the code-editing agent, the test suite is the bottleneck — that's minutes, not milliseconds. You have to decide where the gate lives.
It also costs flexibility. A proof kernel is the wrong tool for genuinely open-ended tasks. If your agent's job is "write me a strategy memo," there's no contract to check it against, and a kernel is pointless. But most production agents aren't open-ended. They're doing a narrow task with a defined contract — and if you can't write the contract, you probably shouldn't be running the agent unattended in the first place.
What it buys you.
The thing that surprised me: the kernel is what made unattended operation possible at all. Before, I was babysitting — watching logs, waiting for the agent to do something weird, ready to kill the process. After the kernel, I stopped watching. The agent can retry, loop, work overnight, because every accepted output has been verified. The failures still happen, but they happen inside the loop, where they're cheap, instead of in production, where they're expensive.
The Salt paper's point about verifying at every layer — application to silicon — is the same principle applied downward. If you only verify the final output, you miss the bugs that happen in the middle. The agent calls a tool, gets a result, and makes a decision based on a hallucinated intermediate value. The final output might look fine. The kernel at the top won't catch it. You need checks at the boundaries, not just at the end.
I haven't run Salt end to end myself — the paper is fresh, and I'm not going to pretend I've deployed it. But the core move, verification as the gate that makes autonomy safe, is something I've been running in production for a year. It's the difference between an agent I supervise and an agent I trust.
The model proposes. The kernel disposes. That's the whole tip.
Top comments (0)