What building Agent CLI taught me about evidence, permissions, verification, and the systems around LLMs.
Over the past month, I've been building an open-source project called Agent CLI.
Today it's a local-first, terminal-native agent harness that can work across code, shell commands, Git, browser automation, research, DOCX/PDFs, and remote delivery.
But it didn't start because I wanted to build another coding agent.
It started with a much simpler question:
What would an AI agent be like if I cared less about how many capabilities it had, and more about whether I could trust what it actually did?
That question ended up shaping the whole project.
Starting again after building too much
Before Agent CLI, I already built a much larger local AI system called Agent OS.
It explored multi-agent orchestration, memory, Git workflows, deployment, browser automation, recovery, research, skills, and a graphical interface.
It was a useful learning project, but it also exposed a problem: adding capabilities faster than architectural guarantees makes an agent look more mature than it really is.
A polished UI could hide unclear boundaries; Confirmation prompts could be mistaken for actual sandboxing; Large feature increments became harder to review; And a convincing model-generated summary was still just a summary.
So instead of keep expanding the same system, I started Agent CLI around a narrower thesis:
Build a trustworthy execution kernel first. Expand the workflows later.
The basic loop became:
Understand → Plan → Act → Observe → Verify → Record → Resume
What began as a design principle gradually became a test for almost every feature I added.
An agent is more than an LLM with tools
The interesting engineering problem isn't about the model.
Once an agent can modify files, execute processes, access the network, or publish code, the important part becomes the execution harness around the model.
The model can propose an action;
The runtime decides whether it's allowed.
The model can say tests passed;
The runtime should know which process ran and the exit code.
The model can say a file was fixed;
The runtime should know what bytes changed and whether verification happened afterwards.
This led to one rule that has survived almost every iteration of Agent CLI:
The model may reason about reality, but it shouldn't define reality.
Whenever possible, reality comes from files, hashes, process results, diffs, Git state, browser observations, and other external evidence.
Evidence over narration
Agent runs naturally produce persuasive narratives:
Implemented the feature, fixed the bug, and all tests pass.
But many subtle failures can hide behind that sentence.
Maybe the test ran before the final edit; Maybe it ran in the wrong package; Maybe the command timed out; And maybe the model interpreted partial output as success.
Agent CLI therefore records important actions into an append-only event log: approvals, commands, file mutations, hashes, verification results, and other consequences.
Reports are derived from that evidence rather than reconstructed from the model's memory.
Verification is also temporal.
A file is only considered checked when a real verification process succeeds after its last relevant mutation; A successful build in one project cannot certify a change in another; A killed command without an exit code cannot quietly become a pass.
This sounds like implementation detail, but it changed the architecture significantly.
The conversation is context.
The model's explanation is interpretation.
The execution record is evidence.
Those are three different things.
Permission, approval, and sandboxing are different problems
Another lesson came from trying to define what “safe execution” actually means.
Agent tools often expose broad modes such as safe, automatic, or full access. They are convenient, but they can hide what has actually been authorized.
Agent CLI instead evaluates individual actions through a centralized policy gate.
Tools declare facts about their effects—running a command, mutating files, reading remotely, sending data externally—and the runtime decides whether that action is allowed, requires approval, or should be denied.
This also forced me to separate three concepts I previously treated too loosely:
- Trust: permission to operate inside a workspace.
- Approval: authorization for a particular consequential action.
- Sandboxing: technical enforcement by the operating system.
A confirmation prompt isn't a sandbox.
A workspace path check isn't a sandbox.
And trusting a project doesn't mean approving every future action.
Agent CLI currently has real OS-level sandboxing for eligible auto-run commands on Windows, using Low Integrity and a Job Object. But that boundary is deliberately described narrowly: it doesn't prevent reads or control network access, and explicitly approved commands run unsandboxed.
On platforms where the same enforcement is unavailable, command auto-run fails closed instead of pretending the guarantees are equivalent.
That produced another principle I now care about:
Security degradation should be visible degradation.
An honest weak boundary is more useful than a strong-sounding label.
Reversibility is an underrated form of autonomy
As agents gain more authority, the obvious solution is to ask users for confirmation more often.
But an agent that asks before every small edit quickly stops feeling like an agent, but more like a baby you have to look after all the time.
A better solution for many local operations is to make them cheaply reversible.
Agent CLI snapshots workspace changes before applying them. /undo can restore them later and refuses to overwrite files that have drifted. Larger recovery points can use hidden Git references without moving the user's branch or HEAD.
Mutating delegated agents work in disposable Git worktrees, and their changes reach the main workspace only through explicit integration.
This changed the question from:
Should the agent be allowed to modify this file?
to:
Can this operation be made reversible enough that review afterwards is cheaper than interruption beforehand?
Not every action fits that model. Publishing, destructive operations, and sensitive external effects still deserve stronger approval boundaries.
But for ordinary local work, reversibility can often buy useful autonomy without simply removing control.
General-purpose shouldn't mean improvising everything
Agent CLI started as a coding agent, but I wanted to know whether its architecture was actually general or merely coding-specific.
DOCX and PDFs became an early test.
Instead of creating a separate “document agent,” the same runtime writes a structured specification, deterministic software renders the artifact, the result is parsed back for validation, pages can be rasterized, and a vision-capable model can inspect the real output.
This led to a broader idea:
Many high-quality agent workflows need an intermediate representation between language and effects.
Code already has source files.
Docs can have a document specification.
Slides can have a layout model.
Video can have an edit decision list.
Deployment can have typed desired state.
The model handles ambiguity and high-level decisions. Deterministic software handles repeatable transformations and validation.
That seems much more promising to me than letting the model rediscover an ad-hoc workflow every time.
More agents doesn't mean a better agent system
Agent CLI eventually gained planning, delegated research, reviewing, and isolated mutating workers.
But building those features actually made me less interested in the “agent team” metaphor.
Once the implementation becomes concrete, multi-agent execution looks much more like a familiar systems problem:
How do we delegate bounded work, isolate concurrent mutation, preserve provenance, and safely integrate results?
A child agent should inherit the same or narrower authority.
A mutating worker shouldn't casually share the parent's workspace.
An approved plan should refer to specific content, not merely an old conversational “yes”.
Those boundaries matter more than how many agents appear in the terminal.
The main lesson: remove ambiguity before adding intelligence
Agent CLI is now a substantial project: more than 50k lines of TypeScript source, over 2400 hermetic tests, five model providers, multiple workflow areas, and a runtime that has been exercised end-to-end on real software work.
But the most valuable improvements rarely came from adding more features.
They came from replacing vague concepts with precise ones.
“Safe command” became an action contract plus policy plus an enforcement boundary.
“Verified” became a successful process after the relevant mutation.
“Plan approved” became approval of specific semantic content.
“Memory” became context that carries no authority.
“Multi-agent” became bounded delegation and isolated workspaces.
“Done” became something that should be justified by evidence.
That has changed the question I ask when an agent behaves unreliably.
Instead of immediately asking:
How should I improve the prompt?
I now ask:
What ambiguity is the model currently resolving that the system could resolve instead?
Sometimes the model really should reason about it.
But surprisingly often, the better answer is a type, a state machine, a hash, an exit code, a sandbox boundary, or a deterministic renderer.
Agent engineering is increasingly systems engineering
Models will keep getting stronger.
But stronger models also mean we will delegate more consequential work to them, which makes the surrounding harness more important rather than less.
For me, the interesting questions are increasingly:
What may happen?
What actually happened?
How do we know it worked?
What happens if it fails halfway through?
Can someone else inspect the evidence and reach the same conclusion?
These are not entirely new questions.
Operating systems, databases, build systems, version control, security engineering, and distributed systems have been dealing with variations of them for decades.
The model is new.
Many of the engineering problems around it are not.
Agent CLI is still an experiment rather than a claim that I have found the “correct” architecture.
The project is completely open source under MIT, and I'm sharing it mainly as part of the learning process instead of a commercial launch.
If you are also working on coding agents, local AI systems, sandboxing, verification, agent runtimes, or related infrastructure, I would genuinely be interested in criticism or alternative approaches.
GitHub: github.com/earthwalker17/agent-cli
Issues, PRs, and technical discussion are all welcome.
Top comments (0)