Over the past month, I've been building an open-source project called Agent CLI.
Today it's a local-first, terminal-native agent harness that can work across code, shell, Git, browser, research, DOCX/PDFs, and delivery.
It started with a simple question:
What would an agent be like if I care less about how many capabilities it has, and more about whether I could trust what it actually does?
Starting again after building too much
Before Agent CLI, I already built a much larger local AI system called Agent OS.
It explored multi-agent orchestration, memory, Git workflows, deployment, browser automation, recovery, research, skills and GUI.
It was a useful learning project, but it also exposed a problem: adding capabilities faster than architectural guarantees makes an agent look overly mature.
So instead of expanding the same system, I started Agent CLI around a narrower thesis:
Build a trustworthy kernel first. Expand the workflows later.
The basic loop became:
Understand → Plan → Act → Observe → Verify → Record → Resume
What began as a design principle gradually became a test for almost every feature I added later.
An agent is more than an LLM with tools
The interesting engineering problem isn't about the model.
Once an agent can modify files, execute commands, access network, or publish code, the important part becomes the harness around the model.
The model can propose an action;
The harness should decide whether it's allowed.
The model can say a test passed;
The harness should know which process ran and the exit code.
The model can say a file was fixed;
The harness should know what bytes changed and whether verification passed.
This led to an important rule:
The model may reason about reality, but it should never define reality.
Whenever possible, reality comes from files, hashes, process results, diffs, Git state, browser observations, and other external evidence.
Evidence over narration
Agent runs naturally produce persuasive narratives:
Implemented the feature, fixed the bug, and all tests pass.
But many subtle failures can hide behind that sentence.
A test may run before the final edit, happen in the wrong package, or result in a command time-out. And the model may interpret partial output as success.
Agent CLI therefore records important actions into an append-only event log: approvals, commands, file mutations, hashes, verification results, and other consequences.
This sounds like implementation detail, but it changed the architecture significantly.
Permission, approval, and sandboxing are different problems
Agent tools often expose broad modes such as safe, automatic, or full-access. They are convenient, but they can also hide what is actually authorized.
Agent CLI instead evaluates individual actions through a centralized policy gate.
Tools declare facts about their effects — running a command, mutating files, reading remotely — and the runtime decides whether it's allowed, requires approval, or denied.
This also forced me to separate three similar concepts:
- Trust: permission to operate inside a workspace;
- Approval: authorization for a particular consequential action;
- Sandboxing: technical enforcement by the operating system.
Agent CLI currently has real OS-level sandboxing for eligible auto-run commands on Windows, using Low Integrity and a Job Object.
On platforms where the same enforcement is unavailable, command auto-run fails closed instead of pretending the guarantees are equivalent.
Reversibility is a form of autonomy
As agents gain more authority, the obvious solution is to ask users for more approvals. But an agent that asks before every small edit becomes a baby you have to look after all the time.
A better solution for many local operations is to make them cheaply reversible.
Agent CLI snapshots workspace changes before applying them. /undo can restore them later and refuses to overwrite files that have drifted. Larger recovery points can use hidden Git references without moving the user's branch or HEAD.
Delegated sub-agents work in disposable Git worktrees, and their changes reach the main workspace only through explicit integration.
However, not every action fits this model. Publishing, destructive operations, and sensitive external effects still deserve stronger approval boundaries.
But for ordinary local work, reversibility can often buy useful autonomy without reducing control.
General-purpose shouldn't mean improvising everything
Agent CLI started as a coding agent, but I wanted to know whether its architecture could be extended to other workflows.
DOCX and PDFs became an early test.
Instead of creating a separate "document agent", the same runtime writes a structured specification, deterministic software then renders the artifact, the result is parsed back for validation, and a vision-model can finally inspect the real output.
This led to a broader idea:
High-quality agent workflows need an intermediate representation between language and effects.
The model handles ambiguity and high-level decisions. Deterministic software handles repeatable transformations and validation.
That seems much more promising than letting the model rediscover an ad-hoc workflow every time.
More agents doesn't mean a better agent system
Agent CLI eventually gained planning, researching, reviewing, and delegated workers.
But building those features actually made me less convinced by the "agent team" metaphor.
Once the implementation becomes concrete, multi-agent execution becomes a familiar system problem:
How can we delegate bounded work, isolate concurrent mutation, preserve provenance, and safely integrate outcomes?
A child agent should inherit the same or narrower authority;
A mutating worker shouldn't casually share the parent's workspace;
An approved plan should refer to specific content, not a conversational "yes".
Those boundaries matter more than how many agents appear in the terminal.
Agent engineering is increasingly system engineering
Models will keep getting stronger.
But stronger models also mean we'll delegate more consequential work to them, which makes the surrounding harness more important rather than less.
For me, the interesting questions are increasingly:
- What may happen?
- What actually happened?
- How do we know it works?
- What happens if it fails halfway through?
- Can someone else inspect the evidence and reach the same conclusion?
These aren't new questions.
Operating systems, databases, version control, security engineering, and distributed systems have been dealing with them for decades.
The model is new.
Many of the engineering problems around it are not.
Agent CLI is still an early experiment rather than a polished product.
The project is fully open source under MIT, and I'm sharing it mainly as part of the learning process instead of a commercial launch.
If you are also working on coding agents, local AI systems, agent runtimes, or similar things, I'd be genuinely interested in criticism or other approaches.
GitHub: https://github.com/earthwalker17/agent-cli
Issues, PRs, and technical discussions are all very welcome.
Top comments (0)