We keep calling them “agents,” and we keep managing them like software.
That’s the mistake.
Software is a fixed artifact. You can diff it, version it, roll it back, and trust that yesterday’s build behaves like today’s. An agent is not a fixed artifact. It’s a live composition of a model, a prompt, a memory store, a set of tools, a permission structure, and a pile of assumptions about the world—any one of which can shift on its own, silently, without anyone touching a line of code.
Treat that like a normal deploy, and you’ll spend the next year debugging ghosts.
It’s time developers had a system of record for agents.
When an agent behaves differently today than it did yesterday, the cause is almost never sitting neatly in a code diff.
It’s a model that quietly updated. A prompt someone tweaked in a Slack thread. A tool schema that shifted upstream. A memory that went stale. A retrieval index that drifted. A permission that got revoked. An environmental assumption that simply stopped being true.
Git tracks code.
It was never built to track a mind.
And the mind is what actually behaves.
We don’t deploy code anymore.
We deploy intelligence.
Agent sprawl happens quickly
Every team’s agent journey begins the same deceptively tidy way: one agent, one workflow, one model, one prompt, one set of tools.
Clean. Legible. Controllable.
It never stays that way.
Another team wants a variation. A customer demands a custom workflow. The underlying model gets swapped. A tool changes shape. A permission tightens or loosens. Someone bolts on memory. Someone forks the agent for a new environment. And somewhere, quietly, an old version is still running—nobody quite remembers where, why, or what it is still doing.
Within months, organizations lose the ability to answer questions that should be trivial:
- What agents do we actually have?
- What can each one do?
- Which components can be safely reused?
- Which versions work and which fail in production?
- Can this agent be trusted somewhere new?
The moment agents become useful, they become infrastructure.
And infrastructure without a system of record isn’t infrastructure.
It’s just risk.
Benchmarks are necessary, but not sufficient
The instinct is to reach for benchmarks and security testing.
Run the eval, get the score, ship if it’s green.
But these methods answer a narrow question: pass or fail?
- They don’t tell you what the agent relied on to get that answer. - They don’t tell you what shifted in the surrounding stack between this run and the last.
- They don’t tell you whether a “fix” genuinely strengthened the system or quietly patched one failing test while leaving the underlying fragility intact.
Git versions code.
Benchmarks score behavior.
Observability tools record executions.
None of them record the complete evolution of an agent.
The real question isn’t whether the agent got the right answer once. It’s whether the agent understands enough about its environment to keep getting the right answer when the environment inevitably changes.
That’s the discipline we think of as epistemic AI: understanding what an agent knows, what it doesn’t, what it’s silently assuming, and precisely where those assumptions fracture.
Physical AI makes this impossible to ignore. A robot that performs flawlessly in one warehouse does not automatically perform in the next. The same is true of a hospital, factory, lab, or home.
Intelligence does not transfer on faith.
Introducing Octo
Every other layer of the stack has a system of record.
Code has Git. Infrastructure has Terraform state. Data has lineage tools.
Agents have nothing equivalent.
Octo changes that.
Every time an agent changes, Octo captures the complete agent stack:
- Models
- Prompts
- Tools and MCP servers
- Memory and retrieval
- Permissions
- Runtime configuration
- Environment assumptions
Not just traces after the agent runs. Not just scores after a test.
The actual deployable shape of the agent itself.
Instead of wondering what changed, teams get a version history of the agent:
- Which customer is running this version?
- What changed in the orchestration?
- Why did this deployment suddenly start failing?
- Can this agent safely move into a new environment?
- How should I change an agent so it works in a new place for a new customer?
Those stop being detective work.
They become ordinary answerable engineering questions.
Imagine debugging a production incident.
Git shows no code changes. Your benchmark still passes. But a retrieval index drifted, a tool schema changed upstream, and one permission quietly disappeared.
None of that lives in a code diff.
Octo shows the complete history of the agent—not just the repository.
We believe every production agent will eventually have a version history.
Every deployment will carry evidence.
Every redeployment will begin with understanding what changed.
That’s the future we’re building toward.
Today, that future starts with Octo.
We’re opening early access to the first 100 users.
Join the Octo early access waitlist: https://pelagicplatforms.com/octo/waitlists/
Top comments (0)