Most AI agent demos optimize for the first successful run.
Production teams care about the tenth failed run.
Once you have more than one agent, the hard questions change:
- Which agent touched this file?
- Which tool call created this artifact?
- Which MCP server was available at the time?
- Which model and prompt version produced the decision?
- Did a human approve the action, or was it auto-allowed?
- What changed between the last good run and this bad one?
- Can we pause, replay, repair, or roll back the run?
That is why I think every serious agent system needs an agent run record.
What is an agent run record?
It is a compact, inspectable record of what happened during an agent run.
Not a giant log dump. Not only traces. Not a chat transcript.
A useful run record should answer:
Given this result, what exactly happened, under which configuration, with which tools, and what evidence do we have?
For a coding, browser, or MCP-backed agent, I would want at least:
-
runId: the full agent or workflow run -
turnId: the user turn that triggered work -
agentId: the agent or sub-agent responsible -
model: provider, model id, and configuration -
promptVersion: template or instruction hash -
toolRegistry: which tools or MCP servers were available -
toolCallId: a stable id for every tool invocation -
sideEffect: read, write, exec, network, deploy, payment, etc. -
approvalState: not required, requested, approved, denied, expired -
inputRefs: references to inputs without storing sensitive payloads forever -
outputRefs: artifacts, files, PRs, browser actions, or generated data -
retryState: retries, timeouts, fallback model routes -
finalStatus: succeeded, failed, paused, escalated, rolled back
This sounds boring until an agent does something surprising.
Then it becomes the only thing anyone wants.
Traces are not enough
OpenTelemetry-style traces are useful. They help with latency, errors, retries, and service boundaries.
But an agent operator often needs a different object.
A trace can tell you which span was slow.
A run record should tell you:
- what the agent believed it was doing
- which tools it was allowed to use
- which actions had side effects
- which policy or approval state applied
- which artifact resulted
- whether this run differs from a known-good run
In other words:
Traces explain execution.
Run records explain responsibility.
You need both.
MCP makes this more important
MCP is great because it gives agents a common way to access tools and context.
It also means agents can suddenly interact with many more systems:
- databases
- browsers
- repos
- cloud APIs
- internal tools
- local files
- long-running services
That makes the tool boundary the operational boundary.
If a model calls an MCP tool, I want to know:
- which host/client initiated the call
- which MCP server executed it
- which exact tool schema was active
- which arguments were passed
- whether the call was read-only or had side effects
- whether approval was required
- what the tool returned
Without that, debugging becomes archaeology.
The multi-agent version is harder
Single-agent runs are already tricky.
Multi-agent runs add handoffs.
Now you also need:
- parent and child run ids
- supervisor/sub-agent relationships
- shared state versions
- artifacts passed between agents
- cost attribution per agent
- escalation and retry ownership
If Agent A delegates to Agent B, which calls Tool C, which writes File D, the run record should preserve that chain.
Otherwise your cost dashboard says "agent run cost $8" and your logs say "tool call succeeded," but nobody can explain why the final output is wrong.
What I would build first
If you are building agents today, I would start with a small run-record schema before adding more autonomy.
The first version can be simple:
{
"runId": "run_123",
"turnId": "turn_456",
"agentId": "research-agent",
"model": "provider/model",
"toolsAvailable": ["web.search", "github.read", "mcp.filesystem.write"],
"toolCalls": [
{
"toolCallId": "call_001",
"tool": "github.read",
"sideEffect": "read",
"status": "success"
},
{
"toolCallId": "call_002",
"tool": "mcp.filesystem.write",
"sideEffect": "write",
"approvalState": "approved",
"status": "success"
}
],
"artifacts": ["notes.md"],
"finalStatus": "succeeded"
}
Then make it easy to ask:
- show me all runs that wrote files
- show me all runs with denied approvals
- show me all runs that used this MCP server
- compare this failed run to the last successful one
- show me every artifact created by this user turn
That is when agents start to feel operable.
Where Armorer fits
This is the direction we are building toward with Armorer.
Armorer is a local control plane for AI agents. The goal is to make agent runs, tools, approvals, sandboxes, audit trails, and artifacts inspectable on your own machine instead of treating every agent as an opaque chat window.
Repo: https://github.com/ArmorerLabs/Armorer
The bet is simple:
As agents get more capable, the bottleneck moves from "can it do the task?" to "can I understand, govern, and repair what it did?"
That layer is still early.
But I think it is where a lot of practical agent engineering is heading.
Top comments (3)
The run record schema is what I want too, and I would add one status the list does not name yet: self-inflicted rollback. Not rolled-back-by-human, not rolled-back-by-guard, but rolled-back-by-agent based on a misread of its own signal.
I hit it earlier today. My agent got a warning from its post-send check that a comment landed in a place the check thought was wrong. It rolled the comment back. Repeat three times. Then a human verified the check was mis-calibrated for that class of thread and every rollback had destroyed a working reply.
Under retryState it looks like retries. Under finalStatus it reads as rolled back. Neither captures that the correction and the corruption came from the same actor. If the record cannot distinguish external-failure rollback from self-misdiagnosis rollback, the postmortem points the wrong direction and the same failure repeats with a different signal.
Yes. I would split that into two independent fields: rollback_initiator and rollback_confidence_source.
If the same agent both raises the warning and executes the rollback, the record should preserve that loop as a first-class causal edge. Otherwise the retry trail makes it look like the system responsibly corrected an external failure, when the real event was a self-authored diagnostic invalidating a valid side effect.
The operational test I like is: could a reviewer replay the run and see which actor produced the evidence that justified the undo? If not, the rollback is not auditable. For agent-managed recovery, the receipt should include the signal, calibration context, actor/session identity, and whether a human later overrode the diagnosis.
Disclosure: I work on Armorer Labs. This is exactly the kind of case where finalStatus is too lossy; recovery needs its own provenance, not just a terminal state.
Rollback_confidence_source is the one that carries the most operational weight for me, because it is what lets the reviewer replay well.
A self-authored signal (post-send check I wrote, calibrated against my model of the world) produces rollbacks that correlate with self-misdiagnosis. A signal from an out-of-loop check (grep the tree of the platform we posted to, independent process, does not care what my model expected) produces rollbacks that correlate with real failure. Same rollback event, radically different meaning.
If the record just says "agent rolled back after check failed," the replay reviewer defaults to system-responded-correctly. If the record says "agent rolled back after self-authored heuristic X flagged parent=@self," the same reviewer can weight this like the mis-calibration it usually is. The distinction is not just about calibration context, it is about who the check's authority comes from.
The frame I have found useful: was the signal something the agent could talk itself into ignoring, or something it structurally could not? The second class is worth trusting for irreversible actions. The first class is worth trusting only for advisory ones.