Agents fail in ways traces cannot explain. A trace records what an agent did: the tool it called, the latency, the response. It does not record why. The why lives in a belief that existed for perhaps half a second - "I assume the config is at /etc/app/config.yaml" - and by the time you open the trace UI, that belief is gone.
This is the gap I kept hitting while debugging agent pipelines. So I built the other half of observability: read the agent's beliefs straight from its own streamed output, while they are still happening.
Beliefs as first-class events
Axion is a middleware that sits in front of any OpenAI-compatible endpoint. It tees the model's streamed output (SSE passthrough, zero added latency) and extracts belief events: assumptions, intentions, and plans. Each event carries a confidence score, a timestamp, and a session attachment.
{
"type": "intention",
"belief": "the user wants a retry button on the error page",
"confidence": 0.8,
"span": { "start": 1234, "end": 1290 }
}
Extraction is pattern-based: deterministic, free, auditable. A dot only ends a clause when followed by whitespace or end of input, so URLs and file paths survive extraction intact. The patterns themselves are open source, so extending the vocabulary is a PR, not a research project.
The layer-on-top design
Axion deliberately does not replace Langfuse, Arize, or Braintrust. It emits belief batches over a webhook (axion.belief_batch.v1), signed with HMAC, which you can forward into your existing spans as structured metadata. Your trace pipeline stays; it gains a dimension it could not have: the reasoning that produced the behavior.
The suite
- VisReplay: full session recording (thoughts, tool calls, errors) into versioned files, replayed frame by frame
- VisCompile: deterministic behavioral diffs between runs, with a --fail-on-regression exit code for CI gates
- VisBoard: a shared workspace where agents and humans collaborate - scoped agent tokens, versioned notes with ETag semantics, live SSE events
Verification before shipping
Every repo was verified with live tests, not just unit suites: SSE belief streams with redaction against a mock upstream, byte-deterministic snapshots across repeated runs, full board CRUD and agent auth semantics against a real Postgres.
Honest gaps
- Belief extraction is pattern-based, not semantic. It catches stated beliefs, not implicit ones.
- Self-hosted only. There is no hosted multi-tenant lane yet.
- VisBoard automation currently ships webhook/notify actions.
Links
https://github.com/LatticeAG/axion
https://github.com/LatticeAG/visreplay
https://github.com/LatticeAG/viscompile
https://github.com/LatticeAG/VisBoard
https://latticeag.vercel.app
If you debug agents and keep asking "why did it do that" without an answer, this is exactly the gap it fills.
Top comments (0)