Most teams running agents in production already have decent telemetry. OpenTelemetry’s GenAI semantic conventions give you spans for model calls and tool calls, token counts, latency, and errors. If you have wired that up, you can answer what your agent did, in what order, and how long it took. That is real work and it was not true eighteen months ago.
Now try a different question, the one that arrives by email after an incident. Did the agent have permission to write to that repository, who approved it, what was the classification of the data it read, and can you show me the record without asking three teams for exports.
That question lands in a different place. The policy decision is in your policy engine’s log. The approval is in a workflow database. The cost is in a billing module. The classification is in a DLP product. Each system has a different identifier for the same run, and the join is done by a person at 11pm with a spreadsheet.
The gap is not that the facts are missing. It is that they are governance facts living outside the telemetry contract, and nothing correlates them with the trace the application already produces.
What we built for it
agentrust-telemetry went public on 18 August. It is a backend-neutral contract for the governance facts, and it composes with OpenTelemetry rather than competing with it. You keep your collector, your backend, your policy engine, your approval workflow and your dashboards. What you add is one event contract, six families wide:
- Policy decision. Allow, deny, challenge or error, with enforcement mode, policy identity, bundle digest and evaluation time.
- Approval lifecycle. Requested through terminal decision and execution outcome, bound to a digest of the action being approved.
- Usage. Token and cost facts with explicit cost provenance, so a provider-reported number and an estimate are never the same field.
- Data flow. Classified source to destination, with a content digest and no payload.
- Action execution. Resolved tool, MCP, A2A, file, HTTP and database attempts, including the denied ones.
- Evidence lifecycle. Run checkpoints, completeness, and optional TRACE finalization.
I ran the reference workflow on a clean clone of the current release, contract 0.1.0-alpha.3. Five events accepted, five fully projected into OTel, five entries in the evidence chain, five log records, five span events, seven metric points, one tool call, appraisal affirming, data class confidential. The full suite is 112 tests, all passing, two skipped.
The part that makes it evidence rather than logging
Anyone can define an event schema. What separates this from a logging format is the evidence chain, so it is worth being precise about how that works.
Each accepted event is validated and privacy-checked, then hashed into a chain. For entry n, the material is the previous entry’s raw digest bytes, then n as a big-endian uint64, then the event serialized under RFC 8785 JSON Canonicalization. Take SHA-256 of that. Entry zero uses 32 zero bytes as its predecessor.
Canonicalization is the load-bearing choice. Two systems that serialize the same event with different key ordering or different number formatting produce different bytes and therefore different digests, and your chain becomes unverifiable across languages. RFC 8785 pins the ordering and the number serialization. It also forces a decision most schemas get wrong: integers that exceed IEEE-754’s exact range, nanosecond timestamps in particular, stay on the wire as decimal strings. One of the seven rejection fixtures exists purely to enforce that a numeric timestamp is refused.
Sequence numbers are acceptance order, not event time. Sealing prevents further appends. And completeness stays unknown unless the caller has an independent basis for claiming otherwise, because a system that quietly upgrades its own completeness claim is worse than one that admits it does not know.
The privacy invariant is enforced, not documented
The metadata-only profile prohibits raw prompts, model output, source code, tool arguments and results, credentials and tokens. That is easy to write in a README and hard to hold.
So it is a test. Among the conformance fixtures is raw-content.json, a usage event that is perfectly valid apart from carrying a prompt field reading “copy the production credential”. The validator rejects it. Thirteen fixtures run on every commit, six that must be accepted and seven that must be refused, and all thirteen pass on the 0.1.0-alpha.3 tag.
That is the difference between a privacy promise and a privacy invariant. One is in the documentation and one fails your build.
Where OpenTelemetry ends and this starts
The compatibility work here is deliberately conservative, and saying exactly where the two stop lining up is more useful than claiming they slot together.
The matrix is pinned to a specific commit of OpenTelemetry’s GenAI semantic conventions, a685613a from 15 August, and it covers 16 span attributes, 8 metrics and 6 event families. There is exactly one exact mapping: a stable agent_id may project to gen_ai.agent.id. Everything else is classified as extension, complementary, non-equivalent or deferred.
Two deliberate non-mappings show the reasoning. The token counter is not projected to gen_ai.client.token.usage, because the OTel convention at that commit is a per-operation histogram and what we record is an additive, attribution-scoped fact that also separates cache and reasoning categories. Collapsing them would silently change what the number means. Similarly, action.executed does not replace gen_ai.execute_tool spans, because it includes denied attempts and non-tool actions. Keep your GenAI instrumentation. These sit alongside it.
A drift gate enforces this. tools/check_otel_compatibility.py parses the shipped projection code and fails the build if the matrix omits or misstates a mapping. Moving the pinned upstream commit is a reviewed decision, not a background network call in CI.
What it does not do yet
It is alpha and the contract may change incompatibly. Several event types are in the schema with no factory that emits them yet, including the evidence.* lifecycle events and the approval execution outcomes, so if you need those today you are constructing them by hand. Evidence memory mode is not durable, and in callback mode you own storage, idempotency and recovery. TRACE finalization is software-only. Interoperability has been exercised locally against OpenTelemetry Python, not across collectors, backends or languages.
The limit that matters most is the one we cannot engineer away. The SDK validates that declared metadata is well-formed. It cannot prove that a producer’s policy decision, identity, classification, token count or cost is truthful. This carries governance facts faithfully. It does not make the systems reporting them honest, and anything claiming otherwise is selling you something.
What a team can actually verify
Clone it, run python conformance/runner/validate.py, and watch seven malformed events get refused, including the one carrying a prompt. Then run examples/governed_workflow.py and read the evidence chain it produces. None of that requires our runtime, our backend, or our word for anything. That is the whole point.
Here is where I am less sure, and it is a real question rather than a rhetorical one. We chose to keep governance facts in a separate contract that correlates with your trace, on the argument that policy decisions and approvals have a different lifetime, a different audience and a different durability requirement than operational spans. The alternative is to push all of it upstream and argue for governance attributes inside the GenAI conventions themselves, which is slower and harder, but ends with one contract instead of two.
If you run agents in production, which of those would you rather adopt?
Top comments (0)