Someone on your team pings you: "The agent approved a refund it shouldn't have. Why?"
You open the logs. There's the user message, there's the final reply, and there's a 200 status code. The reply is polite and well written. What you can't see is the thing you need: what the agent looked at, which tools it called and in what order, and where its reasoning went sideways.
That's the gap with agent logs. They were built for services that return an answer, and agents take actions along the way.
What a normal log is missing
A typical request log gives you input and output. An agent's behavior comes from everything in between:
The model and prompt version that were live at that moment. The documents or chunks that retrieval pulled in, since a wrong or poisoned passage can steer the whole run. Every tool call, with its arguments and result. And the order of those steps, because the same calls in a different order can mean a very different outcome.
Without those, debugging turns into guessing. You re-run the request, get a different answer because outputs vary, and can't tell whether you reproduced the problem or just got lucky.
What belongs in a trace
For each run, we'd capture these, roughly in this order:
- Run metadata: a run ID, timestamp, model identifier, prompt version and config version.
- Input: the user request and any prior turns that were in context.
- Retrieval: the query and the IDs of the chunks that came back.
- Model calls: what was sent and what came back.
- Tool calls: tool name, arguments, result and any error.
- Outcome: the final reply and whether any rules you care about were violated.
The version fields matter more than they look. When behavior changes without a code change, the first question is always "what's different from last week?" Having the model and prompt version on every trace lets you answer that by filtering instead of by memory.
A minimal sketch
Here's a small Python example of structured trace records. It's illustrative, not a library.
import json
import time
import uuid
from contextlib import contextmanager
class Trace:
def __init__(self, meta=None):
self.run_id = str(uuid.uuid4())
self.meta = meta or {}
self.steps = []
@contextmanager
def step(self, kind, **data):
record = {"kind": kind, "started": time.time(), **data}
try:
yield record
except Exception as e:
record["error"] = repr(e)
raise
finally:
record["duration_ms"] = int((time.time() - record["started"]) * 1000)
self.steps.append(record)
def dump(self):
return json.dumps(
{"run_id": self.run_id, "meta": self.meta, "steps": self.steps},
default=str,
)
And a usage example around a refund agent:
trace = Trace(meta={"model": "provider-model-v2", "prompt_version": "a1b2c3"})
with trace.step("retrieval", query=user_msg) as s:
docs = search_kb(user_msg)
s["chunk_ids"] = [d.id for d in docs]
with trace.step("model_call") as s:
plan = llm(prompt, user_msg, docs)
s["output"] = plan
with trace.step("tool_call", tool="issue_refund", args=plan.args) as s:
s["result"] = issue_refund(**plan.args)
store(trace.dump())
Now when someone asks "why did it approve that refund?", you open one record and read the story: which policy passage was retrieved, what the model decided, and exactly what arguments went to the refund tool.
Two practical notes. Redact personal data before storing, since traces contain whatever users typed. And store the full text only where you're allowed to. Elsewhere, store IDs and hashes and keep the content in a controlled place.
Add rules to the trace, not just the logs
Once you have structured traces, you can attach your rules to them. A rule like "no refund after 30 days" or "never share another customer's data" can be checked against the trace, and a violation gets flagged on that run. That turns a trace from a passive record into a live signal. It's also how problems get caught in production instead of by a customer.
Why this is also a compliance question
If you sell to businesses, buyers increasingly ask how you track and document agent behavior. The EU AI Act includes record-keeping and logging expectations for high-risk systems. The Nolte, Rateike and Finck analysis is a good starting point for how those requirements apply, and the OWASP Top 10 for LLM Applications is now a common reference in security reviews. Detailed traces with version information give your compliance owners real material to work from.
To be clear about scope, tracing supports compliance work but doesn't certify it. It can't tell you which laws apply to you or replace legal review.
Start with one agent
Pick the agent that takes real actions. Add a trace around its retrieval, model calls and tool calls. Put the model and prompt version on every record. Make it searchable by run ID. That alone makes the next "why did it do that?" take minutes.
If you'd rather not build and maintain this yourself, BotGauge records the full trace behind every run, including prompts, tool calls, retrieved context and the execution path, in a shared workspace. There's a free tier to try on your own agent.
What do you capture today when an agent does something unexpected? Do you keep the retrieved context and tool arguments, or just the final reply? Tell us in the comments.
Top comments (0)