Visualization tends to get treated as the afterthought of the observability stack. I'd half agree with that, except that the visualization is also what people seemed to like most about my last post. So here's a condensed version of it, focused on the pictures only.
The question I actually want answered mid-run is a boring one: what is this thing doing right now, and can I trust the bit it just finished? There are a few different layers that claim to answer it and they don't answer the same question. Semantic observability gives you the pipeline source, the inputs, the outputs. LLM-call observability gives you the prompts, the tool calls, the spans, the token counts - very useful for the conversation the agent is having with itself, and blind to what the code it wrote then went and did. Code execution observability is the third thing: mapping the code as it runs and tracing artifacts and their lineage deterministically, without manual instrumentation (a bit like extreme auto-logging). What comes out is datasets, dataframes, models, agent states and intermediate outputs, each traced back to the function that produced or transformed it, so what the executed code actually did, which intermediate step fed which result, and potentially (paired with a few other elements) where an error entered the workflow.
So this is the visualization of this code execution graph: the agent moving back and forth through the graph, resolving nodes. At the start, the job has issues throughout every section:
Then, as codex works through the graph nested function by nested function, some of them start clearing:
You can open a node if you want more information:
The visualization can be vastly improved, but hopefully it paints a picture. It's a debugging aid, a trace you can point at afterwards, and for any regulated sector, it is pretty useful as a deterministic account of what was executed by your agent.
Top comments (0)