Linear logs are a convenient lie. An agent that fans out two tool calls, waits, then writes a file will still print those events in whatever order the scheduler flushed stdout. The debug story you read is a serialization of a graph. Treat that serialization as causality and you will “fix” the wrong hop.
The reusable loop is small. Record each hop as a span with a parent, assert a topology contract, and fail the run when required edges are missing. Output text is a claim. The graph is the evidence.
Cheap generation makes this sharper, not softer. As models draft more code per hour, the failure mode shifts from “did it write anything” to “did the run actually take the hops we require.” That is the same pressure showing up in 2026 discussions of AI-era technical debt: verification has to stay cheaper than regeneration, or teams ship transcripts instead of proofs.
Why print order collapses under fan-out
A single-threaded script can pretend that time is a list. Agents do not stay in that world. One planning span can start two retrievals, a linter, and a patch apply. Those children overlap. If you only keep print(f"called {tool}"), the merge of those lines is an accident of buffering, not a happens-before relation.
Think of a git history flattened into git log --oneline with no parent hashes. You still see commits. You cannot tell which commit branched from which. Agent stdout is that flattened log. The missing hashes are span_id and parent_id.
A second failure mode sits next to the first. Teams grep for the last “success” line and ship. The last line can come from a child that finished first while a sibling is still writing a partial file. Sequence in a file is not completion of the tree.
A minimal span record
Keep one JSON object per line. Anything richer can wait until the contract is green. The fields below are enough to reconstruct a tree and to reject a broken one.
{"ts":"2026-09-03T09:14:02.101Z","run_id":"run_7c2","span_id":"s0","parent_id":null,"kind":"run","name":"agent","status":"ok"}
{"ts":"2026-09-03T09:14:02.140Z","run_id":"run_7c2","span_id":"s1","parent_id":"s0","kind":"llm","name":"plan","status":"ok"}
{"ts":"2026-09-03T09:14:02.401Z","run_id":"run_7c2","span_id":"s2","parent_id":"s1","kind":"tool","name":"search","status":"ok"}
{"ts":"2026-09-03T09:14:02.410Z","run_id":"run_7c2","span_id":"s3","parent_id":"s1","kind":"tool","name":"read_file","status":"ok"}
{"ts":"2026-09-03T09:14:03.002Z","run_id":"run_7c2","span_id":"s4","parent_id":"s0","kind":"tool","name":"write_file","status":"ok"}
Timestamps stay for humans. The contract should not use wall-clock order as the source of truth. Clocks skew, free endpoints retry, and a child can be flushed before its parent if you log from multiple workers. Parent pointers are the invariant.
Do not put prompts, secrets, or file bodies in this file. Cardinality already grows with every hop. A constrained disk and a constrained model budget both punish fat spans. Store a hash or a byte length if you need a fingerprint later.
The contract is topology, not prose
A contract is a set of edges you require, plus a few numeric caps. It is not a summary of what the model said. Summaries are generated text. Generated text is another claim.
Here is a contract a coding agent can fail in CI without anyone reading a transcript.
run:
required_root: agent
max_depth: 6
max_spans: 80
children:
agent:
require_any: [plan]
plan:
require_all: [search, read_file]
require_any: [write_file, patch]
integrity:
unique_span_ids: true
parent_must_exist: true
no_cycles: true
single_root: true
require_all is the interesting bit. If plan finished and never produced a search child, the run can still print “done”. The checker should not care about that sentence. Missing children are failing tests. That is closer to a type checker than to log review.
Caps matter on a constrained server. An agent that retries a tool in a tight loop will explode span count. max_spans turns a runaway loop into a red build instead of a full disk.
A checker you can run on JSONL
The following script is a proposal you can save as span_contract.py. It reads traces from stdin, loads a YAML contract, and exits non-zero on the first broken run. Treat it as unexecuted until you point it at traces from your own runner.
#!/usr/bin/env python3
"""Fail a CI job when agent traces violate parent-span topology."""
from __future__ import annotations
import json
import sys
from collections import defaultdict
from typing import Any
try:
import yaml
except ImportError:
yaml = None
def load_events(raw: str) -> dict[str, list[dict[str, Any]]]:
runs: dict[str, list[dict[str, Any]]] = defaultdict(list)
for line_no, line in enumerate(raw.splitlines(), 1):
line = line.strip()
if not line:
continue
try:
ev = json.loads(line)
except json.JSONDecodeError as exc:
raise SystemExit(f"line {line_no}: invalid json: {exc}")
for key in ("run_id", "span_id", "kind", "name"):
if key not in ev:
raise SystemExit(f"line {line_no}: missing {key}")
runs[str(ev["run_id"])].append(ev)
return runs
def build_tree(events: list[dict[str, Any]]):
by_id = {}
children: dict[str, list[str]] = defaultdict(list)
for ev in events:
sid = ev["span_id"]
if sid in by_id:
raise ValueError(f"duplicate span_id {sid}")
by_id[sid] = ev
roots = []
for ev in events:
pid = ev.get("parent_id")
sid = ev["span_id"]
if pid in (None, "", "null"):
roots.append(sid)
continue
if pid not in by_id:
raise ValueError(f"span {sid} parent {pid} does not exist")
children[pid].append(sid)
if len(roots) != 1:
raise ValueError(f"expected one root, found {roots}")
return by_id, children
def has_cycle(root: str, children: dict[str, list[str]]) -> bool:
seen, stack = set(), set()
def dfs(node: str) -> bool:
if node in stack:
return True
if node in seen:
return False
stack.add(node)
for kid in children.get(node, []):
if dfs(kid):
return True
stack.remove(node)
seen.add(node)
return False
return dfs(root)
def depth_of(root: str, children: dict[str, list[str]]) -> int:
def walk(node: str) -> int:
kids = children.get(node, [])
if not kids:
return 1
return 1 + max(walk(k) for k in kids)
return walk(root)
def names(by_id, ids):
return [by_id[i]["name"] for i in ids]
def check_run(events: list[dict[str, Any]], contract: dict[str, Any]) -> list[str]:
errors: list[str] = []
try:
by_id, children = build_tree(events)
except ValueError as exc:
return [str(exc)]
root = next(
ev["span_id"]
for ev in events
if ev.get("parent_id") in (None, "", "null")
)
if has_cycle(root, children):
errors.append("cycle in parent_id graph")
run_cfg = contract.get("run", {})
root_ev = by_id[root]
if root_ev.get("name") != run_cfg.get("required_root", root_ev.get("name")):
errors.append(
f"root name {root_ev.get('name')} != {run_cfg.get('required_root')}"
)
d = depth_of(root, children)
if d > int(run_cfg.get("max_depth", 32)):
errors.append(f"depth {d} exceeds max_depth")
if len(events) > int(run_cfg.get("max_spans", 10_000)):
errors.append(f"span count {len(events)} exceeds max_spans")
rules = contract.get("children", {})
for sid, ev in by_id.items():
rule = rules.get(ev["name"])
if not rule:
continue
kid_names = names(by_id, children.get(sid, []))
for need in rule.get("require_all", []):
if need not in kid_names:
errors.append(f"{ev['name']} ({sid}) missing child {need}")
any_need = rule.get("require_any", [])
if any_need and not any(n in kid_names for n in any_need):
errors.append(f"{ev['name']} ({sid}) missing any of {any_need}")
return errors
def main() -> None:
if yaml is None:
raise SystemExit("install pyyaml")
if len(sys.argv) != 2:
raise SystemExit("usage: span_contract.py contract.yaml < traces.jsonl")
contract = yaml.safe_load(open(sys.argv[1], encoding="utf-8"))
runs = load_events(sys.stdin.read())
failed = 0
for run_id, events in runs.items():
errs = check_run(events, contract)
if errs:
failed += 1
print(f"FAIL {run_id}")
for e in errs:
print(f" - {e}")
else:
print(f"PASS {run_id} spans={len(events)}")
raise SystemExit(1 if failed else 0)
if __name__ == "__main__":
main()
A failing fixture should live in the repo, not in a screenshot. The next JSONL is a plan span that never searched. The checker must exit 1.
{"ts":"2026-09-03T09:20:00.000Z","run_id":"run_bad","span_id":"s0","parent_id":null,"kind":"run","name":"agent","status":"ok"}
{"ts":"2026-09-03T09:20:00.100Z","run_id":"run_bad","span_id":"s1","parent_id":"s0","kind":"llm","name":"plan","status":"ok"}
{"ts":"2026-09-03T09:20:00.400Z","run_id":"run_bad","span_id":"s2","parent_id":"s0","kind":"tool","name":"write_file","status":"ok"}
Wire it like any other test binary. The exit code is the API.
python3 span_contract.py contract.yaml < traces.jsonl
echo $?
If you already emit OpenTelemetry, map span_id and parent_span_id into this shape and keep the contract. The value is the assertion, not the vendor of the exporter.
Recording spans from a stub agent
Wrap tool dispatch. Do not wrap print. A short wrapper is enough to prove the contract against a local fake, then against a remote model.
import json, uuid
from contextlib import contextmanager
from datetime import datetime, timezone
def now():
return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.%f")[:-3] + "Z"
class Tracer:
def __init__(self, path, run_id=None):
self.path = path
self.run_id = run_id or uuid.uuid4().hex[:8]
self._stack = []
def emit(self, **fields):
rec = {"ts": now(), "run_id": self.run_id, **fields}
with open(self.path, "a", encoding="utf-8") as fh:
fh.write(json.dumps(rec, ensure_ascii=False) + "\n")
@contextmanager
def span(self, kind, name):
span_id = uuid.uuid4().hex[:6]
parent = self._stack[-1] if self._stack else None
self._stack.append(span_id)
status = "ok"
try:
yield span_id
except Exception:
status = "error"
raise
finally:
self.emit(
span_id=span_id,
parent_id=parent,
kind=kind,
name=name,
status=status,
)
self._stack.pop()
Call tracer.span("tool", "search") around each hop. Nested with blocks produce nested parents even when two tools finish out of wall-clock order, because the stack records start, not flush. That is the whole point.
A local fixture can stand in for a model. Feed a canned plan, run two fake tools, write a file, then run the checker. Only after that fixture is green should a remote call append to the same JSONL path. Cheap verification before quota-gated generation is the same rule CI uses for compilers.
Where a free model and a free server actually help
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode offers free model access and a free server option. Those two facts change the cost of this loop, not the shape of it. The checker is ordinary Python. It does not need a GPU. Putting span_contract.py on a free server next to a drop directory of JSONL files gives you a gate that does not share a process with the agent. Isolation matters: if the agent process crashes, the trace file still exists, and the checker can still fail the job.
Free model access is useful for generating the traces you assert against, as long as you keep the contract stable. Swap the model, keep require_all. If a cheaper model stops emitting a search child, the checker flags a topology change. That is a regression signal you can read without opening a transcript. It is also a completeness signal. A model that skips required tools is not faster. It is incomplete.
Do not treat a free server as an archive. Rotate JSONL by run_id, delete passing fixtures, and keep the failing ones. Trace volume follows retry count, and retry count follows prompt drift.
If you already have traces from a local stub, point the same contract file at JSONL produced through MonkeyCode’s free model access on the free server. The host can change. The required edges should not.
Limitations
This checker does not prove the patch was correct. It proves the run grew the children you said were mandatory. A search span can exist and still return junk. Pair the topology check with a separate content test if the tool output is what you ship.
Parent pointers can be forged. If the agent writes its own trace, a buggy wrapper can attach write_file to the root to satisfy require_any. Prefer emitting spans from the tool runner you control, not from model-authored JSON.
JSONL is not a lock-free log. Two processes appending without a lock will tear lines. Use one writer, or write per-span files and concat in the checker.
Wall-clock ts is for display. Do not add “child started after parent” rules unless you control a monotonic clock. Retries on remote endpoints will violate naive time order.
Skip this approach when the agent is a single synchronous tool with no children. A span contract is ceremony in that case. Skip it if you cannot wrap tool dispatch, because you cannot trust parent ids. Skip it for interactive chat with no required tools. There is no topology to assert.
The loop to reuse is record, assert topology, fail on missing edges, then inspect the transcript. Print order is a rendering. Causality is a graph.
Top comments (0)