A parent span marked OK does not close an agent debug loop. Head-based sampling keeps that parent and can drop the failed tool child, so a dashboard reports a healthy run while the working tree is unchanged. Close the loop only after an error-biased sampler, a content hash of the tool arguments, and a local diff record all agree on the same tool_call_id.
The failure looks like a hospital that files the discharge summary and shreds the abnormal lab. The visit has an end time. The evidence that the treatment missed is gone. Agent traces fail the same way when the sampler decides at the start of the trace, before any tool has exited.
What head sampling actually keeps
Head sampling chooses a keep-or-drop bit when the trace starts. That bit is cheap, and it is also blind. A later tool can return a non-zero exit, an empty patch, or no end event at all, and the child span is already doomed if the head decision was drop.
Tail sampling waits until the trace has a status. It costs more, because spans must sit in a buffer until the decision. It is also the common policy that can promise to keep every child whose status is ERROR, UNSET, or missing an end. The parent may still be sampled down. The failure should not be.
None of this is a claim about a vendor default. It is a property of the two policies. If you do not know which policy sits in front of a free remote runner, measure it. Do not infer it from a green parent.
A join the dashboard will not do for you
Picture one coding-agent turn. The model span lists two tool calls, edit_file and run_tests. The remote runner executes them. Your laptop holds the git diff. Three records exist, and none of them is the truth alone.
The model span can claim both calls were issued. The runner can record one exit and lose the other to a full buffer. The diff can show a one-line change that matches neither call if a human edited the tree between runs. A reusable loop treats the run as open until each issued tool_call_id has a surviving child and a diff hash, or an explicit error status that explains why no diff exists.
Disclosure: This article was prepared as part of MonkeyCode's product outreach. The operator describes MonkeyCode as an open-source project with free model access and a free server option. Those two availability claims are why this loop is practical to rehearse: a model route and a remote runner can share one trace id without first buying a trace-vendor seat. Current quotas, model names, retention, and hardware are not stated here, because they were not verified against a primary source for this draft. Read the project page before you depend on a limit. The checker below does not require MonkeyCode. Any runner that emits the fields will do.
Fields worth keeping when the payload is not
Free remote hops often cap attribute size. Stuffing a whole file into a span attribute is how the tail of a tool call disappears, after which the truncation looks like a model mistake. Store a bounded preview and a hash. Keep the raw payload, if you must keep it at all, in a local artifact directory keyed by trace id, and redact secrets before either copy is written.
The envelope is small on purpose. It is an unexecuted reference snippet, not a measured production client.
import hashlib
PREVIEW_BYTES = 240
def arg_envelope(payload: bytes) -> dict:
digest = hashlib.sha256(payload).hexdigest()
return {
"arg_sha256": digest,
"arg_bytes": len(payload),
"arg_preview": payload[:PREVIEW_BYTES].decode("utf-8", errors="replace"),
}
A child span then needs trace_id, span_id, parent_span_id, tool_call_id, status_code, arg_sha256, and, after the runner finishes, exit_code plus diff_sha256. The preview is a hint for humans. The hash is what the joiner trusts. If the preview is cut mid-character, the hash still identifies the original bytes, so a later fetch from the local artifact store can restore the full argument without trusting the truncated attribute.
Sampler policy as code, not as a slogan
The following module is a proposal. It has not been executed against a live server for this article, and it is not a description of any product sampler. It shows the decision you want in front of a runner whose policy you have not measured.
from dataclasses import dataclass
@dataclass(frozen=True)
class Span:
span_id: str
parent_span_id: str | None
tool_call_id: str | None
status_code: str # OK, ERROR, UNSET
ended: bool
def keep_span(span: Span, head_keep: bool) -> bool:
if span.tool_call_id and (span.status_code != "OK" or not span.ended):
return True
return head_keep
OK tool children may still be dropped when head_keep is false. That is acceptable only if a separate diff ledger already recorded diff_sha256 for that tool_call_id. If the ledger is empty and the child is gone, you no longer have a debug loop. You have a story.
Think of the head bit as a coin flipped in the lobby. The coin does not know which lab result will come back abnormal. Error-biased retention is the clerk who refuses to shred a folder once a red stamp appears, even if the lobby coin said drop. The analogy stops at the policy. You still have to implement the stamp, which here is status_code plus ended.
The joiner
Export three JSONL files. issued.jsonl comes from the model span's tool-call list. children.jsonl comes from whatever the remote runner actually retained. diffs.jsonl is local, written by your wrapper after git diff. The joiner below is unexecuted reference code. Adapt the field names if your exporter uses a different key, but do not join on wall-clock timestamps.
import json, sys
def load(path):
with open(path, encoding="utf-8") as fh:
return [json.loads(line) for line in fh if line.strip()]
def open_calls(issued, children, diffs):
by_call = {row["tool_call_id"]: row for row in children}
diff_ids = {row["tool_call_id"] for row in diffs if row.get("diff_sha256")}
still_open = []
for row in issued:
call_id = row["tool_call_id"]
child = by_call.get(call_id)
if child is None:
still_open.append((call_id, "dropped_child"))
elif not child.get("ended") or child.get("status_code") == "UNSET":
still_open.append((call_id, "unclosed_child"))
elif child.get("status_code") == "ERROR":
still_open.append((call_id, "tool_error"))
elif call_id not in diff_ids:
still_open.append((call_id, "missing_diff"))
return still_open
if __name__ == "__main__":
issued, children, diffs = map(load, sys.argv[1:4])
pending = open_calls(issued, children, diffs)
for call_id, reason in pending:
print(f"{reason}\t{call_id}")
sys.exit(1 if pending else 0)
A wrapper can build the diff ledger without a trace vendor. The hash is of the patch text, not of the whole repository, so two runs that touch the same line in different ways do not collide. Generate the trace id before the first remote hop, and pass that same id into the model route, the runner, and this local file.
TRACE=4f0c0c2a9b1e4d77a0c1
export TRACE TOOL_CALL_ID=call_edit_01
git diff --binary > "/tmp/${TRACE}.patch"
python - <<'PY'
import hashlib, json, os
trace = os.environ["TRACE"]
blob = open(f"/tmp/{trace}.patch", "rb").read()
row = {
"trace_id": trace,
"tool_call_id": os.environ["TOOL_CALL_ID"],
"diff_sha256": hashlib.sha256(blob).hexdigest(),
"diff_bytes": len(blob),
}
open("diffs.jsonl", "a", encoding="utf-8").write(json.dumps(row) + "\n")
PY
python join_open_calls.py issued.jsonl children.jsonl diffs.jsonl
Exit status is the contract. No lines printed means every issued call joined a closed child and a diff hash. Any printed line means the run stays open, even if the parent status is OK. A silent report is the only pass condition. One green parent is not.
Count the reasons across repeated runs before you tune a prompt. If dropped_child dominates, the sampler or the exporter is the defect. If missing_diff dominates, the runner is reporting success without a tree change. If tool_error dominates, the model route is not the first place to look. Those ratios are the data. A single green trace is an anecdote.
You can rehearse that count on a free model route and a free server, then move the same three files to any other runner. The route supplies generations. The server supplies tool exits. The laptop supplies the diff hash. Remove any one of the three and the parent status becomes a rumor again.
What this will not prove
The join proves recording integrity, not that the patch is correct. A wrong but present diff still hashes. Semantic review stays outside this loop. Treat a matching hash as a gate that lets a human or a later test look at the patch, not as a verdict on the patch.
Tail retention of every error can fill a small free disk when a retry storm emits thousands of failed tool children. Cap the local artifact store by trace id and by age, and drop previews before you drop hashes. A hash without a preview is still joinable. A preview without a hash is not.
Do not put secrets, tokens, or customer payloads into span attributes or into the preview field. The envelope function above does not redact. Run a redaction step before arg_envelope, or do not use this pattern on those traces. A free server is still a remote hop. What you attach to a span can leave the laptop.
Skip the approach if you need legal hold of full prompts, multi-tenant access control, or a guaranteed retention window. A free server option is a place to rehearse the loop, not a promise about durability. Also skip it if your runner cannot emit tool_call_id on both the model side and the tool side. Without that shared key, you are back to joining on timestamps, and timestamps across machines do not order causality.
If a free model route and a free server are already available to you, point one failing run at this joiner before you trust the parent status again. Confirm the current access terms on the project page first. The parent can outlive the child. The child is the run.
Top comments (0)