Log Where Each Tool Argument Came From

The calendar tool returned HTTP 200 without errors. The invite still landed at 09:00 UTC. The user sat in Tokyo that same morning.

Nobody typed a timezone into the prompt. The model sent timezone UTC on its own. The JSON trace file only named the tool. It did not name the source of that field.

This failure class hides inside successful traces. The payload is valid JSON against the schema. The user never asserted the risky field.

Agent write-ups this week keep circling that gap. The model fills a missing field. The tool still reports success.

Why name-only tool logs fail

Typical agent logs still keep three thin columns. They usually keep only the raw tool name. They also keep the raw argument object. They finally keep the raw tool result.

Those three columns cannot assign real blame. You cannot see who invented each key. You cannot see which value was a guess.

Plain user text is only one source. Retrieved context is a second source. JSON schema defaults are a third source. Model inference is a fourth source.

Only the fourth source should page you. The first three are explainable after the fact. The fourth source is an assumption span.

Successful calls are the dangerous ones here. Hard failures already wake the on-call engineer. Silent field defaults never wake the on-call.

Four provenance tags

Tag every argument with exactly one source.

1. user_explicit — the value appears in user text
2. context_retrieved — a prior tool result supplied it
3. schema_default — the JSON schema filled it
4. model_inferred — the model guessed a missing field

You should keep this provenance tag set closed. Extra ad-hoc tags quickly create unreadable traces. Four tags are enough for a first loop.

Walkthrough fixture

This is a lab fixture, not a field study. No production rates are claimed in this writeup.

User text: Book a follow-up with Priya tomorrow morning.

The calendar tool schema required a timezone field. The user message contained no timezone token. The model still sent the string UTC.

A normal trace records a clean success. An assumption span records a guessed field.

Artifact: argument provenance wrapper

The wrapper sits in front of tool dispatch. It does not change tool side effects. It writes one JSON line per argument.

The following code is a labeled example. Adapt the symbol names to your runtime.

# arg_provenance.py
from __future__ import annotations

import json
import re
import hashlib
from dataclasses import dataclass, asdict
from datetime import datetime, timezone
from typing import Any, Callable, Literal

Source = Literal[
    "user_explicit",
    "context_retrieved",
    "schema_default",
    "model_inferred",
]

DENY_INFERRED = {
    "timezone",
    "account_id",
    "price",
    "currency",
    "email",
    "to",
    "recipient",
}

@dataclass
class ArgSpan:
    ts: str
    run_id: str
    parent_span: str
    tool: str
    key: str
    value_hash: str
    source: Source
    excerpt: str

def _norm(text: str) -> str:
    return re.sub(r"\s+", " ", text).strip().lower()

def _contains(haystack: str, needle: str) -> bool:
    if needle is None:
        return False
    h = _norm(str(haystack))
    n = _norm(str(needle))
    if len(n) < 2:
        return False
    return n in h

def flatten(prefix: str, obj: Any, out: dict[str, Any]) -> None:
    if isinstance(obj, dict):
        for k, v in obj.items():
            key = f"{prefix}.{k}" if prefix else k
            flatten(key, v, out)
        return
    if isinstance(obj, list):
        for i, v in enumerate(obj):
            flatten(f"{prefix}[{i}]", v, out)
        return
    out[prefix] = obj

def classify_arg(
    key: str,
    value: Any,
    user_text: str,
    context_blobs: list[str],
    schema_defaults: dict[str, Any],
) -> Source:
    leaf = key.split(".")[-1].split("[")[0]
    if leaf in schema_defaults and schema_defaults[leaf] == value:
        return "schema_default"
    if key in schema_defaults and schema_defaults[key] == value:
        return "schema_default"
    raw = "" if value is None else str(value)
    if _contains(user_text, raw):
        return "user_explicit"
    for blob in context_blobs:
        if _contains(blob, raw):
            return "context_retrieved"
    return "model_inferred"

def hash_value(value: Any) -> str:
    payload = json.dumps(value, sort_keys=True, default=str)
    return hashlib.sha256(payload.encode()).hexdigest()[:12]

def emit_spans(
    *,
    run_id: str,
    parent_span: str,
    tool: str,
    args: dict[str, Any],
    user_text: str,
    context_blobs: list[str],
    schema_defaults: dict[str, Any],
    sink: Callable[[str], None],
) -> list[ArgSpan]:
    flat: dict[str, Any] = {}
    flatten("", args, flat)
    now = datetime.now(timezone.utc).isoformat()
    spans: list[ArgSpan] = []
    for key, value in flat.items():
        source = classify_arg(
            key, value, user_text, context_blobs, schema_defaults
        )
        span = ArgSpan(
            ts=now,
            run_id=run_id,
            parent_span=parent_span,
            tool=tool,
            key=key,
            value_hash=hash_value(value),
            source=source,
            excerpt=str(value)[:80],
        )
        sink(json.dumps(asdict(span), ensure_ascii=False))
        spans.append(span)
    return spans

def gate_inferred(spans: list[ArgSpan]) -> list[str]:
    hits: list[str] = []
    for span in spans:
        if span.source != "model_inferred":
            continue
        leaf = span.key.split(".")[-1].split("[")[0]
        if leaf not in DENY_INFERRED:
            continue
        hits.append(f"{span.tool}.{span.key}")
    return hits

The sink can be stdout or a JSONL file. Do not persist raw emails in shared logs. Keep value_hash as the durable column. Keep excerpt on local disks only.

Driver for the Tokyo invite

# demo_assumption.py
from arg_provenance import emit_spans, gate_inferred

lines: list[str] = []
spans = emit_spans(
    run_id="run-2026-09-03-01",
    parent_span="turn-4",
    tool="calendar.create",
    args={
        "title": "Follow-up with Priya",
        "when": "tomorrow morning",
        "attendee": {"name": "Priya"},
        "timezone": "UTC",
    },
    user_text="Book a follow-up with Priya tomorrow morning.",
    context_blobs=[
        "calendar connected",
        "Priya is on the support roster",
    ],
    schema_defaults={},
    sink=lines.append,
)

print("\n".join(lines))
print("blocked:", gate_inferred(spans))

Run the driver on any small host.

python demo_assumption.py > /tmp/arg-spans.jsonl
grep model_inferred /tmp/arg-spans.jsonl

Filter the same file without grep if needed.

python -c "import json; rows=[json.loads(l) for l in open('/tmp/arg-spans.jsonl')]; print([r for r in rows if r['source']=='model_inferred'])"

Expected local result is one blocked path. That path is calendar.create.timezone. title should classify as user_explicit. attendee.name should classify as user_explicit. when should classify as user_explicit.

Re-run after you change the schema. The fixture stays the regression test.

Decision table

Observed match	Provenance tag	Action
User text contains the value	user_explicit	Allow the call
Prior tool blob contains the value	context_retrieved	Allow, keep parent_span
Schema default equals the value	schema_default	Allow only if default is safe
No match, leaf in denylist	model_inferred	Block or ask one question
No match, leaf not in denylist	model_inferred	Log only
Tool raised	n/a	Use the existing error path

Keep the denylist short on purpose. Timezone, money, identity, and destination belong there. Cosmetic keys do not belong there.

Reusable debug loop

1. Store the user turn beside the run id.
2. Store retrieved blobs under the same id.
3. Intercept tool args before execution.
4. Flatten nested objects into leaf keys.
5. Emit one assumption span per leaf.
6. Run gate_inferred before the side effect.
7. On a hit, refuse and ask one clarifying question.
8. Write JSONL next to the run id.

The loop is intentionally boring. Boring loops survive the next agent rewrite.

Parent spans matter for later review. turn-4 is the cause. calendar.create is the effect. You can join them with run_id.

Join is a one-liner after the run.

python -c "import json; from collections import defaultdict; g=defaultdict(list);
[g[json.loads(l)['parent_span']].append(json.loads(l)['key']) for l in open('/tmp/arg-spans.jsonl')];
print(dict(g))"

That map answers a narrow question. Which fields did this turn invent. It does not replay the whole agent.

Test plan

Label these as unexecuted checks. Run them against the fixture above.

1. User mentions UTC. Expect user_explicit for timezone.
2. User omits timezone. Expect model_inferred and a gate hit.
3. Schema default is UTC. Expect schema_default, not a guess.
4. Retrieved memory contains Asia/Tokyo. Expect context_retrieved if copied.
5. Nested attendee.email is guessed. Expect a gate hit.
6. Excerpt must not appear in the hashed column.
7. Two runs with the same args share the same value_hash.

A failed test is a classifier bug. Do not ship a gate that fails test 1. That gate will nag users who were explicit.

Failure analysis of the classifier

Substring matching is a blunt instrument. A docs blob that mentions UTC will re-tag a guess. That hides an assumption as context.

Short values collide often. to=US can match the word status. The minimum length check is a weak filter.

Date language is not normalized. tomorrow morning is not a timestamp. The wrapper will not catch time math errors.

Lists of recipients need extra policy. One guessed address in a list is enough. The sample flattens indexes but does not score lists.

Hashes are not encryption. They only reduce casual log leakage. Assume a determined reader can brute short values.

What name-only traces miss

• Timezone guesses on calendar writes
• Currency guesses on invoice writes
• Recipient guesses on mail writes
• Account ids copied from the wrong blob
• Schema defaults that the user never accepted

Each item looks healthy in a tool-name log. Each item is still a user-facing incident.

Limitations

This is not a tracing vendor product. It is a JSONL convention you can grep.

It is not a substitute for authz. A guessed account_id that passes the gate is still theft if authz is missing.

It does not reconstruct prompts. It only attributes argument values.

It will false-positive on paraphrases. The user said Pacific time. The model sent America/Los_Angeles. That is inferred under this classifier.

Flattening does not preserve sibling intent. Two nested objects can share a leaf name. Disambiguate with the full dotted path.

Who should skip this

Skip it if the agent has no write tools. Read-only summarizers do not need argument gates.

Skip it if a human confirms every write. The confirmation dialog is already the gate.

Skip it if you cannot retain run-scoped logs. Provenance without retention cannot be audited.

Skip it if legal requires full payload archives. Twelve-character hashes will not meet that bar.

A small host is enough

The wrapper is process-local Python. A laptop can run the fixture in seconds. Overnight replay of many traces wants a small always-on host.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is an open-source coding agent project with free model access, stated by the operator as 10 million tokens, and a free server option. That pair is relevant only as a place to run the provenance loop and keep JSONL. No model names, hardware, or duration claims are made here. If a host already exists, keep the wrapper and ignore the product.

Gate one denylist field before you automate refusals.