Hash Three Boundaries Before You Blame the Model

A night operator opened two logs for the same tool call. The process log held valid JSON with two rows. The next model turn claimed the tool result was unreadable.

This scene is reconstructed for teaching, not a quoted outage. The figures below are fixtures, not measured production rates.

The handoff has three owners

A tool process writes the first byte string. An adapter then parses, wraps, or redacts that string. A prompt builder copies one slice into the next model message.

Any owner can change bytes without changing the tool name. A single stored hash cannot say which owner did it. You need one digest per boundary, plus the length at that boundary.

Fields worth storing

Keep these fields on the same tool-call row. Derive mutation_hop in code, not in an editor. A later reader should recompute that label from hashes.

• process_sha256 is the hash of raw process output.
• adapter_sha256 is the hash after parse, wrap, and redact.
• model_sha256 is the hash of the tool-result slice only.
• Store process_len, adapter_len, and model_len as byte lengths.
• mutation_hop is the label chosen from the table below.

Hash the tool-result slice, not the full prompt. A full-prompt hash changes when any sibling message changes. That noise hides a stable tool payload from review.

Sample row

This JSON row is an illustrative teaching fixture only. It is not a captured trace from a production agent. Replace the short hash placeholders with real SHA-256 values.

{
  "run_id": "run_7f3a",
  "tool_call_id": "call_91",
  "tool_name": "read_file",
  "process_sha256": "replace-with-real-sha256",
  "adapter_sha256": "replace-with-real-sha256",
  "model_sha256": "replace-with-other-sha256",
  "process_len": 36,
  "adapter_len": 20,
  "model_len": 20,
  "mutation_hop": "adapter_shrink"
}

Read the hop table first

Read this table before you open a provider status page. Compare the three digests first, then use length as a tie break. Route the review to the hop named in the matching row.

Comparison	Length signal	Hop to inspect
Process hash differs, adapter shorter	Shrink	Adapter truncate or parse drop
Process hash differs, adapter longer	Growth	Adapter wrap or injected default
Process equals adapter, model longer	Growth	Prompt builder added a wrapper
Process equals adapter, model shorter	Shrink	Prompt builder dropped fields
All three hashes equal, lengths equal	None	Not a payload mutation
Hashes equal, model length disagrees	Mismatch	You hashed the wrong slice

The last row tests your instrumentation, not the model. Matching hashes with unequal lengths should stay rare. If you see that row, the model hash covered a different byte range.

Hand-counted fixture lengths

These lengths are hand counts of the fixture literals. They are not byte counts from a live agent trace. Confirm every number with len() when you run the file.

The wrap process literal is 20 ASCII bytes long. The wrap model slice literal is 48 ASCII bytes long. The extra 28 bytes are the type field and body wrapper.

The shrink process literal is 36 ASCII bytes long. The shrink adapter literal is 20 ASCII bytes long. Those 16 missing bytes are the inserted note field.

Classifier you can copy

The script below is an unexecuted teaching proposal. It was not executed inside a hosted agent for this article. It uses only the Python standard library for hashing.

#!/usr/bin/env python3
"""Classify which hop mutated a tool result. Unexecuted teaching fixture."""

import hashlib
import json
from typing import Any


def sha256_bytes(data: bytes) -> str:
    return hashlib.sha256(data).hexdigest()


def canonicalize(obj: Any) -> bytes:
    text = json.dumps(
        obj,
        sort_keys=True,
        separators=(",", ":"),
        ensure_ascii=False,
    )
    return text.encode("utf-8")


def classify(process: bytes, adapter: bytes, model_part: bytes) -> dict:
    p = sha256_bytes(process)
    a = sha256_bytes(adapter)
    m = sha256_bytes(model_part)
    if p != a and len(adapter) < len(process):
        hop = "adapter_shrink"
    elif p != a and len(adapter) > len(process):
        hop = "adapter_grow"
    elif p == a and a != m and len(model_part) > len(adapter):
        hop = "prompt_builder_wrap"
    elif p == a and a != m and len(model_part) < len(adapter):
        hop = "prompt_builder_drop"
    elif p == a and a == m and len(process) == len(adapter) == len(model_part):
        hop = "no_payload_mutation"
    elif p == a and a == m:
        hop = "hash_slice_mismatch"
    else:
        hop = "needs_manual_review"
    return {
        "process_sha256": p,
        "adapter_sha256": a,
        "model_sha256": m,
        "process_len": len(process),
        "adapter_len": len(adapter),
        "model_len": len(model_part),
        "mutation_hop": hop,
    }


def fixture_wrap() -> dict:
    process = b'{"ok":true,"rows":2}'
    adapter = canonicalize({"ok": True, "rows": 2})
    model_part = b'{"type":"tool_result","body":{"ok":true,"rows":2}}'
    return classify(process, adapter, model_part)


def fixture_shrink() -> dict:
    process = b'{"ok":true,"rows":2,"note":"abcdef"}'
    adapter = b'{"ok":true,"rows":2}'
    model_part = adapter
    return classify(process, adapter, model_part)


if __name__ == "__main__":
    print(json.dumps({"wrap": fixture_wrap(), "shrink": fixture_shrink()}, indent=2))

Run the fixture on a laptop before you trust the labels. Check two expectations in the printed JSON object. The wrap fixture should report the prompt_builder_wrap label.

The shrink fixture should report the adapter_shrink label. If the wrap fixture reports adapter_grow, stop and compare inputs. You hashed canonical adapter bytes against raw process bytes.

Key order and spaces change the hash even when values match. Treat that mismatch as a fixture bug before you blame the adapter. Rerun both fixtures after any change to canonicalize.

python3 three_boundary.py

Keep a raw hash and a canonical hash

Store both a raw adapter hash and a canonical adapter hash. Use the raw hash to detect any byte change. Use the canonical hash to ignore object key order.

Do not replace the raw hash with the canonical hash. A later incident may depend on exact bytes. Exact bytes include spaces the model actually received.

Canonical form here is compact JSON with sorted keys. It is not a universal JSON standard for every payload. Dates, large integers, and duplicate keys need local rules.

Five-step debug loop

Use this loop on one tool_call_id from the failing turn. Do not start a retry before step four finishes. A second run can replace the only clean copy.

1. Copy the failing tool-call id from the model turn.
2. Save process stdout bytes under that same id.
3. Save adapter output bytes under that same id.
4. Save the tool-result slice from the outbound model request.
5. Run classify and append the row to the trace log.

A shell sketch for those files looks like the block below. The flags are a proposed interface for a later script. The teaching script still uses in-memory fixtures only.

python3 three_boundary.py \
  --process traces/call_91.process.bin \
  --adapter traces/call_91.adapter.bin \
  --model traces/call_91.model_part.bin \
  >> traces/mutation_hops.jsonl

After the row exists, route the next owner from the label. adapter_shrink goes to the adapter author for review. prompt_builder_wrap goes to the prompt author for review.

no_payload_mutation leaves transport and enters behavior review. Filter the log so that review starts on real mutations.

jq -c 'select(.mutation_hop != "no_payload_mutation")' traces/mutation_hops.jsonl

This filter keeps only rows where a hop changed bytes. Start the review with that shorter list, not the full file.

What equal hashes do not prove

Equal digests do not mean the tool chose the right file. They mean the hashed slices matched each other exactly. A wrong path can be preserved perfectly across every hop.

A model can still misread a faithful tool payload. That case is a behavior question, not a mutation question. Split those questions or the retry will touch the wrong layer.

Do not delete process stdout after a green digest row. The digest is an index, and the bytes are the evidence. Keep both artifacts until the incident review is closed.

Share digest rows, keep payloads local

Raw tool output often contains source code or secrets. A digest row usually does not contain those secrets. You can share the row more safely than the raw file.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode provides free model access and a free server option. Both points are operator-supplied for this draft only.

This piece states no quota, model name, or machine size. It also states no end date for either availability claim. Leave process files on the tool host that created them.

A teammate can group mutation_hop without opening file contents. Use free model access for one controlled replay check. Send the saved tool-result slice, not a fresh tool run.

Ask whether that exact slice still looks unreadable. If the replay fails, the saved slice reproduces the complaint. If the replay succeeds, the live request differed from the slice.

That check is a workflow suggestion, not a quality score. No accuracy rate is reported anywhere in this article. Policy still decides whether metadata may leave the network.

Store a redaction receipt

Upload nothing until local redaction finishes on the host. A hash of a secret can still be sensitive here. A small private corpus can make even a digest identifying.

Prefer hashing the payload only after replacement finishes. Store a receipt next to the digest row in the log.

• Record field_path for each replaced value in the payload.
• Record rule_id so later policy changes stay visible.
• Record original_len counted only on the tool host.
• Record the replacement token, never the original characters.

If rule_id changes between two runs, mark policy drift. Do not call that drift a model regression by default. The hop table cannot see a rule change without the receipt.

Who should skip this loop

Skip the loop when the tool returns no response body. An exit-code-only tool needs a different record shape. Three empty hashes will only waste the review hour.

Skip it when you cannot capture the outbound model request. Token totals on a dashboard cannot fill model_sha256. You would be guessing the slice instead of hashing it.

Skip a shared free server when metadata must stay inside. Digests plus tool names can still identify a private file. A small corpus makes that identification risk concrete.

Skip this loop as a model leaderboard or vendor score. The classifier locates a hop inside one recorded call. It does not rank coding models against each other.

Limits to keep on the page

The classifier ignores streaming chunk boundaries in this sketch. It also ignores compression and content-encoding differences. It assumes the process hash is taken on raw bytes.

canonicalize will change hashes on purpose in this sketch. If key order is meaningful, skip that function for comparison. Keep the canonical hash only as a second stored column.

Clock skew stays out of scope for this method. This method never reads timestamps from either host. A slow but correct hop will look healthy here.

No fixture in this article ran on a hosted agent. Treat the printed labels as expected results to confirm. If your runtime adds a byte-order mark, hashes will differ.

Streaming tools

Streaming tools need one extra rule before classification. Hash the concatenated process output after the stream closes. A mid-stream hash will not match the final adapter hash.

That false adapter_grow label sends you to the wrong owner. Wait for stream close, then hash, then classify. Do not classify a partial buffer during the call.

Close the record before any retry

Select one failed tool-call id from the latest trace. Write the three slices to disk under that id. Run the classifier and store mutation_hop before any retry.

If the team needs a shared digest log, a free server can hold the rows. A MonkeyCode free server is one place for those digest rows. Keep raw payloads on the tool host unless policy allows the copy.