Evidence Grades for Agent Claims: Terms, Branches, and Six Leaves

A composite night review, not a logged incident from this account, starts with a ticket already marked ready to close. Three agent notes sit on it. One says the suite passed. One blames a flake on the runner. One calls a dependency bump safe. The sentences are complete. No exit code, command, or job id is attached.

That gap is the failure this tree is built to catch. Fluent claims close work that never produced an artifact. A reviewer who rereads the prose is still guessing. A grade is a smaller, checkable decision.

What the gate is allowed to decide

The procedure answers one question. May this claim mutate ticket state, or may it only remain a comment?

It does not rank models, estimate cost, or certify a suite. The checker later in this article is an unexecuted proposal. Nothing here is a measured production result, and no latency or accuracy figure is implied.

Glossary

Stable words matter. A loose "pass" in chat becomes a closed ticket when terms drift.

Claim. A statement an agent wants applied to a job. "Tests passed", "the flake is environmental", and "the bump is safe" are claims. Tone is not part of the definition.

Evidence packet. A host-side record with a job id, the command that ran, an exit code, and a bounded log excerpt. A paragraph of prose is not a packet, even if it sounds specific.

Grade. One of Missing, Asserted, Derived, or Observed. The grade describes the packet. It does not describe how confident the model sounded.

Leaf. A terminal branch with one allowed action. Leaves do not negotiate, and urgency in the ticket title is not a fifth grade.

Mutation. A write that changes status, merge state, or assignees. Posting a comment is not a mutation in this tree.

Host. The place that stores the packet and runs the grader. A model response is not a host, and a chat scroll is not storage.

Model call. An optional labeling step over a packet the host already has. A model call may restate a grade. It may not create missing fields.

Redaction. Removal of secrets, tokens, and customer rows before any packet leaves the host. Redaction is a precondition, not a leaf.

MonkeyCode enters only after those terms are fixed. The operator describes it as an open-source project with free model access and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Those two availability claims are operator-supplied for this draft. This article does not name models, quotas, hardware, retention, or a time window. Confirm the current project terms before you treat either option as durable. A number read on an older page is not evidence you can plan a job around.

Free model access fits one step: suggesting a grade label for a redacted packet. The free server option fits one other step: running the same grader off a laptop, if you accept that server as a host. Neither step may promote a Missing packet to Observed.

Numbered walk

1. Store the claim and look for a packet on the host. If the packet is absent, assign Missing and stop. Do not call a model to invent an exit code.

2. Compare job ids. A packet from job A cannot support a claim about job B. A mismatch stays Missing for this claim, even if the log text looks related.

3. Classify the source. Exit code plus command plus log excerpt is Observed. A summary that points at a log you did not keep is Derived. A bare sentence is Asserted.

4. Read the requested action. Comment is not a mutation. Closing, merging, and reassigning are mutations.

5. Land on one leaf. Write the grade beside the claim. Apply that leaf's action and nothing stronger.

Branches

Grade	Action asked	Leaf	Allowed result
Missing, or job id mismatch	any	L0	Reject. No model call.
Asserted	comment	L1	Comment only, tagged ungraded.
Asserted	mutation	L2	Reject the mutation.
Derived	any	L3	Hold for a human with the raw artifact.
Observed, ids match	comment	L4	Comment allowed, grade recorded.
Observed, ids match	mutation	L5	No auto-close. Host re-check, then human accept.

Six leaves, four grades. L1 and L2 split Asserted by action. L4 and L5 split Observed the same way. The grade never rises because the request is urgent, cheap, or already written in confident prose.

L0, missing packet

Claim: "tests passed", job id job-18, action close. Packet: none.

The branch ends at Missing. The ticket stays open. A free model must not be asked what the exit code would have been. Guessed digits are still a missing packet.

A second case lands here too. The host has a packet, but its job_id is job-17. Related logs are not interchangeable. Treat the mismatch as Missing for job-18.

L1, asserted comment

Claim text: "Looks green from here." Fields present: job id only. Action: comment.

Grade is Asserted. The leaf allows a comment that says the note is ungraded. It does not allow a bot to move the column. The comment should quote the missing fields, not smooth them over.

L2, asserted mutation

Same Asserted claim, action close. The leaf rejects. A deadline in the title does not create a command, an exit code, or a log excerpt.

If a reviewer still wants the column to move, that is a new human action with its own reason. It is not a successful grade of the agent claim.

L3, derived summary

Packet: {"job_id": "job-18", "summary": "model says the log showed exit 0"}. The raw log was never stored. Action: close.

Grade is Derived. Hold the ticket. A human needs the artifact, not a second paraphrase. Copying the summary to a free server does not make it Observed. Another model restating "exit 0" is still Derived.

L4, observed comment

Packet includes job_id job-18, command python -m unittest tests.test_gate, exit_code 0, and a 20-line excerpt. Action: comment.

Grade is Observed. The comment may quote the exit code and the command. That is the full permission of this leaf. It is not a silent close, and it is not a claim that the suite covered every risk in the diff.

L5, observed mutation

Same Observed packet. Action: close. The leaf refuses automatic close. The host runs the recorded command again, or a human accepts the stored packet explicitly. A model may explain the excerpt. It may not be the acceptor.

Non-zero exits stay on this side of the tree if the fields are complete. They are Observed evidence of failure. They are not a prompt to shop for a kinder summary.

Proposal code

The module below is an unexecuted proposal. It encodes the leaves. It does not call a network, and it does not depend on a vendor SDK.

from enum import Enum

class Grade(Enum):
    MISSING = "missing"
    ASSERTED = "asserted"
    DERIVED = "derived"
    OBSERVED = "observed"

def grade_packet(packet, claim_job_id):
    if not packet or packet.get("job_id") != claim_job_id:
        return Grade.MISSING
    has_exit = "exit_code" in packet
    has_log = bool(packet.get("log_excerpt"))
    has_cmd = bool(packet.get("command"))
    if has_exit and has_log and has_cmd:
        return Grade.OBSERVED
    if packet.get("summary") and not (has_exit and has_log and has_cmd):
        return Grade.DERIVED
    return Grade.ASSERTED

def leaf_action(grade, action):
    if grade is Grade.MISSING:
        return "L0", "reject"
    if grade is Grade.ASSERTED and action == "comment":
        return "L1", "comment_ungraded"
    if grade is Grade.ASSERTED:
        return "L2", "reject_mutation"
    if grade is Grade.DERIVED:
        return "L3", "hold"
    if action == "comment":
        return "L4", "comment_observed"
    return "L5", "human_accept_after_host_recheck"

Expected mappings for a local check, also unexecuted in this draft:

Packet sketch	Action	Expected
None	close	L0, reject
{"job_id":"job-18"}	comment	L1, comment_ungraded
{"job_id":"job-18"}	close	L2, reject_mutation
{"job_id":"job-17","command":"pytest","exit_code":0,"log_excerpt":"ok"}	close	L0, reject
{"job_id":"job-18","summary":"exit 0"}	close	L3, hold
full observed fields for job-18	comment	L4, comment_observed
full observed fields for job-18	close	L5, human_accept_after_host_recheck

A host can build a packet without a model. This shell sketch is an example workflow, not a timed run. The command is fixed on purpose. Do not interpolate untrusted text into a shell.

job_id="job-18"
python -m unittest tests.test_gate > "/tmp/${job_id}.log" 2>&1
code=$?
python - "$job_id" "$code" "/tmp/${job_id}.log" <<'PY'
import json, sys
job_id, code, path = sys.argv[1], int(sys.argv[2]), sys.argv[3]
excerpt = open(path, encoding="utf-8", errors="replace").read()[-2000:]
print(json.dumps({
    "job_id": job_id,
    "command": "python -m unittest tests.test_gate",
    "exit_code": code,
    "log_excerpt": excerpt,
}))
PY

The excerpt bound is intentional. Unbounded logs and secrets do not belong in the packet. They also do not belong on a server you do not administer.

Where an optional model step sits

After the host has a grade, a model may restate it for a human comment. That helps when the reviewer wants a short note. It does not help when the grade is Missing or Derived and someone hopes wording will upgrade the leaf.

If you use free model access, send the redacted packet and the leaf table. Require the reply to echo the leaf id from leaf_action. If the reply names a stronger action than the function returned, keep the function's result. The function is the gate. The model is a narrator.

If you use the free server option, copy the grader and the redacted packet only. Do not copy environment files, credentials, or customer fixtures. Price does not describe access control. Until you have read the current retention notes, assume the packet could persist, and treat the server as a shared host.

Limitations

Four grades hide partial truth. An Observed packet can still record the wrong command. Job id match does not prove the suite covered the change. A human accept at L5 can be rubber-stamped, which turns the leaf into theater.

The tree ignores retry policy on purpose. A non-zero code is still Observed. Whether to rerun belongs in a separate rule set. Mixing those rules here would let a flake story erase an exit code.

Bounded excerpts drop context. A 2,000-character tail can miss the real failure higher in the log. Raise the bound only after you have a redaction pass, not before.

No accuracy, latency, or token figures are reported. If a page quotes a quota, verify it on the day you rely on it. This draft will not freeze a number that was not supplied as a durable term.

Who should skip it

Skip this gate if you need a formal attestation for an auditor. The tree is an engineering control, not a certification, and it keeps no tamper-evident history by itself.

Skip it if logs exist only inside a chat scroll. Build a host packet first, or accept that every claim stays Missing. A free run does not retroactively create the artifact.

Skip it if the team will override L5 whenever a run feels cheap. A gate that loses to convenience does not grade evidence. It decorates the close button.

If a sandbox for redacted packets is the missing piece, the operator points at MonkeyCode's free model access and free server option. Read the current terms, keep the leaf function authoritative, and leave secrets off that server until retention is clear to you.