The Model Replied. That Is Not a Job Receipt

Did the model finish, or did the job finish? Those two events are not the same. I still watch teams treat a token stream like an exit code. Why do we keep doing that?

A completion can look finished and still touch nothing. A free model can draft a script you never launched. A free server can sit idle while the chat celebrates.

This FAQ is about that mix-up. Not about tests the agent invented. Not about pinning. Not about chat logs versus git status. This is model output versus job execution.

The claim I keep hearing

Someone pastes a long answer. Then they say the work ran. Did it? Where is the host? Where is the exit code? Where is the artifact hash?

I treat those questions as the real review. The prose is not evidence. The receipt is evidence.

FAQ: five myths, then a better model

Myth 1: The model printed a command, so the command ran

The claim: The assistant showed pytest or make. Therefore it executed.

What you actually have: Text. Maybe a fenced block. Maybe a plausible flag list. That is generation, not a process.

Corrected model: A printed command is a proposal. A job receipt needs argv, cwd, an exit code, and timestamps. No process table, no run.

Ask yourself this. Could you paste the same block into a gist? If yes, nothing executed yet.

Myth 2: Using a free model already used a free server

The claim: Inference and execution are one product moment. Pay nothing, so both halves happened.

What you actually have: A language model call. That call does not boot your workspace. It does not mount your repo. It does not keep your secrets out of the prompt.

Corrected model: Free model access and a free server option are two knobs. One writes tokens. One runs a job. Mixing them hides failures on both sides.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I mention MonkeyCode only because its free model access and free server option map onto that split. I will not invent model names, quotas, hardware, or runtimes. Record whatever your client actually reports.

Myth 3: A green chat is a green job

The claim: The thread ended with “done.” So CI would be green too.

What you actually have: Narrative closure. Models are trained to sound finished. Finished sentences are cheap. Finished jobs are not.

Corrected model: Green means a recorded status from the runner. 0 is a number. “Looks good” is a vibe. I refuse to merge on vibe.

Myth 4: The same transcript proves the same host

The claim: Yesterday’s thread plus today’s paste equal a rerun on the same machine.

What you actually have: Scrollback. Hosts rotate. Workspaces linger. Env vars drift. Caches lie. A transcript cannot fingerprint a box.

Corrected model: Identity the host like you identity a commit. Capture hostname, image id, uname, and a workspace hash before the command. After the command, capture them again. Drift is a failed receipt.

Myth 5: I can reconstruct the run from the answer text

The claim: If the model quoted logs, the logs existed.

What you actually have: A quotation. Quotations are easy to invent. They are also easy to trim. They rarely include the failing stderr you needed.

Corrected model: Logs live in files you hash. If you cannot sha256sum the log, you do not have the log. Quoted output is fan fiction until the file exists.

The dual-receipt mental model

I keep two envelopes. Never one.

1. Generation receipt — prompt hash, client metadata, completion hash, clock.
2. Execution receipt — host fingerprint, argv, cwd, exit code, artifact hashes, clock.

If either envelope is empty, the work is not done. That rule is boring. Boring is the point.

Artifact: a dual-receipt workflow you can copy

This is a proposed local workflow. I am not claiming I ran these numbers in production. Label it unexecuted until you run it on your box.

Step 1 — freeze the prompt

Write the prompt to disk. Hash it. Do not trust the chat UI as storage.

mkdir -p receipts/gen receipts/job
cat > receipts/gen/prompt.txt <<'EOF'
Write a Python script that prints the current working directory
and the value of RECEIPT_RUN_ID, then exits 0.
EOF
sha256sum receipts/gen/prompt.txt | tee receipts/gen/prompt.sha256

Step 2 — record the generation receipt

Fill only fields your client truly returns. Leave unknowns as null. Do not invent a model name.

# dual_receipt.py — proposed helper, not a benchmark
import hashlib, json, os, time
from pathlib import Path

def sha256_file(path: Path) -> str:
    h = hashlib.sha256()
    h.update(path.read_bytes())
    return h.hexdigest()

def write_json(path: Path, payload: dict) -> None:
    path.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")

def generation_receipt(
    prompt_path: Path,
    completion_path: Path,
    client_reported_model,
    out_path: Path,
) -> None:
    payload = {
        "kind": "generation",
        "recorded_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
        "prompt_sha256": sha256_file(prompt_path),
        "completion_sha256": sha256_file(completion_path),
        "client_reported_model": client_reported_model,  # null if unknown
    }
    write_json(out_path, payload)

def host_fingerprint() -> dict:
    return {
        "hostname": os.uname().nodename,
        "sysname": os.uname().sysname,
        "release": os.uname().release,
        "cwd": os.getcwd(),
        "run_id": os.environ.get("RECEIPT_RUN_ID"),
    }

def execution_receipt(argv, exit_code, artifacts, out_path: Path) -> None:
    payload = {
        "kind": "execution",
        "recorded_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
        "argv": argv,
        "exit_code": exit_code,
        "host_before": host_fingerprint(),
        "artifacts": {
            str(p): sha256_file(Path(p)) for p in artifacts
        },
    }
    write_json(out_path, payload)

Save the raw completion next to the prompt. Hash that too.

# You paste the model output. Then you hash it.
sha256sum receipts/gen/completion.txt | tee receipts/gen/completion.sha256
python3 - <<'PY'
from pathlib import Path
from dual_receipt import generation_receipt
generation_receipt(
    Path("receipts/gen/prompt.txt"),
    Path("receipts/gen/completion.txt"),
    client_reported_model=None,
    out_path=Path("receipts/gen/receipt.json"),
)
print("generation receipt written")
PY

Step 3 — run on a real host, not in the transcript

Extract the script the model proposed. Put it in git. Then run it where jobs actually run. A free server option is useful here because it is a separate machine from the chat. Chat cannot pretend to be argv.

export RECEIPT_RUN_ID="run-$(date -u +%Y%m%dT%H%M%SZ)"
chmod +x ./proposed_job.py
set +e
./proposed_job.py > receipts/job/stdout.txt 2> receipts/job/stderr.txt
echo $? > receipts/job/exit_code.txt
set -e
sha256sum receipts/job/stdout.txt receipts/job/stderr.txt

Then write the execution envelope.

python3 - <<'PY'
from pathlib import Path
from dual_receipt import execution_receipt
exit_code = int(Path("receipts/job/exit_code.txt").read_text().strip())
execution_receipt(
    argv=["./proposed_job.py"],
    exit_code=exit_code,
    artifacts=["receipts/job/stdout.txt", "receipts/job/stderr.txt"],
    out_path=Path("receipts/job/receipt.json"),
)
print("execution receipt written")
PY

Step 4 — fail closed

test -s receipts/gen/receipt.json || { echo "missing generation receipt"; exit 1; }
test -s receipts/job/receipt.json || { echo "missing execution receipt"; exit 1; }
test "$(cat receipts/job/exit_code.txt)" = "0" || { echo "job failed"; exit 1; }
echo "both receipts present; job exit 0"

No completion hash? Stop. No exit code file? Stop. Chat said success? Ignore the chat.

Decision table: what you may claim

You observed	You may say	You may not say
Completion file hashed	The model returned bytes	The job ran
Client reported a model field	The client reported that field	That field is true forever
Exit code file equals 0	This argv exited 0 on this host	Every environment is green
Artifact hashes on disk	These files existed at hash time	The model “saw” them
Free model call succeeded	Generation happened	A server job happened
Free server accepted a job	A host was scheduled	The answer text is the log

Print this table near your agent loop. I do. It kills arguments fast.

A tiny assertion test for the helper

Still a proposal. Run it only on your machine.

# test_dual_receipt.py
from pathlib import Path
from dual_receipt import generation_receipt, sha256_file

def test_generation_receipt_hashes_files(tmp_path: Path):
    prompt = tmp_path / "prompt.txt"
    completion = tmp_path / "completion.txt"
    out = tmp_path / "receipt.json"
    prompt.write_text("hello\n")
    completion.write_text("world\n")
    generation_receipt(prompt, completion, None, out)
    text = out.read_text()
    assert sha256_file(prompt) in text
    assert sha256_file(completion) in text
    assert "execution" not in text

Notice the last assert. A generation document must not pose as execution. That is the whole article in one check.

Limitations

This workflow does not prove correctness of the program. It proves you separated thinking from running. Hashes do not replace code review. Exit code 0 can still mean a script that skipped the real work.

Clocks can skew. Hostnames can be reused. os.uname() is a weak fingerprint. If you need stronger identity, record an image digest your orchestrator already knows. Do not invent one.

Free model access can disappear. A free server option can disappear. I am not claiming permanence. If either knob vanishes, the dual-receipt rule still holds. You just need another generator and another host.

Secrets do not belong in prompts or in receipt JSON. Hash inputs. Redact env. Store tokens in the runner’s secret mechanism, not in receipts/.

Who should not use this

Skip this if you already have a real CI system with artifact attestation. That system is the execution receipt. Do not wrap it in extra JSON for theater.

Skip this if the task is a question with no side effects. “Explain this error” needs no host fingerprint.

Skip this if you cannot write files. A receipt that lives only in a chat window is another myth.

Skip this if you need guaranteed capacity. Free generation and free execution are availability claims, not SLAs. I will not dress them up as benchmarks.

What I actually want in review

Open the two JSON files. Then ask four questions.

• Did we hash the prompt we thought we sent?
• Did we hash the completion we thought we got?
• Did a host run a concrete argv?
• Do the artifact hashes match the files in the change?

If a teammate answers with a screenshot of a chat bubble, I bounce the review. Screenshots are not sha256sum.

Would you ship a binary with no build log? Then do not ship an agent change with no job receipt.

The model can be free. The server can be free. The discipline is not free. You still have to keep two envelopes. If you already have both knobs in one workspace, use them as two receipts, not one story.