48-Hour Field Notes: Two Free Surfaces, One Prompt, and the Run Manifest That Caught My Drift

What do you actually learn in 48 hours when nothing catches fire? Not much, unless you force the run to leave behind evidence you can diff later. That is the premise of these field notes: pick one boring task, run it on two different free execution surfaces, then compare what each run left behind. This time the task was a prompt-driven text cleanup job, and the two surfaces were free model access and a free server option.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. As of the time of writing, the availability claims I am working from — free model access and a free server option — are operator-supplied, and I have not invented quotas, model names, hardware, or duration limits to make the story tidier. Where I wanted a number and did not have one, you will see a placeholder instead of a guess.

The question I actually wanted answered

The question was never "which one is faster," because that question produces a screenshot and nothing reusable. The question I cared about was narrower and much less glamorous: if this environment disappears tomorrow, can I rebuild the run from artifacts alone? If the answer is no, then every result I report is really just a memory of a result.

So I spent two days treating the boring parts as the actual experiment. Same task, same prompt text, two surfaces, and one rule that I refused to break: no run starts until the capture script is already committed. Sounds excessive? That single rule is what turned a vague feeling of drift into three concrete, fixable problems.

Ground rules before hour zero

The rules matter more than the tools here, so let me put them on the table before any code shows up.

1. One run, one manifest directory, no shared scratch space between surfaces.
2. The prompt is a file on disk, hashed before the call and hashed again after it.
3. Any environment variable that reaches the process gets logged by name, never by value.
4. If two runs disagree, I diff artifacts first and theorize second.
5. Nothing gets published as a result until a replay produces the same manifest shape.

Does that sound slow? It is slow for the first run and roughly free for every run after it.

The artifact: one manifest per run

Here is the capture script, trimmed to the parts that have actually earned their place. It writes a fingerprint file plus an allowlisted environment dump into a per-run directory.

#!/usr/bin/env bash
# capture_run.sh - one manifest per run, no exceptions
set -euo pipefail
RUN_ID="$(date -u +%Y%m%dT%H%M%SZ)-$(git rev-parse --short HEAD 2>/dev/null || echo nogit)"
OUT="runs/${RUN_ID}"
mkdir -p "$OUT"

{
  echo "run_id=$RUN_ID"
  echo "utc_start=$(date -u +%FT%TZ)"
  echo "host=$(hostname)"
  echo "kernel=$(uname -sr)"
  echo "arch=$(uname -m)"
  echo "python=$(python3 -VV 2>&1 | head -n1)"
  echo "git_sha=$(git rev-parse HEAD 2>/dev/null || echo none)"
  echo "git_dirty_files=$(git status --porcelain | wc -l)"
  echo "deps_hash=$(python3 -m pip freeze 2>/dev/null | sort | sha256sum | cut -d' ' -f1)"
} > "$OUT/fingerprint.env"

# names only: never write secret values to disk
env | sort | cut -d= -f1 | grep -Ev '^(HOME|PATH|PWD|OLDPWD|SHLVL|_|TERM)$' > "$OUT/env.keys"

cp prompts/*.txt "$OUT/" 2>/dev/null || true
sha256sum "$OUT"/*.txt > "$OUT/prompt.sha256" 2>/dev/null || true

Why a fingerprint file beats a README paragraph

A README says "runs on Python 3" and quietly assumes that sentence is still true next month. A fingerprint file says exactly which interpreter, which architecture, and how many dirty files existed at the moment the run started. When two runs disagree, the fingerprint tells you whether you are debugging the task or debugging the machine.

The manifest itself stays deliberately small, because a manifest you cannot read is a manifest you will not read:

{
  "run_id": "<utc-timestamp>-<short-sha>",
  "surface": "free-model-access | free-server",
  "prompt_sha256": "<hash>",
  "deps_hash": "<hash>",
  "artifact_paths": ["out/result.txt"],
  "notes": "<one sentence about anything unusual>"
}

Notice what is not in there: no latency claims, no quality score, no "10x faster" line. I am not publishing a benchmark, because I did not run a controlled one, and dressing anecdote up as measurement is how field notes turn into marketing.

What broke, in order

Hour 4: the dependency hash moved under me

The first replay produced a result that looked slightly different, and my instinct was to blame the model. The diff said otherwise: the dependency hash had changed because a transitive package resolved differently on the second surface. Did the task change? No. Did my environment change? Yes, and I had nearly spent an afternoon theorizing about the wrong layer.

Hour 19: "the same prompt" was not the same request

I had been building the prompt string inline, which meant whitespace normalization and one stray trailing newline differed between surfaces. Hashing the prompt as a file, not as a variable, killed that entire class of bug in about ten minutes. This is the one change I would make on day one next time, without hesitation.

Hour 40: my retry loop was the loudest variable

My own client retried on timeouts and logged only the final attempt, so the manifest described a run that never really happened once. The fix was boring: log every attempt with its own line, and never let a retry silently replace the record of the original call.

The diff script that turned guessing into a checklist

This is the whole payoff of two days of bookkeeping, and it fits in a screen.

#!/usr/bin/env bash
# replay_diff.sh - compare two manifests, fail loudly on drift
set -euo pipefail
A="$1"; B="$2"

echo "== fingerprint drift =="
diff -u "$A/fingerprint.env" "$B/fingerprint.env" || true

echo "== env key drift (names only) =="
cut -d= -f1 "$A/env.keys" | sort > /tmp/a.keys
cut -d= -f1 "$B/env.keys" | sort > /tmp/b.keys
diff -u /tmp/a.keys /tmp/b.keys || true

echo "== prompt hash =="
cat "$A/prompt.sha256" "$B/prompt.sha256" 2>/dev/null || echo "missing prompt hash"

Running this before reading any output is the habit I would keep forever. It answers the only question that matters at that moment: am I comparing two runs, or two machines?

Decision table: which free surface first

Task shape	Where I would start	Why
One-shot, stateless text transform	Free model access	The request is the whole unit of work; nothing needs to persist.
Long-running process, cron, file watching	Free server option	You want a shell that outlives a single call.
Anything holding secrets or customer data	Your own machine	Free surfaces are for throwaway work, not compliance.
A result you plan to publish	Neither, until manifests exist	Without evidence you are publishing a vibe.
Latency-sensitive production path	Neither	Prototype freely, promise nothing.

What I would repeat, and what I would drop

• Repeat: hashing the prompt as a file, before and after the call.
• Repeat: logging environment variable names only, never values.
• Repeat: writing the manifest before reading any output at all.
• Drop: inline prompt construction, which hides whitespace drift.
• Drop: silent retries that erase the first attempt from the record.
• Drop: reading results before diffing fingerprints, which wastes hours.

Limitations, and who should skip this

This workflow tells you whether two runs are comparable; it does not tell you which output is better. If your task needs human judgment, a manifest will not rescue you. If you operate under an SLA, handle regulated data, or need pinned images for audit, free surfaces are the wrong starting point. And if you only ever run one task once, the harness costs more than it returns.

What I am changing in the next 48 hours

Next round I am adding one thing only: a --dry-run mode that writes the manifest and exits before the actual call. That catches prompt and environment drift without spending a single request. If you want a sandbox to try the manifest habit without a bill attached, free model access and a free server option were the surfaces I used here, and the disclosure above is worth taking literally. Keep the boring parts boring, and the interesting parts will finally be visible.