Borrowed Cycles Are Not a Baseline: A Myth-Busting FAQ

A reviewer opened a Monday pull request that already claimed a full suite pass on a complimentary remote host. The comment thread treated that machine like a shared CI worker, even though nobody could name the image, the clock, or the compiler. By afternoon a second engineer tried to reproduce the failure locally, and the loaner box had already been recycled. The chat transcript was the only remaining artifact, which is a poor substitute for a recorded run.

This pattern is spreading because coding agents now sit one command away from a scratch kernel. Teams confuse a green stream of tokens with a baseline they can cite next week. The corrected mental model is simpler than the folklore: borrowed cycles are a notepad, and evidence is a bundle you export before the box evaporates.

Myth: a green loaner run is continuous integration

The claim sounds reasonable in Slack. An agent opened a shell, installed dependencies, invoked the test runner, and pasted a summary that ended with a zero exit. People then write “CI is green on the remote box,” as if the complimentary host were a named worker with a pinned image and retained logs. That sentence smuggles three unearned properties: identity of the machine, retention of artifacts, and a contract that the same tree will be rebuilt tomorrow.

A hotel key is not a house deed, even when the door opens on the first try. Continuous integration is a recorded contract among a commit SHA, an image digest, a command, and stored outputs. A free remote shell is usually none of those things unless you wrap it. The agent’s success message is closer to a witness statement than to a junit file stored beside the commit.

You can see the difference with commands that CI systems treat as mundane and loaner boxes often skip. The following block is a proposed snapshot, not a claim about any particular vendor image.

# Proposed local or remote snapshot — run inside the repo worktree.
git rev-parse HEAD
git status --porcelain=v1
git diff --stat HEAD
uname -a
python3 --version
${CC:-cc} --version | head -n 1
test -f poetry.lock && sha256sum poetry.lock
test -f package-lock.json && sha256sum package-lock.json

If those lines were never captured, the “green” result cannot be replayed. A later checkout may match the path and still miss the lockfile, the toolchain, or an uncommitted fixture the agent wrote only on the loaner disk. Treating that gap as CI noise is how merge queues absorb folklore.

Myth: a free model endpoint is a frozen evaluation baseline

The second claim appears when teams start weekly agent evals. They point a prompt corpus at whatever complimentary model access is currently advertised, then compare this Wednesday’s patch quality with last Wednesday’s. The spreadsheet looks quantitative. The axis on the chart is unlabeled time, not a pinned model snapshot, so the comparison is mostly weather.

Baselines need three pins that marketing pages do not provide: the model identifier as the server actually received it, the decoding settings, and the tool schema hash. Without those, a “better” week may be a quieter sampler, a shorter context, or a different hidden system prompt. The scientific analogy is calibrating a scale after someone swapped the weights and kept the same sticker on the bench.

A small provenance record makes the missing pins visible. Label this as a schema proposal you can write beside any agent run, including one that used free model access.

{
  "git_sha": "REPLACE_WITH_rev_parse",
  "tree_dirty": true,
  "prompt_sha256": "",
  "tool_schema_sha256": "",
  "model_id_reported": "unspecified",
  "endpoint_host": "unspecified",
  "temperature": null,
  "max_tokens": null,
  "started_at_utc": "",
  "finished_at_utc": "",
  "junit_path": "artifacts/junit.xml",
  "patch_path": "artifacts/agent.patch"
}

Fill empty strings from the run, not from memory. If the product UI does not disclose a model identifier, write unspecified rather than inventing a brand name. Honesty in the JSON is more useful than a confident guess that future readers will treat as ground truth.

Some teams rehearse that export step with open coding-agent projects. MonkeyCode currently offers free model access and a free server option that can host an exploratory loop while you practice the bundle. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The recorder below does not require that project; it needs a shell, a git tree, and a writable artifacts directory.

Myth: an ephemeral workspace means you can skip pinning the tree

The third claim is a cousin of the second. Engineers say the complimentary server will be wiped, so there is no point in recording git status or copying the patch off the box. Ephemerality is then used as a reason to keep less evidence, which is the opposite of what a lab notebook would do with a disposable bench.

Scratch paper still needs a photograph if the calculation matters. Before logout you want three files that survive the recycle: the exact diff the agent produced, the test XML or log, and the provenance document. Those files turn a vanished kernel into something a teammate can apply on a laptop.

#!/usr/bin/env bash
# record_run.sh — proposed recorder. Review before running on a shared host.
set -euo pipefail
ROOT="$(git rev-parse --show-toplevel)"
OUT="${ROOT}/artifacts/run-$(date -u +%Y%m%dT%H%M%SZ)"
mkdir -p "${OUT}"

{
  echo "git_sha $(git rev-parse HEAD)"
  echo "git_status"
  git status --porcelain=v1 || true
  echo "uname $(uname -a)"
  echo "python $(python3 --version 2>&1)"
} > "${OUT}/env.txt"

git diff HEAD > "${OUT}/agent.patch" || true

if [[ -f "${ROOT}/.pytest_cache" ]] || command -v pytest >/dev/null; then
  pytest -q --junitxml="${OUT}/junit.xml" || true
fi

python3 - <<'PY' "${OUT}"
import hashlib, json, pathlib, sys
out = pathlib.Path(sys.argv[1])
prompt = pathlib.Path("prompt.txt")
def sha(p):
    return hashlib.sha256(p.read_bytes()).hexdigest() if p.exists() else ""
doc = {
    "git_sha_line": (out / "env.txt").read_text(encoding="utf-8").splitlines()[0],
    "prompt_sha256": sha(prompt),
    "patch_sha256": sha(out / "agent.patch"),
    "junit_present": (out / "junit.xml").exists(),
}
(out / "provenance.json").write_text(json.dumps(doc, indent=2), encoding="utf-8")
print(out / "provenance.json")
PY

Keep prompt.txt next to the repo root when you start the agent so the hash is not reconstructed from chat later. If pytest is absent, the script still stores the patch and environment, which is already more than a streamed “all tests passed.” A reviewer can then run git apply artifacts/run-*/agent.patch on a known laptop image and believe or reject the claim.

Myth: free tokens plus an open agent equal an audit trail

Open source helps you read the client. It does not automatically archive what happened on a particular Tuesday. People repeat the opposite because the repository is public, the allowance is complimentary, and the server is described as free, which feels like transparency. Transparency of code is not retention of a run.

An audit trail answers who invoked which tool against which tree with which bytes out. Chat products optimize for continuity of conversation, not for those bytes. If your compliance question is “what did we merge and why,” you still need the patch, the test artifact, and the hashes. The license on the agent cannot generate those files after the working directory is gone.

A tiny checker makes the absence obvious instead of polite. This example is executable once provenance.json exists; it does not grade patch quality.

# check_provenance.py — fail closed when evidence is missing.
from pathlib import Path
import json, sys

def main(path: str) -> int:
    doc = json.loads(Path(path).read_text(encoding="utf-8"))
    missing = [k for k in ("git_sha_line", "patch_sha256") if not doc.get(k)]
    if not doc.get("junit_present"):
        missing.append("junit_present")
    if missing:
        print("incomplete provenance:", ", ".join(missing))
        return 2
    print("provenance bundle looks complete")
    return 0

if __name__ == "__main__":
    sys.exit(main(sys.argv[1]))

Exit status two is more honest than a green paragraph in the agent transcript. Teams that adopt the checker usually discover that half of their “successful” loops never wrote junit at all, which is an evidence problem rather than a model-quality problem.

When borrowed cycles still help

Complimentary remote execution remains useful when the question is exploratory. You may want to know whether a refactor even parses, whether a flaky test blows up under a second architecture, or whether an agent can navigate a large tree without filling a laptop SSD. Those are notepad questions. They become merge questions only after the recorder has run and a human has replayed the patch on a machine the team actually owns.

Think of the free server as a wind tunnel you do not calibrate. You can learn that a shape is noisy. You should not publish the airspeed as a certified number. The same caution applies to free model access: it is a way to exercise prompts and tools, not a reason to skip pins.

If you want a scratch box for record_run.sh, the free server option mentioned above is one place to try the export without buying hardware first. Check the project’s current docs for whatever allowance is offered that week, because complimentary tiers change and this article will not freeze a quota.

Limitations, and who should not use this approach

The recorder does not make a shared kernel safe for secrets. Do not export .env files, cloud keys, or production dumps onto a complimentary host, and do not paste customer data into a free model endpoint. The script also does not replace a real CI system with image digests, signed logs, and retention policies.

Skip this workflow if you are in a regulated environment that already forbids unsanctioned runners. Skip it if you need performance numbers; nothing here is a benchmark, and no latency from a free pool should be cited as capacity planning. Skip it if the team will not replay patches locally, because the JSON file then becomes another unread attachment.

Borrowed cycles can shorten the path to a hypothesis. They cannot, by themselves, tell you what landed in the tree. Treat the complimentary box as a bench you photograph, not as the laboratory whose calibration certificate you file.