FAQ: Five Myths About "It Worked on the Free Host"

Did that free remote host just bless your patch?

I hear that line in too many agent threads.

A green remote command feels like a merge vote.

Is it a vote, or just one noisy sample?

This FAQ kills five claims I still hear.

Each myth gets evidence and a corrected mental model.

I also give you a host claim file.

I run that recorder on two different hosts.

Then I argue with hashes, not with vibes.

Why cheap runs still lie

Are you mixing cheap retries with a safe ship?

I do that when the loop looks busy and sure.

A free model makes another attempt feel free.

A free server makes another machine feel free.

Neither one makes a bad merge feel free.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I use MonkeyCode when I want free model access.

I also use the free server option for isolated runs.

The product gives me a host and a model path.

It does not give me a merge decision.

Strip every product name out and the checks still stand.

The artifact: a host claim file

Want one JSON file that ends the fight?

I record identity, argv, hashes, and replay policy.

Treat the script below as a proposal, still unexecuted.

I am not publishing timings, quotas, or hardware claims.

#!/usr/bin/env python3
"""host_claim.py — proposal. Not a benchmark harness."""
from __future__ import annotations

import argparse
import hashlib
import json
import os
import platform
import subprocess
import sys
from datetime import datetime, timezone
from pathlib import Path


def sha256_text(s: str) -> str:
    return hashlib.sha256(s.encode("utf-8", "replace")).hexdigest()


def git(*args: str) -> str:
    p = subprocess.run(
        ["git", *args],
        check=False,
        capture_output=True,
        text=True,
    )
    return p.stdout.strip() if p.returncode == 0 else ""


def snapshot(cmd: list[str]) -> dict:
    proc = subprocess.run(cmd, check=False, capture_output=True, text=True)
    return {
        "recorded_at": datetime.now(timezone.utc).isoformat(),
        "argv": cmd,
        "exit_code": proc.returncode,
        "stdout_sha256": sha256_text(proc.stdout),
        "stderr_sha256": sha256_text(proc.stderr),
        "cwd": os.getcwd(),
        "python": sys.version,
        "platform": platform.platform(),
        "executable": sys.executable,
        "git_sha": git("rev-parse", "HEAD"),
        "git_dirty": bool(git("status", "--porcelain")),
        "replay_required": True,
        "merge_ready": False,
    }


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("command", nargs=argparse.REMAINDER)
    parser.add_argument("--out", default="claim.json")
    parser.add_argument("--replay", default="")
    args = parser.parse_args()

    if args.replay:
        previous = json.loads(Path(args.replay).read_text())
        cmd = previous["argv"]
        now = snapshot(cmd)
        now["matches_previous"] = (
            now["exit_code"] == previous["exit_code"]
            and now["stdout_sha256"] == previous["stdout_sha256"]
            and now["git_sha"] == previous["git_sha"]
        )
        Path(args.out).write_text(json.dumps(now, indent=2) + chr(10))
        print(json.dumps(now, indent=2))
        return 0 if now["matches_previous"] else 2

    if not args.command:
        print("pass a command after --", file=sys.stderr)
        return 2
    cmd = args.command[1:] if args.command[0] == "--" else args.command
    data = snapshot(cmd)
    Path(args.out).write_text(json.dumps(data, indent=2) + chr(10))
    print(json.dumps(data, indent=2))
    return data["exit_code"]


if __name__ == "__main__":
    raise SystemExit(main())

How do I actually use this recorder?

1. Run it on the free host after the agent.
2. Copy claim.json next to your working patch.
3. Replay it on your laptop at the same sha.
4. Diff the two files before anyone merges.

No match means you lack a portable result.

You only have a story from one box.

python3 host_claim.py --out claim.remote.json -- python3 -m compileall -q src
python3 host_claim.py --replay claim.remote.json --out claim.local.json
echo $?

Exit 2 means the host story did not travel.

I fail closed and I refuse silent mismatches.

Myth 1: The free host is my laptop

The claim

It worked there, so it will work here.

Does the same repo name imply the same runtime?

I treat every remote host as a stranger first.

The evidence

Have you compared Python, libc, and the working tree?

Run this on both sides before you argue.

python3 -c "import sys,platform; print(sys.version); print(platform.platform())"
git rev-parse HEAD
git status --porcelain
uname -a

A mismatch means the green check is host-scoped.

Same files can still hide a different interpreter.

The corrected model

A free host is a sample, not a twin.

Record the sample instead of impersonating your laptop.

Portability is proven by replay, not by confidence.

Myth 2: Exit code zero is a merge signal

The claim

The agent printed done and the last command returned zero.

Does zero mean the product behavior is right?

The evidence

I have seen zero from skipped tests.

I have seen zero from a printed success string.

I have seen zero from a dry run flag.

Demand this evidence every single time:

• I want the exact argv, not a summary sentence.
• I want the working directory and the git sha.
• I want stdout and stderr hashes, not vibes.
• I want a second run that I typed myself.

CMD=(python3 -m compileall -q src)
"${CMD[@]}"
echo "exit:$?"
echo "sha:$(git rev-parse HEAD)"
echo "cwd:$PWD"

Zero is cheap to mint, while hashes are harder to fake.

The corrected model

Zero is a sensor, and it is not a release.

Sensors lie when you measure the wrong thing.

Merge on a review, not on a smiling prompt.

Myth 3: Free models need a lower evidence bar

The claim

It is just a free model. Ship the diff.

Why would a cheaper attempt need less proof?

The evidence

I see this when people vibe-code on a budget.

The cost of another attempt dropped overnight.

The cost of a production bug did not drop.

A free model can still write a wrong tree.

A free server can still run that wrong tree.

The transcript looks complete while the invariant is not.

The corrected model

Price of inference is not price of review.

I still want the same diff, tests, and human gate.

Cheap attempts are for exploration, not skipped gates.

Ask: what evidence would I demand from a paid model?

Demand that evidence and do not discount it.

Myth 4: The free server is my CI

The claim

I ran it on a clean free server. That is CI.

Is an ad-hoc SSH session a pipeline?

The evidence

CI has a pinned image and a durable log.

CI has a trigger from a known git ref.

CI has policy, not a tab you might close.

A scratch host has a prompt and a hope.

I refuse to call that a required check.

What CI has that a scratch host lacks:

• CI has an image digest or runner lock.
• CI has a repeatable trigger from the same ref.
• CI keeps artifacts you can pull tomorrow.
• CI has access control beyond a leftover tab.

The corrected model

A free server is a lab bench.

Lab benches are for experiments, and pipelines are for gates.

Promote the command into CI after it is boring.

Do not promote the bench itself into production.

Myth 5: Disposable hosts mean I can skip replay

The claim

The box is free and throwaway. Why reproduce locally?

Because merge happens on your branch, not theirs.

The evidence

A disposable host disappears, but the bug will not.

If I cannot replay the argv, I lack a report.

I have a ghost with a nice timestamp.

Minimal local replay at the recorded sha:

git fetch
git checkout --detach "$REMOTE_SHA"
python3 host_claim.py --replay claim.remote.json --out claim.local.json

If git sha differs, you are not even close.

Stop, align the trees, and then replay the argv.

The corrected model

Disposable compute still needs a durable claim file.

Throw away the VM, but keep the evidence.

Ghosts do not belong in the merge box.

Decision table I actually use

Heard in the thread	What you possess	Merge?
Free host said OK	One env sample	Not yet
Free model finished	A chat transcript	Not yet
Clean server, no drama	Unknown image	Not yet
Looks like CI to me	An ad-hoc shell	Not yet
Two claims match, plus CI	Portable evidence	Then review

Read the last row again, and read it slowly.

Two matching claims are necessary, but they are not sufficient.

You still read the diff like an adult.

What a matching pair still is not

Two claim files can match and still hide a logic bug.

Hashes agree on stdout but they do not understand domain rules.

I still read the diff after the recorder goes green.

The recorder answers same output, never right output.

That gap is where engineering still lives.

A tiny test plan for the recorder

Do not trust host_claim.py without abusing it.

1. Dirty the tree on purpose, then record a claim.
2. Confirm git_dirty is true in the JSON.
3. Commit, then rerun, and confirm the sha moved.
4. Flip one environment variable the command needs.
5. Confirm replay fails closed with a non-zero exit.

If step five still passes, your recorder is theater.

I fail closed and I refuse silent mismatches.

# test_host_claim.py — proposal, unexecuted in this article
import unittest
import host_claim


class ClaimTests(unittest.TestCase):
    def test_zero_is_not_merge_ready(self):
        snap = host_claim.snapshot(["python3", "-c", "print('ok')"])
        self.assertEqual(snap["exit_code"], 0)
        self.assertFalse(snap["merge_ready"])
        self.assertTrue(snap["replay_required"])

    def test_hash_changes_with_stdout(self):
        a = host_claim.sha256_text("ok\n")
        b = host_claim.sha256_text("ok\n ")
        self.assertNotEqual(a, b)


if __name__ == "__main__":
    unittest.main()

Notice that merge_ready stays false on purpose.

The recorder must never promote itself to CI.

Limitations

This workflow will annoy you on tiny throwaway gists.

It will not replace a real pipeline.

It will not pin models, quotas, or server hardware.

I am not claiming those product details here.

It assumes the same argv can run in two places.

Some bugs are truly host-only, so the files will disagree.

That disagreement is the point, so do not hide it.

Flaky tests will also disagree, and that is useful.

You still need to classify flake versus drift.

The claim file cannot do that classification for you.

Who should not use this

Do not paste secrets into a free host claim file.

Do not send regulated data to a shared free server.

Do not use this as production change management.

Do not skip review because two JSON hashes matched.

If your org forbids unknown remote runners, stop here.

Use approved CI and ignore the lab bench.

If you cannot replay at a pinned sha, stop here too.

You are not ready for this workflow yet.

What I do instead of believing the host

I ask four questions before I type merge.

1. What argv actually ran on that host?
2. On which git sha and which platform string?
3. Did I replay it locally or in CI?
4. What evidence would falsify this claim tomorrow?

If I cannot answer, I do not have a result.

I have a vibe with a log attachment.

Cheap models and cheap hosts are great for drafts.

They are bad at being the last reviewer.

Keep the free attempt and raise the evidence bar.

Park a claim file beside the diff on that free box.