Ask Again Is Not a Build: A Reproducibility FAQ

Can you rebuild last Tuesday without opening the chat?

I keep hearing the same shrug. People say they will just ask again.

That shrug is the bug. A free model and a free server make the shrug feel cheap. Cheap is not the same as recoverable.

This FAQ is for that gap. Not latency. Not quota theater. Recoverability.

Why this FAQ, not another benchmark post

I already wrote enough about p50 lies. This is a different failure.

The failure looks like progress. The model installs a tool. Tests pass on a box you did not pin. You close the tab.

Two days later the same prompt yields a different tree. Who owns that drift?

Where a free pairing actually helps

I use a throwaway box when I want a clean PATH. I do not treat that box as memory.

MonkeyCode offers free model access and a free server option. That pairing is useful as a scratch space. It is not a source of truth.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I am not claiming model names. I am not claiming quotas, hardware, or uptime. Those numbers go stale overnight. Pin your own receipts instead.

The five claims I still hear

Read each claim. Then read the evidence you can collect in one sitting. Then steal the corrected model.

Myth 1: The chat log is the runbook

Claim: If the transcript is long, the work is documented.

Evidence to collect: Clone a fresh directory. Follow only committed files. Do not open the chat.

Did the project boot? If not, the chat was oral tradition.

Corrected model: Chat is a scratch pad. Git is the runbook. Prompts expire. Commits do not.

Ask yourself one rude question. Could a teammate finish without your tabs?

Myth 2: The free server PATH equals my laptop PATH

Claim: It ran there, so it runs here.

Evidence to collect: Dump both environments. Diff them. Do not eyeball.

#!/usr/bin/env bash
set -euo pipefail
out="${1:-receipt.env.txt}"
{
  echo "# host $(uname -a)"
  echo "# date $(date -u +%Y-%m-%dT%H:%M:%SZ)"
  echo "SHELL=${SHELL:-}"
  echo "PWD=${PWD}"
  command -v python3 >/dev/null && echo "PYTHON3=$(command -v python3) $(python3 -V 2>&1)"
  command -v node >/dev/null && echo "NODE=$(command -v node) $(node -v)"
  command -v go >/dev/null && echo "GO=$(command -v go) $(go env GOVERSION 2>/dev/null)"
  echo "PATH=${PATH}"
} > "$out"
echo "wrote $out"

Run it locally. Run it on the remote box. Diff the files.

Corrected model: Same repo, different ghosts. PATH is a dependency. Treat it like one.

Myth 3: If the model installed it, it is pinned

Claim: pip install foo in chat equals a lockfile.

Evidence to collect: Search the repo for a lockfile. Then search git history for the install command.

No lockfile? You have a vibe, not a pin.

git ls-files | rg -n 'package-lock.json|pnpm-lock.yaml|poetry.lock|go.sum|Cargo.lock' || true
git log --oneline --all -- package-lock.json pnpm-lock.yaml poetry.lock go.sum Cargo.lock

Corrected model: Installation is an event. Pinning is a file. Events vanish. Files review.

Would you accept a production deploy that says "ask the bot for versions"?

Myth 4: Ephemeral boxes make secrets casual

Claim: The server is free and temporary, so .env in the prompt is fine.

Evidence to collect: Grep the transcript export. Grep the repo. Grep shell history.

rg -n -i 'api[_-]?key|secret|token|BEGIN OPENSSH|AKIA' . \
  --glob '!.git/**' || true

Did anything match? Rotate it. Do not debate the box lifetime.

Corrected model: Free does not mean forgetful. Logs outlive the VM story in your head.

I do not paste live credentials into any model. Use a dummy value in examples. Inject secrets outside the prompt.

Myth 5: Asking again reproduces the same tree

Claim: The prompt is deterministic enough. Rerun it later.

Evidence to collect: Save the prompt. Save the receipt. Do not save the vibe.

Change one thing. Model routing. Package index. Base image. Clock.

Did you get the same git status? Probably not.

Corrected model: Prompts sample. Builds replay. Sampling is for ideas. Replay is for shipping.

Artifact: a session receipt you can actually commit

I wanted one file that answers a mean question. Could I recreate this without the chat?

Label this as a checklist script, not a production platform. I am not publishing timed benchmarks here. Dates move. Receipts should not.

Save this as scripts/write_receipt.sh.

#!/usr/bin/env bash
# Label: example workflow, not a vendor SLA.
set -euo pipefail

root="$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
cd "$root"
out="SESSION_RECEIPT.md"
now="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
head_sha="$(git rev-parse HEAD 2>/dev/null || echo 'NO_GIT')"
branch="$(git branch --show-current 2>/dev/null || echo 'NO_BRANCH')"
dirty="$(git status --porcelain 2>/dev/null | wc -l | tr -d ' ')"

lock_hashes=""
for f in package-lock.json pnpm-lock.yaml yarn.lock poetry.lock \
         requirements.txt go.sum Cargo.lock Gemfile.lock; do
  if [[ -f "$f" ]]; then
    lock_hashes+="$(cksum "$f")"$'\n'
  fi
done

{
  echo "# Session receipt"
  echo
  echo "- utc: $now"
  echo "- git_head: $head_sha"
  echo "- branch: $branch"
  echo "- dirty_paths: $dirty"
  echo "- uname: $(uname -a)"
  echo "- pwd: $root"
  echo
  echo "## Toolchain"
  echo
  command -v python3 >/dev/null && echo "- python3: $(python3 -V 2>&1) @ $(command -v python3)"
  command -v node >/dev/null && echo "- node: $(node -v) @ $(command -v node)"
  command -v npm >/dev/null && echo "- npm: $(npm -v)"
  echo
  echo "## Lockfile checksums"
  echo
  if [[ -n "$lock_hashes" ]]; then
    echo '```

'
    printf '%s' "$lock_hashes"
    echo '

```'
  else
    echo "- none found. this is a smell."
  fi
  echo
  echo "## Dirty files"
  echo
  echo '```

'
  git status --porcelain 2>/dev/null || echo "no git"
  echo '

```'
  echo
  echo "## Recreate test"
  echo
  echo "1. Fresh clone at \`$head_sha\`."
  echo "2. Install from lockfiles only. No chat."
  echo "3. Run the project's documented test command."
  echo "4. Fail the session if extra manual steps appear."
} > "$out"

echo "wrote $out"

Then add a tiny gate. I want the receipt in review, not in folklore.

# Label: local check, not CI vendor magic.
test -f SESSION_RECEIPT.md || { echo "missing receipt"; exit 1; }
grep -q 'git_head:' SESSION_RECEIPT.md || { echo "receipt incomplete"; exit 1; }

If the dirty path count is not zero, stop. Commit or discard. Do not "ask the model to remember."

Decision table: ship it, or keep it on the box?

Use this when a teammate says the free server already proved it.

Question	If yes	If no
Can a fresh clone boot from lockfiles?	Eligible to share	Still a demo
Are secrets out of prompts and git?	Keep going	Rotate first
Is the test command written in-repo?	Runnable by others	Oral tradition
Does PATH get documented in a receipt?	Drift is visible	Drift is a ghost
Would you rerun without the original chat?	It is a build	It is a jam session

Score it in two minutes. No dashboard required.

Four yes answers? Promote the branch. Fewer? Keep the box. Do not rename a jam session as a release.

A 20-minute drill I actually run

This is a procedure, not a war story with fake metrics.

1. Open a clean remote session. Clone the repo only.
2. Generate SESSION_RECEIPT.md before any model edit.
3. Let the model change code. Do not let it hide installs in chat.
4. Update lockfiles yourself. Read the diff like a hawk.
5. Re-run the receipt. Diff the two files.
6. Clone again in a second directory. Follow the receipt. No transcript.

Did step 6 fail? The chat still owns you.

What changed between receipts? That delta is your real dependency list.

Limitations, said plainly

This workflow does not measure model quality. It measures whether work can leave the room.

It will not freeze a remote image for you. I did not document vendor snapshot APIs here. Those claims rot.

It will not stop a model from hallucinating a package name. Your lockfile and your registry still matter.

It assumes you can run shell. It assumes git exists. Windows notes: use Git Bash or WSL for the scripts above.

Receipts can lie too. A human can type a fake SHA. Pair the file with git fsck and a second clone.

Who should not use this approach

Skip this if you are doing a five-minute syntax question. You do not need a receipt for a rename.

Skip this if you cannot keep secrets out of prompts. Fix that first.

Skip this if your org forbids unapproved remote execution. A free server is still execution. Policy wins.

Skip this if you need formal provenance for regulated releases. This is a developer checklist. It is not an audit system.

The mental model I want stuck in your head

Free compute is a whiteboard. Whiteboards get erased.

Models are samplers. Samplers are not compilers.

Servers are rooms. Rooms are not backups.

If the work matters, it must survive a cold clone. Cold clone is the only honest demo.

Would you bet a Friday deploy on a chat you cannot find?

I would not. I write the receipt. Then I close the tab on purpose.

If you try the pairing on MonkeyCode, use the free server as the second room in step 6. Keep the receipt in git. That is the whole trick.