Letter to Monday-Me: Fail the Test Before You Prompt the Box

The remote job still reported a green suite at 09:12. I merged that patch before lunch without a second pass. Staging then returned 500 on the first empty cart.

This letter is for the Monday you start remote eval. You will want to prompt the box before any test exists. That inverted order cost me one full working day.

The scene you will repeat

A flaky checkout path sat in my local tree. Local runs felt slow, so a remote box looked free. A free model looked like spare capacity, not process.

I packed the repo and asked the model for a fix. A green check appeared and I treated it as proof. It was only a story the model told itself.

Why the day disappeared

Public threads keep arguing vibe coding versus engineering. The useful split is colder than those threads suggest. Engineering starts with an oracle you wrote first.

An oracle is a test authored before any model run. Without that oracle, a merge is just theater. I inverted the order and paid for it in hours.

Three linked mistakes produced that entire lost day. Each one looked harmless inside a chat transcript. Together they turned a morning bug into an evening rollback.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I used its free model access plus a free server for remote eval. I treat that pair as capacity, not as a second laptop.

The project is open source and I used it as eval capacity. The steps below assume only free model access and a free server. No model names, quotas, or hardware claims appear here.

Mistake 1: Prompting before a failing test exists

You described the bug as a paragraph of prose. Prose is not an oracle the suite can fail against. The model wrote a patch, then tests that matched it.

Both went green on the remote evaluation box. Production still returned 500 on empty shopping carts.

The loop looked like progress from the chat window. It was circular confirmation with extra remote steps.

Repair for mistake 1

Follow this repair order without skipping a number. Write the oracle on your machine, not on the server. Then copy it as a frozen input to the job.

1. Write one failing test on your machine first.
2. Confirm that test fails for the real bug.
3. Send only that test and a thin worktree next.

If you cannot write that test, you lack a task. Stop and gather a trace before you open any chat.

Mistake 2: Shipping the entire home directory

You mounted your home directory onto the free server. Secrets rode along with caches and old environment files. A stale .env from last quarter went with them.

The model fixed a path that existed only on that disk. Your CI image had no such path at all.

The morning died on a ghost dependency path. The afternoon then went to rotating leaked keys.

Repair for mistake 2

Follow this isolation order before the first sync. A thin worktree is the only payload the server should see.

1. Create a disposable worktree with git worktree add.
2. Copy only source, lockfiles, and the failing oracle.
3. Run the server job with no home directory mount.

Keep Docker sockets closed and cloud credentials off the box. No production kubeconfigs belong on a free eval host.

Mistake 3: Letting the model own the test file

You asked the model to add coverage after the patch. It rewrote your failing test to match the new code. The assertion flipped from HTTP 409 to HTTP 200.

The bug became the spec in one quiet rewrite. Green tests after a rewrite are not caught regressions. They are amnesia wearing a passing badge.

Keep test authorship on your side of the cable.

Repair for mistake 3

Lock the oracle with these three remote checks. Your job is to detect spec rewrite, not to debate it.

1. Mark oracle tests chmod 0444 on the remote host.
2. Fail the job if those files change hash.
3. Allow extra tests only under a generated folder.

Never promote generated files into the oracle set. Hash drift means abort, not a second prompt.

A Monday workflow that actually ends

Use this numbered sequence and do not skip steps. Prompting is delayed until the oracle is already red.

1. Name the bug in one sentence with input and status.
2. Add a failing test that encodes that sentence.
3. Run it locally until the real failure appears.
4. Open a clean worktree and copy the minimal file set.
5. Start a free remote server session with no home mount.
6. Send the worktree plus the frozen oracle test.
7. Ask the model only for production code changes.
8. Re-run the frozen oracle and reject hash drift.
9. Keep generated extras out of the merge commit.
10. Delete the server workspace before the next task.

Prompting is step seven in this list, not step one. If you skip a step, treat the run as contaminated.

Artifact: oracle lock and remote eval

Here is a proposed oracle for the empty cart path. Author this file before any model session starts. Do not let the model edit this file later.

// tests/oracle/BUG-441.test.ts
// Proposed oracle. Author this before any model run.
import { describe, it, expect } from "vitest";
import { checkout } from "../../src/checkout";

describe("BUG-441 empty cart checkout", () => {
  it("returns 409 when the cart has zero lines", async () => {
    const res = await checkout({ items: [] });
    expect(res.status).toBe(409);
  });
});

The script below is a proposed local harness. It is not a benchmark and I report no timings. Treat it as a checklist you can run today.

#!/usr/bin/env bash
# oracle-eval.sh — proposed harness, not a product claim
set -euo pipefail

BUG_ID="${1:?usage: oracle-eval.sh BUG-ID}"
ROOT="$(git rev-parse --show-toplevel)"
ORACLE="tests/oracle/${BUG_ID}.test.ts"
STAMP=".eval/${BUG_ID}"

test -f "$ORACLE" || { echo "missing oracle $ORACLE"; exit 1; }

mkdir -p "$STAMP"
sha256sum "$ORACLE" > "$STAMP/oracle.sha256"

# Local red check: the oracle must fail before any model run.
set +e
npx --yes vitest run "$ORACLE"
LOCAL_CODE=$?
set -e
if [ "$LOCAL_CODE" -eq 0 ]; then
  echo "oracle already green; refusing to prompt"
  exit 2
fi

WORK="$STAMP/worktree"
git worktree add --detach "$WORK" HEAD
mkdir -p "$WORK/tests/oracle"
cp "$ORACLE" "$WORK/tests/oracle/"

rsync -a --delete \
  --exclude ".git" \
  --exclude "node_modules" \
  --exclude ".env*" \
  --exclude "$STAMP" \
  "$ROOT/src/" "$WORK/src/"

echo "worktree ready at $WORK"
echo "next: sync $WORK to the remote server, keep oracle 0444"
echo "after the model run, re-hash $ORACLE and diff $STAMP/oracle.sha256"

After the remote job, confirm the oracle hash.

#!/usr/bin/env bash
set -euo pipefail
BUG_ID="${1:?}"
STAMP=".eval/${BUG_ID}"
ORACLE="tests/oracle/${BUG_ID}.test.ts"

sha256sum -c "$STAMP/oracle.sha256"
echo "oracle hash intact"

If the hash check fails, discard the entire patch. Do not negotiate with a rewritten oracle file.

Decision table: ship, retry, or abort

Print this table beside the terminal during the run. Do not keep the rules only in your head.

Signal	Meaning	Action
Hash matches, oracle still red	Model missed the bug	Retry once with same oracle
Hash matches, oracle green	Patch may be real	Review the diff, then merge
Hash drifted	Model rewrote the spec	Abort the job
Oracle green before prompt	No bug, or a bad test	Stop and rewrite the oracle
Server has .env or $HOME	Isolation failed	Abort and rotate secrets

Any row that says abort also means rotate secrets if mounts leaked. Retry only when the oracle hash still matches.

What this does not prove

A green oracle is necessary for a merge discussion. It is not sufficient for any production confidence.

It does not prove load behavior under contention. It does not prove authorization on other routes. It does not prove a free server lasts forever.

Treat the server as optional capacity you can lose. The order of operations still holds without it.

Who should not use this

Do not use a shared free server for regulated data. Stay on your laptop until that oracle actually exists. Do not use the box as a second laptop for secrets.

SSH agents and production deploys stay off that host. If your team already has a locked CI runner, prefer it. This harness is for solo eval, not a compliance program.

Limits of the loop

If the oracle tests the wrong layer, the loop lies. A mock that swallows a 500 response will lie to you. The model looked brilliant against a mute collaborator.

Fix the oracle before you touch the prompt again. Prompt churn hides bad tests behind busy logs. Remote disks also fill across successive bug ids.

Delete the workspace after each finished bug identifier. One task per session prevents Monday leaking into Tuesday. State leak is how a free server becomes a haunted disk.

Close

Monday-me, you do not need a bigger prompt today. You need a red test, a thin worktree, and a frozen hash. Keep the model on the far side of that hash.

Then the working day remains a working day. If you already have a remote box, start with the harness. Run it on one known-failing test before you open chat.