DEV Community

Sam Li
Sam Li

Posted on

Close the Log Before the Second Attempt

A job on a remote host printed exit status 1 while deploy.log was still growing. Twelve seconds later an assistant proposed a retry with --force added and the same working directory. The first process had not been reaped. These notes cover a 48-hour lab worksheet built around that shape. They are not a production postmortem. The timings inside the harness are fixture constants the test asserts, not measured customer latency.

The failure is ordinary. Pipes hide status. Assistants prefer a next step. A disposable host is useful here only if you treat it as a clean room for one frozen command, not as a place to keep secrets or to try again because the prompt said the first run looked close.

What the forty-eight hours were for

Hour 0 through hour 6 went to freezing the suggestion. The assistant's prose is not the artifact. The artifact is a directory that holds the exact argv, an environment allowlist, a transcript path, and a refusal to run twice until the first pid is gone and the transcript hash has stopped changing.

That freeze is the step worth repeating whenever a model offers a retry. Pasting a live production log into a prompt is the step worth dropping. The worksheet uses a synthetic log generated on the lab host, with no customer names and no credentials.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. The operator states that MonkeyCode currently offers free model access and a free server option. This worksheet uses those two availability claims only as a way to draft a command and to replay it once on a machine that is not the production host. It does not assume a named model, a token quota, a hardware size, a region, or a lifetime. Read those from the current console before you depend on them. If the console and this paragraph disagree, trust the console.

What broke in the first pass

The first harness trusted set -e and a pipeline.

set -euo pipefail
./migrate.sh --plan | tee deploy.log
echo "plan_ok"
Enter fullscreen mode Exit fullscreen mode

tee can succeed when migrate.sh fails, unless pipefail is on. With pipefail, the pipeline status is the rightmost non-zero status, which is what a reviewer wants, but only if nothing later in the script masks it. The assistant's retry wrapper did this:

./migrate.sh --plan --force | tee -a deploy.log || true
echo "retry_sent"
Enter fullscreen mode Exit fullscreen mode

|| true makes the wrapper exit 0. The log is appended, so the second attempt's lines sit under the first attempt's unfinished tail. A reader who tails the file cannot tell which attempt wrote the last brace. That is the break that should not ship.

A second break showed up when the disposable host's /bin/sh was dash, while the suggestion assumed bash arrays. The same text file was not the same program. The worksheet now records readlink -f /bin/sh and refuses to run if the shebang and the host shell disagree. Hour 6 ended with that refusal firing on purpose. The command had not been wrong about SQL. It had been wrong about which shell would parse it.

The artifact: one run, then a lock

The harness below is the piece worth keeping. It is a lab script, not a product feature. Run it on a disposable host. Point WORK at an empty directory. Do not put credentials in argv.json.

#!/usr/bin/env bash
# freeze-once.sh — lab worksheet, not a production runner
set -euo pipefail

WORK="${WORK:-./freeze-work}"
mkdir -p "$WORK"
ARGV_FILE="$WORK/argv.json"
TRANSCRIPT="$WORK/transcript.log"
LOCK="$WORK/run.lock"
HASH_FILE="$WORK/transcript.sha256"

if [[ -f "$LOCK" ]]; then
  echo "refused: lock present at $LOCK" >&2
  exit 17
fi

if [[ ! -f "$ARGV_FILE" ]]; then
  echo "refused: missing $ARGV_FILE" >&2
  exit 18
fi

mapfile -t ARGV < <(python3 -c 'import json,sys; print("\n".join(json.load(open(sys.argv[1]))))' "$ARGV_FILE")

SHELL_NOW="$(readlink -f /bin/sh || true)"
printf 'shell=%s\nstarted_at=%s\n' "$SHELL_NOW" "$(date -u +%Y-%m-%dT%H:%M:%SZ)" > "$WORK/fingerprint.txt"

TIMEOUT_S="${TIMEOUT_S:-30}"

echo "$$" > "$LOCK"
set +e
timeout --signal=TERM "${TIMEOUT_S}" "${ARGV[@]}" > >(tee "$TRANSCRIPT") 2>&1
status=$?
set -e

python3 - <<'PY' "$TRANSCRIPT" "$HASH_FILE"
import hashlib, sys, time
path, out = sys.argv[1], sys.argv[2]
def snap():
    with open(path, "rb") as f:
        data = f.read()
    return len(data), hashlib.sha256(data).hexdigest()
a = snap()
time.sleep(0.2)
b = snap()
if a != b:
    sys.exit(19)
open(out, "w").write(b[1] + "\n")
PY
stable=$?

if [[ "$stable" -ne 0 ]]; then
  echo "refused: transcript still changing" >&2
  exit 19
fi

printf 'status=%s\nsha256=%s\n' "$status" "$(cat "$HASH_FILE")" >> "$WORK/fingerprint.txt"
echo "recorded status=$status; lock remains"
exit "$status"
Enter fullscreen mode Exit fullscreen mode

A companion check rejects a retry file that only differs by a force flag. It does not try to understand the migration. It only answers whether the second argv is a disguised first argv.

# compare_retry.py — proposal, run against two frozen argv files
import json, sys

def load(path):
    with open(path) as f:
        return json.load(f)

first, second = load(sys.argv[1]), load(sys.argv[2])
added = [x for x in second if x not in first]
removed = [x for x in first if x not in second]
force_like = [x for x in added if x in {"--force", "-f", "--retry", "--yes"}]
if force_like and not removed:
    sys.exit("refused: second argv only adds a force-like flag")
print("argv delta added=", added, "removed=", removed)
Enter fullscreen mode Exit fullscreen mode

Write the first argv by hand, or accept a model draft only after you have saved it:

python3 - <<'PY'
import json
json.dump(["./migrate.sh", "--plan"], open("freeze-work/argv.json", "w"))
PY
WORK=./freeze-work bash freeze-once.sh
python3 compare_retry.py freeze-work/argv.json freeze-work/argv.retry.json
Enter fullscreen mode Exit fullscreen mode

In the worksheet, hour 6 through hour 30 is repetition of one fixture: a migrate.sh that writes 200 lines, sleeps past one sample window, then exits 1. The harness must exit 19 if the stability sleep is shortened to 0 while the writer is still appending. That assertion is the point. Two samples of a growing file are not a closed transcript. Nothing here is a field measurement beyond that fixture.

Hour 30 through hour 48 is the part worth repeating on a disposable host, and only there. Copy argv.json and the two scripts. Do not copy .env. Run once. Read fingerprint.txt. If status is non-zero, stop. If you use a model at all, ask it to explain the transcript. Do not ask it to invent a second argv until a person has deleted the lock on purpose.

Free model access fits that explanation step if you are already using the product named in the disclosure: paste the frozen transcript from the disposable host, not the production log. A free server option fits the single replay if the current offer gives you a host you can wipe. Neither one decides the retry. The lock file does. Capacity, image, and how long the host remains yours are console facts, not article facts.

What not to repeat

Do not let the model edit freeze-once.sh and the command under test in the same turn. That collapses the judge and the subject. Do not treat a green wrapper (|| true) as a successful migrate. Do not store the lock on a laptop and the transcript on the server. They have to live in one WORK directory so a later reader sees one story.

Skip this approach if the change cannot be replayed on a clean host, if production credentials would have to sit inside argv.json, or if you need a durable multi-tenant environment. A free server whose lifetime you do not control is a worksheet bench. It is not a staging tier. If wiping it would destroy the only copy of a result, the setup is already wrong.

The Unix pieces are older than any assistant: timeout from coreutils, pipefail in bash, and SHA-256 over a file that has stopped growing. The GNU timeout manual describes the signal behavior used above. Check the man page on the host you actually run, because a busybox timeout is a different program and may not accept --signal the same way.

Run the fixture locally first. Then, if a free disposable host is actually listed for your account today, replay the same argv.json once and keep the lock. The model can narrate the transcript. It should not receive a second attempt for free.

Top comments (0)