Do Not Let the Explain Job Authorize the Retry

A model can explain a red job. It must not authorize the retry, and it must not mint the merge. You freeze a local decision record before any assistant runs, and you point branch protection at that record alone.

This is a practical split you can add beside checks you already trust. The scripts are a proposed example. They were not executed for this draft. Run them on a branch, read the diff, and only then consider requiring the new check.

Stop treating a summary as an exit code

Logs are easier to summarize than they were a year ago. Your merge rule did not get stricter by itself. A paragraph that says "this looks flaky" invites a rerun. The rerun sometimes goes green for a reason the paragraph never named, and the paragraph becomes the story the team remembers.

You break that habit by refusing to parse prose in the gate. The classifier sees an exit class, a fixture digest, and a remaining flake budget. The explain job sees a redacted log. Those readers do not share a pen.

Give each artifact one writer

You want three files, and you want three different write permissions.

ci/decision.json is the merge evidence. The trusted test job writes it once per attempt, then the file becomes read-only.

ci/notes/failure.md is a comment. An explain job may create it. No required check scores its wording.

ci/retry-ledger.jsonl is an append-only list of rerun reasons. The allowed reasons are a short local vocabulary. "The note sounded flaky" is not in that vocabulary.

If one token can write all three, you do not have a gate. You have a notebook with permission to go green. Split the tokens before you split the workflow file.

Seal the record before any network call

Run the classifier on the trusted runner, immediately after the suite, with no model call and no socket. Exit code 75 is a convention you define in the harness for a known flake signature. It is not a platform default. If that signature does not exist in code, set retry_allowed to false and leave it there.

GITHUB_SHA below is the GitHub Actions name for the commit under test. Map it to your forge's own SHA variable. The rule is the seal, not the vendor.

#!/usr/bin/env bash
set -euo pipefail
# Proposed post-test hook. Unexecuted. No network.
rec=ci/decision.json
digest="$(sha256sum fixtures/manifest.json | awk '{print $1}')"
budget="$(jq -r '.flake_budget' policy/ci.json)"
used="$(wc -l < ci/retry-ledger.jsonl || true)"
used="${used:-0}"
remain=$((budget - used))
class="fail"
retry=false
if [[ "${TEST_EXIT}" == "0" ]]; then
  class="pass"
elif [[ "${TEST_EXIT}" == "75" && "${remain}" -gt 0 ]]; then
  class="flake_candidate"
  retry=true
fi
umask 022
jq -n \
  --arg head "${GITHUB_SHA}" \
  --arg digest "${digest}" \
  --arg class "${class}" \
  --argjson remain "${remain}" \
  --argjson retry "${retry}" \
  '{schema:1,job:"trusted-test",head:$head,fixture_sha256:$digest,exit_class:$class,flake_budget_remaining:$remain,retry_allowed:$retry,note_status:"absent"}' \
  > "${rec}"
sha256sum "${rec}" > ci/decision.json.sha256
chmod a-w "${rec}" ci/decision.json.sha256

note_status starts as absent and is not a slot for the model to fill in later. A rewrite of exit_class or retry_allowed must fail the hash check. You are freezing a small tuple, not collecting a richer story.

Keep policy/ci.json in git, with flake_budget as a plain integer. Review a budget change the way you review a production flag. A budget edited in the same pull request that is currently red is a smell. It is not a fix.

Walk the red job in five steps

1. Run the suite on the trusted runner. Save the raw exit code. Do not hide a retry inside the test process, and do not let a wrapper convert a timeout into success.
2. Run the hook above. It writes and freezes ci/decision.json for this attempt. If the hook fails, the attempt fails. You do not continue to an explain job without a seal.
3. If retry_allowed is true, append one ledger line with reason=local_flake_signature, then rerun once under a new attempt id. Write a new sealed record. Do not edit the first file in place.
4. Start the explain job only after a terminal failure. It may draft ci/notes/failure.md. It must not call the status API, and it must not push to the branch.
5. Require a pinned check that runs the verifier in the next section. Do not require the explain job. A missing note leaves the merge decision unchanged.

The ledger line is supposed to look boring. One reason, one attempt, no prose:

printf '%s attempt=%s reason=local_flake_signature head=%s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" "${ATTEMPT}" "${GITHUB_SHA}" >> ci/retry-ledger.jsonl

A second automatic retry means you do not have a budget. You have a loop. If you want another attempt, a person changes the policy in git and reviews that change in the open.

Host the note where it cannot set status

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The explain job is the only step where a hosted assistant belongs. MonkeyCode's free model access and free server option can host that optional job when you want a drafted note and you do not want to stand up another build machine. Treat both as current product terms you must read, not as facts this article measured. No token quota, machine size, time limit, or permanence is claimed here. If the docs you can cite disagree with the button you clicked, skip the remote job until they match.

Put a read-only redacted log on that runner. Limit its write path to ci/notes/. Do not mount the decision record read-write. Do not inject a credential that can create check runs. A free server is still someone else's CPU. You would not paste production secrets into a chat box. Do not upload them in a log either.

Stamp the note before any prose:

---
role: advisory
writer: explain-job
may_affect_status: false
---

Redact first. Remove Authorization headers, echoed environment files, signed URLs, and customer identifiers. If you cannot redact with confidence, write a local note that says the log was withheld, and do not call the model. The gate does not need a guess. It already has an exit class.

When the explain call times out, returns empty text, or never starts, you leave the sealed record alone. A missing comment is not a missing test. People get this backwards when a red X from the note job sits next to a real failure. Fix the job so it cannot publish a commit status. Do not "fix" the X by teaching reviewers to ignore red.

Verify the seal and ignore the tone

This verifier is the required check. It is a proposed, unexecuted script. It does not grade sentences. It assumes a Unix mode bit, which matches the chmod line above.

#!/usr/bin/env python3
"""Merge gate. Reads the sealed record. Does not score note prose."""
import hashlib, json, os, sys
from pathlib import Path

rec = Path("ci/decision.json")
seal = Path("ci/decision.json.sha256").read_text().split()[0]
body = rec.read_bytes()
if hashlib.sha256(body).hexdigest() != seal:
    sys.exit("decision record hash mismatch")
if rec.stat().st_mode & 0o222:
    sys.exit("decision record still has a write bit")
data = json.loads(body)
if data.get("schema") != 1:
    sys.exit("unexpected schema")
expected = os.environ.get("EXPECTED_SHA", "")
if not expected or data.get("head") != expected:
    sys.exit("head mismatch")
if data.get("job") != "trusted-test":
    sys.exit("unexpected writer")
if data.get("exit_class") != "pass":
    sys.exit("not green: %s" % data.get("exit_class"))
if data.get("retry_allowed") is True:
    sys.exit("final record still allows retry")
note = Path("ci/notes/failure.md")
if note.exists():
    text = note.read_text(encoding="utf-8")
    if "may_affect_status: false" not in text or "role: advisory" not in text:
        sys.exit("note missing advisory stamp")
print("merge-gate ok")

Give this check a stable name and protect that name. Do not let the explain workflow reuse it. The success line is there so you can grep the log. The script does not print the note. Once the note is in the required log, someone will screenshot it into the merge thread and treat the screenshot as evidence.

An empty EXPECTED_SHA fails closed. A verifier on the wrong checkout should not pass. Set that variable from the same commit the trusted job sealed.

Use a table when you want to rerun

Local signal	Note	Retry	Merge
exit 0, digest matches, record sealed	ignored	no	only through this required check
exit 75, budget left, first attempt	optional, after the seal	one local retry	no, until a new sealed pass exists
exit 75, budget already spent	optional	no	no
exit 1, unknown failure	optional, advisory	no	no
note says flake, exit is 1	present	no	no
explain job down or empty	absent	unchanged	unchanged
hash mismatch, or record still writable	any	no	no

The sixth row is the one to rehearse. Model downtime must not block a sealed pass. It also must not turn a fail into a pass. If your forge fails the whole commit whenever any job fails, the explain job is publishing a status it should not own. Narrow the token. Do not widen the retry rule to compensate.

The last row is the corruption case. A writable record, or a hash that does not match, means you stop. You do not ask the model whether the mismatch looks harmless. Harmless-looking mismatches are how a second writer learns it can edit the gate.

Limits, and who should skip this

This workflow does not rank models, and it does not show that a free server is faster or cheaper than a runner you already operate. It does not replace a fixture manifest check, a clean worktree assertion after tests, or a lock on replay artifacts. Those answer different questions. If you lack them, add them. Do not trade them for a clearer paragraph.

You should not use this approach when branch protection is off, when every green check is treated as equal, or when "flake" is a feeling instead of an exit code in the harness. You should not use the remote explain lane for release signing or for a production deploy. A diagnostic box is not a promotion environment. Skip the remote call entirely if you cannot deny status scope on that job's token, or if the log holds data you would refuse to paste into a third-party prompt.

Turn the explain job on only after the verifier already passes with no note file present. Until that rehearsal is done, a model in the pipeline is extra surface, not extra certainty. If the rehearsal holds and you still want a drafted note without another machine of your own, MonkeyCode's free model access and free server option are enough to try the explain job. Stop if that job can write the check your branch protection requires, or if the current docs no longer match the workflow you merged.