DEV Community

Emery Lin
Emery Lin

Posted on

Keep the Draft Lane From Applying Its Own Patch

A remote green is not merge authority. If a free runner can both write the change and certify it, the required check is theater. Freeze the proposal, apply it on a trusted job, and fail closed when the receipt is missing, cancelled, or quota-stopped.

That is the gate. The rest of this piece is how you wire it so a noisy draft lane cannot bless itself.

The failure you are actually blocking

The badge goes green and the log is long, so the pull request looks finished. You merge. Later the fixture no longer matches the assertion, because the same job edited both.

A second failure mode is quieter. The inner test process exits non-zero, the wrapper still exits zero, and branch protection records success. Cancellation and quota stops get laundered the same way when a skip is treated as a pass.

Generated workflow files make this easier to miss. The diff is large, the job name sounds official, and the first green check arrives before anyone reads the runner label. You do not fix that with a longer prompt. You fix it by splitting lanes.

Where a free draft lane fits

Use the free lane to draft. Do not use it as the merge gate.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. The operator describes MonkeyCode as an open-source project with free model access and a free server option. Those two availability claims are the only product facts used here.

This article does not state a token quota, a hardware size, a time limit, a benchmark, or a promise that the free option stays fixed. Read the current project docs before you depend on either lane.

Point free model access at a patch draft. Point the free server option at a non-required smoke job. You get a cheap place to explore a change. The required job still has to live on a runner you control, with a name you pin in branch protection.

Decide the lanes before you write YAML

Put the rule in a table, then make the hook enforce the table. A comment in the workflow will not stop a later edit.

Lane May write May execute fixtures May satisfy the required check
Draft: free model plus free server proposals/<id>/ only short smoke, non-required no
Trusted CI runner apply the frozen diff full fixture set, once yes, only when the apply receipt matches
Laptop pre-push hook nothing, unless you are the author same fixture digest, local only no; it only blocks your push

If one job can edit workflow files and also set the required check name, stop. You have collapsed the table into a single lane, and a single lane cannot police itself.

Receipt contract

The draft lane writes proposals/<id>/receipt.json next to change.diff. The trusted job writes artifacts/apply-receipt.json after it applies that diff and reruns fixtures. Your merge hook compares the overlapping fields. It does not compare log tone.

Require all of these:

  • proposal_digest: sha256: plus the digest of change.diff
  • fixture_digest: digest of the fixture tree recorded before apply
  • runner_class: draft on the proposal, trusted on the apply artifact
  • exit_class: passed, failed, cancelled, or quota_stopped
  • inner_exit: exit code of the test process, not the wrapper
  • draft_attempts: integer, used only to stop noisy draft reruns

A missing field is a failure. An absent file is a failure. Do not default either one to pass.

Do not skip the trusted fixture suite just because the draft receipt says passed. That field means the draft lane thinks it finished. It is not merge evidence.

Numbered path from draft to merge

  1. Freeze fixtures first. Compute fixture_digest in the workflow environment, and do not let the draft patch rewrite that value.
  2. Restrict the draft lane to proposals/<id>/. If the diff touches any other path, the guard exits non-zero and you do not apply.
  3. Run smoke tests on the free server job. Leave that job off branch protection. continue-on-error is acceptable only because this job is not the merge gate.
  4. On the trusted runner, recompute the patch digest. Refuse apply when it differs from proposal_digest.
  5. Confirm the worktree fixture digest still equals the frozen value. Then apply the diff and run the fixture suite once.
  6. Write the apply receipt with runner_class set to trusted. Set exit_class to passed only when inner_exit is 0.
  7. Name the required check after that trusted job. Keep the draft job name off the protection rule, even if the draft name looks cleaner in the UI.
  8. If exit_class is cancelled or quota_stopped, leave the proposal unapplied. Do not convert that status into a skipped success.

Guard script

The following script is an unexecuted example. Review it before you add it to a repository. It is not a measured benchmark, and it was not run against a live server for this article.

#!/usr/bin/env python3
"""Refuse a draft proposal that is incomplete or out of bounds."""
import hashlib
import json
import sys
from pathlib import Path

REQUIRED = (
    "proposal_digest",
    "fixture_digest",
    "runner_class",
    "exit_class",
    "inner_exit",
    "draft_attempts",
)
BLOCKED = {"cancelled", "quota_stopped", "failed"}

def added_paths(text):
    paths = []
    for line in text.splitlines():
        if line.startswith("+++ b/"):
            path = line[6:]
            if path != "/dev/null":
                paths.append(path)
    return paths

def main():
    proposal = Path(sys.argv[1])
    receipt = json.loads((proposal / "receipt.json").read_text())
    patch = (proposal / "change.diff").read_bytes()
    missing = [key for key in REQUIRED if key not in receipt]
    if missing:
        print("missing fields: " + ", ".join(missing), file=sys.stderr)
        return 2
    if receipt["runner_class"] != "draft":
        print("runner_class must be draft", file=sys.stderr)
        return 2
    if receipt["exit_class"] in BLOCKED:
        print("refuse apply: " + receipt["exit_class"], file=sys.stderr)
        return 3
    if int(receipt["draft_attempts"]) > 2:
        print("draft attempts exceeded; stop and review", file=sys.stderr)
        return 3
    digest = "sha256:" + hashlib.sha256(patch).hexdigest()
    if receipt["proposal_digest"] != digest:
        print("proposal_digest does not match change.diff", file=sys.stderr)
        return 4
    bad = [p for p in added_paths(patch.decode()) if not p.startswith("proposals/")]
    if bad:
        print("patch leaves proposals/: " + ", ".join(bad), file=sys.stderr)
        return 5
    print("draft proposal is eligible for trusted apply")
    return 0

if __name__ == "__main__":
    raise SystemExit(main())
Enter fullscreen mode Exit fullscreen mode

Call the guard from the draft job, then again at the start of the trusted job. These commands are part of the same unexecuted example.

python3 scripts/check_proposal.py "proposals/${PROPOSAL_ID}"
sha256sum "proposals/${PROPOSAL_ID}/change.diff"
Enter fullscreen mode Exit fullscreen mode

After trusted apply, assert the artifact. set -euo pipefail matters here. A missing file must fail the job, not print a warning and continue.

set -euo pipefail
test "$(jq -r .runner_class artifacts/apply-receipt.json)" = "trusted"
test "$(jq -r .exit_class artifacts/apply-receipt.json)" = "passed"
test "$(jq -r .inner_exit artifacts/apply-receipt.json)" = "0"
test "$(jq -r .fixture_digest artifacts/apply-receipt.json)" = "${FIXTURE_DIGEST}"
Enter fullscreen mode Exit fullscreen mode

Wire the jobs so the required name is the trusted job. if: always() is deliberate. You want the trusted job to run and fail when the draft receipt is blocked. A skipped required check is how a cancelled free-server run sneaks toward merge.

# Unexecuted example. Job names are illustrative.
jobs:
  draft_on_free_server:
    continue-on-error: true
    steps:
      - run: python3 scripts/check_proposal.py "proposals/${PROPOSAL_ID}"
  trusted_apply:
    needs: draft_on_free_server
    if: always()
    steps:
      - run: python3 scripts/trusted_apply.py "proposals/${PROPOSAL_ID}"
      - run: bash scripts/assert_apply_receipt.sh
Enter fullscreen mode Exit fullscreen mode

Do not gate trusted_apply on needs.draft_on_free_server.result == 'success' alone. A wrapper can report success after a quota stop. The receipt check, plus a fresh fixture run, is the control. The draft result is only a hint.

Flake control that cannot borrow a draft green

You may rerun the draft lane when smoke is noisy. Record the count in draft_attempts. Above 2, stop and ask a person to look at the proposal.

Those reruns never satisfy the required check. The trusted job runs the fixture suite once per apply receipt. If you need another trusted run, write a new receipt with a new digest comparison. Do not reuse an older green artifact from the draft lane.

This is narrower than a repository-wide flake budget. You are blocking one specific wash: a red inner exit that later becomes a zero from a wrapper on a free server.

Debug a merge that should not have happened

When a pull request lands and the fixture looks wrong, reconstruct the lane before you rerun anything.

  1. Read the required check name on the merge commit. If it equals the free-server job, the gate was never split.
  2. Download artifacts/apply-receipt.json. If the artifact is absent, the job skipped or never started. Treat that as a failed gate, not as an empty pass.
  3. Hash the diff that landed and compare it with proposal_digest. A mismatch means apply did not use the frozen proposal.
  4. Recompute fixture_digest at the parent commit. Drift means something moved the fixture between freeze and apply.
  5. Read inner_exit and any wrapper status separately. Keep inner_exit. Discard the wrapper.

Use a stable file order when you recompute the fixture digest. Directory iteration order is not a contract.

# Unexecuted illustration of a stable fixture digest.
find fixtures -type f -print0 | sort -z | xargs -0 sha256sum | sha256sum
Enter fullscreen mode Exit fullscreen mode

If step 1 already shows the draft job as the required check, stop digging through model logs. Fix the protection rule first. The rest of the receipt work will not help while the wrong job name can still go green.

Limitations

Matching digests does not prove the patch is good. It proves the trusted job applied the same bytes, against the fixture digest you froze, and recorded the inner exit you checked.

A compromised trusted runner can forge the apply receipt. Treat that runner like a release credential. Do not place it on the same account or the same machine as the free server lane.

The +++ b/ scan misses some rename and copy headers. Extend the parser before you treat this guard as a security boundary. Schema-check fixture files after apply as well. A draft model can still invent a field inside a path you allowed.

Free-server availability is not something this article can promise. Queue loss, image drift, and quota stops belong on the exit_class list. None of them is a pass, and none of them should delete the proposal you still need to inspect.

Who should not use this flow

Do not use it if you have only one runner and you plan to label it both draft and trusted. The split is the control. One machine wearing both labels removes the control.

Do not auto-apply patches that edit workflow files, branch protection, or signing configuration. Those edits need a human review path with a different required check.

Do not treat a proposal receipt as a release signature. It is a CI comparison. It is not a key ceremony, and it does not replace code review.

Try the split on one non-production branch

If a free model lane and a free server option are already available to you, aim them at proposals/ only. Confirm current access terms in the project docs, add the receipt asserts, and only then consider naming trusted_apply as the required check.

Top comments (0)