Stamp the Verdict Card on the First Run

A composite checkout failed on Friday and passed on the retry. The invoice property missed the total by one cent. The second run saw a newer wall clock.

This note reconstructs that failure mode for review. This note is not a personal incident log. No customer metric is claimed in this draft.

Agent patches move faster than most review notes. Test oracles drift when the same patch may edit them. A quiet retry can hide the first miss.

A retry can help a dead socket recover. It does not repair a moved test oracle. Those two failures need two different kinds of gates.

The useful split stays small, strict, and ordered. Seal the fixture before any property is allowed to speak. Run the suite once, then freeze every flake.

That order matters more than another green badge. A second run is always a different experiment. It must not erase the first recorded miss.

The fixture is a sealed box

Think of the fixture as a packed lunch for the trip. You weigh the box before you leave the kitchen. You do not swap the apple after the scale.

A sealed fixture carries three locks on purpose. Input bytes stay fixed for the whole run. The clock stays fixed, and the network stays closed.

Hash the input tree before tests start on the patch. Store that digest beside the proposed agent change. Refuse the run when the digest drifts at all.

find fixtures/invoices -type f -print0 \
  | sort -z \
  | xargs -0 sha256sum \
  | sha256sum \
  | awk '{print $1}' > .fixture-seal

The command above is a local recipe for a clean worktree. Skip editor swap files and any local cache files. Commit the seal only when the inputs truly change.

# proposal: not an executed benchmark
import hashlib
from pathlib import Path

def seal_tree(root: Path) -> str:
    digest = hashlib.sha256()
    files = sorted(p for p in root.rglob("*") if p.is_file())
    for path in files:
        rel = path.relative_to(root).as_posix().encode()
        digest.update(rel)
        digest.update(b"\0")
        digest.update(path.read_bytes())
        digest.update(b"\0")
    return digest.hexdigest()

Compare this digest with the verdict card later. A mismatch means someone opened the sealed box. Opened boxes do not cast a merge vote.

Label the clock inside the fixture file too. A file named clock.json can hold one frozen instant. Tests read that instant, not the host time.

{"frozen_at": "2026-03-01T00:00:00Z", "zone": "UTC"}

The timestamp is sample data for the recipe. It is not a report of any live outage. Swap it for the instant your fixture owner chose.

Properties speak a single time

A property check is a pure question over bytes. The same fixture should yield the same answer. Wall time and network stay outside the function.

Here is a small invoice property for totals. Line amounts sum to a net figure first. Tax uses the rate stored in the fixture file.

# proposal: illustrative property, not a production suite
from decimal import Decimal, ROUND_HALF_EVEN

def line_total(lines, tax_rate: str) -> Decimal:
    net = sum((Decimal(row["amount"]) for row in lines), Decimal("0"))
    rate = Decimal(tax_rate)
    tax = (net * rate).quantize(Decimal("0.01"), rounding=ROUND_HALF_EVEN)
    return net + tax

def property_invoice_total(fixture: dict) -> None:
    got = line_total(fixture["lines"], fixture["tax_rate"])
    expect = Decimal(fixture["expected_total"])
    if got != expect:
        raise AssertionError(f"total drift {got} != {expect}")

Run that function against the sealed invoice file. Do not let the same patch rewrite expected_total. Oracle edits still need a later human review.

python -m agent_gate.run \
  --fixture fixtures/invoices \
  --seal .fixture-seal \
  --retries 0 \
  --card artifacts/verdict-card.json

The zero-retry flag is the whole point of this gate. A red property stays red until someone explains it. You do not buy a greener clock with a second attempt.

If the module path does not exist yet, treat the command as a contract. Your runner must honor the same four flags. The names matter less than the refusal to retry.

Flakes freeze instead of voting

A flake is a test that changed color without a code change. It behaves like bad weather over the suite. That weather is not evidence for a merge.

When the single run marks a test flaky, write a freeze file. Frozen tests stay out of the merge vote. They also stay off the retry path for this patch.

{
  "freeze_id": "invoice-clock-sample",
  "tests": ["test_invoice_total_wall_clock"],
  "reason": "color changed across identical fixture bytes",
  "retry_allowed": false
}

Replace the sample id with your own run id. The reason string should name the fixture seal. A freeze without a seal is just a shrug.

The freeze file is a lease on silence, not a pass. A frozen test is debt on the board. Someone must fix the oracle or delete the test.

Do not auto-retry a frozen name in the same patch. A retry would launder red into green over time. That laundering is how the Friday miss survived review.

Keep the freeze set stable across the vote. A patch that adds a freeze and a feature together needs two commits. The first commit only records the testing debt.

The next commit may change the product code. Reviewers can then see the debt apart from the feature. Mixed commits hide which change actually earned the green.

The verdict card binds the run

The card binds four facts from the single run. It stores the fixture hash and the property rows. It stores the freeze set and the retry count.

# proposal: gate sketch, unexecuted
def accept(card: dict, expected_seal: str) -> bool:
    if card["retry_count"] != 0:
        return False
    if card["fixture_hash"] != expected_seal:
        return False
    if card["new_oracle_edits"]:
        return False
    frozen = set(card["flake_freeze"])
    for row in card["properties"]:
        if row["name"] in frozen:
            continue
        if row["status"] != "pass":
            return False
    return True

Read that function as policy, not as a benchmark. This draft does not report timings or pass rates. No suite size is claimed for any real repository.

A pass requires every voting property to pass cleanly. Frozen names are skipped, not counted as green. Retry count must stay zero, and the seal must match.

If the agent edited an expected file, mark oracle edits true. The card then rejects the patch without debate. A human must bless the new oracle in a later change.

That rule stops a quiet cheat during review. The model cannot move the goalposts and then grade itself. The expected file is evidence, not a scratch pad.

Store the card under artifacts/ and keep it with the patch. Reviewers should diff the card, not only the source. A missing card is the same as a failed gate.

python -m agent_gate.accept \
  --card artifacts/verdict-card.json \
  --seal .fixture-seal

Treat a non-zero exit as a blocked merge. Fill the seal from the seal file on the same host. Do not hand-copy a hash from a different tree.

A hand copy is how seals start to lie. The accept command should read both files itself. Humans compare the diff, not the raw digest.

Where a clean runner fits

You can draft these properties on a daily laptop. You still want a clean machine for the sealed run. Laptops leak clocks, agents, and half-saved files into the tree.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The operator describes MonkeyCode as an open-source project for this workflow. The supplied notes mention free model access and a free server option. This draft does not state quotas, hardware, duration, or permanence.

Use the free model access to sketch property stubs from fixture shapes. Keep secrets and customer rows out of the prompt. Paste field names and ranges, not live invoices.

Use the free server option for the one sealed run. The point is a clean clock and a closed network. It is a convenience, not a permanent budget for CI.

If that offer changes, the gate still stands on its own. Run the same card on any clean host you already trust. The product does not replace the hash or the freeze file.

One practical next step stays narrow on purpose. Sketch the property stubs with that free access. Then commit the verdict card from a sealed run you can rerun.

What this gate will not save

Skip this gate when tests must call a live partner. A closed network will fail those checks every time. That suite needs a contract test, not a sealed box.

Skip it when nobody owns the fixture format. A seal without an owner becomes stale folklore fast. Stale seals block good patches and wave bad ones through.

Skip it as a substitute for security review. A green card says the named properties held once. It does not say the tool surface stayed narrow.

Oracle changes still need a person in the loop. The card can only refuse to vote on them. It cannot decide the new business truth by itself.

Free model output can be wrong in quiet ways. Treat every model draft as an untrusted patch. The sealed run is the check, not the model's confidence line.

Do not widen fixtures to make a red property pass. That move is an oracle edit with extra steps. The card should see it and return false.

A short operating rhythm

Refresh the seal in the morning only when inputs change. Commit the new digest in the same change as the fixture. Leave the seal untouched for a code-only patch.

For each agent patch, run once with retries forced off. Write the verdict card next to the diff. If a voting property fails, stop and inspect the box.

If a test flickers, add it to the freeze file now. Leave it out of the vote for this merge. File a repair task that names the run id and the seal.

Merge only when accept returns true on the card. Revert if production later disagrees with that card. The card is a gate, not a pardon for the release.

This rhythm is slower than a blind retry loop. It is cheaper than a Monday ledger chase after a false green. One sealed miss is a fact you can still read.