Tag Agent Patch Tests by Clock Class Before They Touch a Shared Runner

A green local run is not evidence that an agent patch is safe on a shared runner. The missing artifact is a clock class on every test the agent added or touched. Without that label, retries hide load, DNS jitter, and filesystem delays that the patch did not actually fix.

This article proposes a merge gate, not a vibe. Classify tests as logical, wall, or mixed. Run property checks only under a logical clock and a hard wall budget. Freeze wall and mixed cases off the merge set when the runner is contended. The workflow below is a labeled example. It is not a report of a production campaign.

Why the local clock lies

Agent patches often add tests that read time.time(), sleep, or wait on a network round trip. Those tests pass on a quiet laptop. They then flake when the same suite shares a CPU with other jobs.

Retries do not convert a wall-clock assertion into a specification. They convert load into a coin flip. A shared runner is hostile to that coin. Treat it that way on purpose.

Three failure modes show up repeatedly in agent-authored suites:

1. Duration assertions. assert elapsed < 0.05 encodes the laptop, not the function.
2. Unseeded sleeps. time.sleep(0.01) to “wait for flush” couples the test to scheduler noise.
3. Live I/O as a fixture. DNS, package indexes, and metadata endpoints move while the patch stays still.

Property checks make this worse if they are unbounded. A campaign that “finds no counterexample” after 40 noisy seconds is not a proof. It is a timeout with extra steps.

Clock classes, not retry counts

Give every test the agent touches one of four labels. Store the labels in a file the patch cannot rewrite without a human diff.

Clock class	Allowed on a shared runner	Allowed in merge gate	Typical signals
logical	Yes	Yes	Fake clock, pure functions, hashed fixtures
wall	No	No	time.time, sleeps, duration thresholds
mixed	No	No	Logical core plus one live call
undeclared	No	No	Agent added a test and left the map empty

logical is the only class that may vote on merge. wall and mixed may live in a nightly job on dedicated hardware. undeclared fails closed. That is the entire policy.

The inventory is data. Keep it small enough to review in one screen.

# tests/clock_map.yaml  (example inventory, not a measured suite)
version: 1
rules:
  undeclared: fail_closed
  shared_runner_allows: [logical]
cases:
  - id: parse_invoice_v1
    path: tests/test_invoice.py::test_parse_stable
    clock: logical
    fixture_hash: sha256:4b1c0e9a2f77...
    property: true
    wall_budget_ms: 1500
  - id: flush_buffer_timing
    path: tests/test_io.py::test_flush_under_50ms
    clock: wall
    fixture_hash: null
    property: false
    wall_budget_ms: 0
  - id: retry_then_parse
    path: tests/test_io.py::test_retry_then_parse
    clock: mixed
    fixture_hash: sha256:91aa...
    property: false
    wall_budget_ms: 0

A hash belongs on fixtures that the property check consumes. The hash covers canonical bytes. It does not cover timestamps, PIDs, or runner hostnames.

Artifact: a campaign runner that refuses the wrong clock

The following Python is a proposal you can run locally. Label it as such in review. It does not claim timings from a particular fleet.

# campaign.py — example gate, not a benchmark harness
from __future__ import annotations

import hashlib
import json
import os
import subprocess
import sys
import time
from pathlib import Path

import yaml

ALLOWED_ON_SHARED = {"logical"}


def canonical_bytes(path: Path) -> bytes:
    raw = path.read_bytes()
    try:
        payload = json.loads(raw.decode("utf-8"))
    except (UnicodeDecodeError, json.JSONDecodeError):
        return raw
    def strip(node):
        if isinstance(node, dict):
            return {
                k: strip(v)
                for k, v in node.items()
                if k not in {"ts", "timestamp", "host", "pid", "elapsed_ms"}
            }
        if isinstance(node, list):
            return [strip(x) for x in node]
        return node
    return json.dumps(strip(payload), sort_keys=True, separators=(",", ":")).encode()


def fixture_sha(path: Path | None) -> str | None:
    if path is None or not path.exists():
        return None
    return "sha256:" + hashlib.sha256(canonical_bytes(path)).hexdigest()


def load_map(path: Path) -> dict:
    data = yaml.safe_load(path.read_text())
    if not isinstance(data, dict) or "cases" not in data:
        raise SystemExit("clock map missing cases")
    return data


def collect_pytest_nodeids() -> set[str]:
    proc = subprocess.run(
        [sys.executable, "-m", "pytest", "--collect-only", "-q"],
        check=False,
        capture_output=True,
        text=True,
    )
    ids = set()
    for line in proc.stdout.splitlines():
        line = line.strip()
        if "::" in line and not line.startswith("="):
            ids.add(line.split()[0])
    return ids


def main() -> int:
    root = Path.cwd()
    clock_map = load_map(root / "tests" / "clock_map.yaml")
    shared = os.environ.get("RUNNER_KIND", "shared") == "shared"
    collected = collect_pytest_nodeids()
    mapped = {c["path"] for c in clock_map["cases"]}

    undeclared = sorted(collected - mapped)
    if undeclared and clock_map.get("rules", {}).get("undeclared") == "fail_closed":
        print("undeclared tests (fail closed):")
        for node in undeclared:
            print(f"  {node}")
        return 2

    errors = []
    runnable = []
    for case in clock_map["cases"]:
        clock = case["clock"]
        if case["path"] not in collected:
            errors.append(f"mapped but missing: {case['path']}")
            continue
        if shared and clock not in ALLOWED_ON_SHARED:
            print(f"frozen off merge set: {case['id']} clock={clock}")
            continue
        runnable.append(case)

    if errors:
        print("\n".join(errors))
        return 2
    if not runnable:
        print("no logical tests remain; merge gate has nothing to score")
        return 2

    budget_ms = sum(int(c.get("wall_budget_ms") or 0) for c in runnable)
    nodeids = [c["path"] for c in runnable]
    env = os.environ.copy()
    env["CLOCK_MODE"] = "logical"
    started = time.monotonic()
    proc = subprocess.run(
        [sys.executable, "-m", "pytest", "-q", *nodeids],
        check=False,
        env=env,
    )
    elapsed_ms = int((time.monotonic() - started) * 1000)
    if elapsed_ms > budget_ms:
        print(f"campaign exceeded wall budget: {elapsed_ms}ms > {budget_ms}ms")
        return 3
    return proc.returncode


if __name__ == "__main__":
    raise SystemExit(main())

The budget is a fuse. It is not a quality score. If the fuse blows, you shrink the property campaign or you move the test out of logical. You do not raise the number until green.

Inject a logical clock in the tests the agent may extend

Property checks need a clock they do not own. Fake it in conftest.py so an agent cannot “fix” a flake by sleeping longer.

# tests/conftest.py — example; agent patches must not edit CLOCK_MODE
import os
from dataclasses import dataclass

import pytest

@dataclass
class LogicalClock:
    now_ms: int = 1_700_000_000_000

    def time(self) -> float:
        return self.now_ms / 1000.0

    def sleep(self, seconds: float) -> None:
        self.now_ms += int(seconds * 1000)

    def advance_ms(self, delta: int) -> None:
        self.now_ms += delta

@pytest.fixture
def clock(monkeypatch):
    mode = os.environ.get("CLOCK_MODE", "logical")
    if mode != "logical":
        pytest.skip("wall-clock tests are frozen on this runner")
    clk = LogicalClock()
    monkeypatch.setattr("time.time", clk.time)
    monkeypatch.setattr("time.sleep", clk.sleep)
    return clk

A property example stays boring on purpose. The invariant is about data, not duration.

# tests/test_invoice.py — example property, unexecuted in this article
import json
from pathlib import Path

from hypothesis import given, settings
from hypothesis import strategies as st

def parse_invoice(raw: dict) -> dict:
    cents = int(raw["cents"])
    if cents < 0:
        raise ValueError("negative")
    return {"cents": cents, "currency": raw.get("currency", "USD")}

@settings(max_examples=80, deadline=None)
@given(
    cents=st.integers(min_value=0, max_value=10**9),
    currency=st.sampled_from(["USD", "EUR", "JPY"]),
)
def test_parse_stable(clock, cents, currency):
    payload = {"cents": cents, "currency": currency, "ts": clock.time()}
    out = parse_invoice(payload)
    assert out["cents"] == cents
    assert out["currency"] == currency
    frozen = json.dumps({"cents": cents, "currency": currency}, sort_keys=True)
    assert "ts" not in json.loads(frozen)

Notice the timestamp is present in the input and absent from the assertion. That split is the point. Fixtures that still contain wall fields will fail the hash helper before pytest runs.

Numbered merge workflow

Use this as a checklist on the pull request. Do not collapse it into “run tests again.”

1. Freeze the map. Diff tests/clock_map.yaml as a human-owned file. An agent may propose rows. A person accepts the clock class.
2. Canonicalize fixtures. Strip ts, host, pid, and elapsed_ms before hashing. Store the hash next to the test id.
3. Collect, then compare. Any new nodeid not in the map is undeclared. Fail closed.
4. Drop non-logical cases on shared runners. Print them as frozen, not as skipped-pass.
5. Run only logical nodeids with CLOCK_MODE=logical and a summed wall_budget_ms.
6. Treat budget overflow as a red campaign, equal in rank to a failed assertion.
7. Park wall and mixed tests on a dedicated job if you still need them. They do not vote on merge.

Commands stay ordinary.

export RUNNER_KIND=shared
export CLOCK_MODE=logical
python campaign.py

If you generate patches from a free model and execute the campaign on a free server, set RUNNER_KIND=shared on that server every time. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode provides free model access and a free server option. Those two facts are why the clock map exists. A free server is still a contended runner. It is not a laboratory timer, and this article does not assign it a quota, a model name, or a hardware profile.

What the gate refuses on purpose

The campaign runner above will fail a patch that does any of the following:

• Adds tests/test_new.py::test_x without a clock row.
• Marks a sleep-based test as logical.
• Lets a property check run until the process is killed.
• Hashes a fixture that still embeds elapsed_ms.
• Converts a wall test to green by raising a duration threshold.

Each refusal is cheap to explain in review. That is the design. Reviewers argue about the label, not about a screenshot of a retry.

Limitations

Logical time does not model production timeouts, GC pauses, or disk stalls. A function that must meet a real latency SLO still needs a measurement job on stable hardware. This gate will not provide that job.

Canonical JSON is not a universal fixture format. Binary images, protobuf, and order-sensitive logs need their own strip rules. If you cannot define those rules, do not pretend the hash is a lock.

Hypothesis-style campaigns remain incomplete. Bounding wall time prevents runaway jobs. It does not prove the input space is covered. Eighty examples is a number in an example file, not a coverage theorem.

Shared runners stay noisy even after this policy. The policy removes wall-clock tests from the merge vote. It does not denoise the machine.

Who should not use this

Skip this workflow if you already run merge tests on a single-tenant runner with a fake clock and no agent-authored tests. You already have the hard part.

Skip it for hard real-time or safety-critical systems. A YAML map and a monotonic fuse are not a timing argument.

Skip it if the agent is allowed to edit campaign.py, conftest.py, or clock_map.yaml without a human-owned diff. The map is the oracle for the clock. Once the author of the patch owns the map, the gate is theater.

Close

Agent patches fail in boring ways when the runner is shared. Duration tests, sleeps, and live I/O look like product failures and are usually clock failures. Label the clock. Hash the fixture without wall fields. Bound the campaign. Freeze everything else off the merge set.

If you already keep agent work on a free workspace, put the clock map beside the patch instead of wrapping the suite in another retry. The labels are what you merge.