The merge looked clean late on Friday night. The agent had rewritten a billing cutoff helper. The unit tests passed in fourteen quiet seconds. Monday's batch job then billed three extra hours. The helper called datetime.now with no timezone. The agent's sandbox lived entirely in UTC. The production batch box lived in US/Eastern. Daylight saving time had flipped overnight on Sunday. No assertion in the suite ever named a clock.
Agent patches often fail through small ambient assumptions. They treat the generating machine as the world. File order looks stable until a network disk shuffles names. Locale settings change comma parsing for money fields. A remote build box and a laptop rarely share timezone data. Green tests then mean green here, not later.
This piece treats that failure as a testing problem. It is not a lecture on model quality. The patch can read clever and still be wrong. The suite must freeze every world the patch can see. Humans own that freeze for the life of the repo. Agents may write helpers and extra test assertions. Agents must not edit the freeze layer.
Think of the test process as a wind tunnel. The plane may change shape between two runs. The air speed cannot change between those runs. Fixtures form the walls of that tunnel. Clocks, RNG, cwd, and env are the moving air. If those move, you are not testing the patch. You are only testing weather around the patch.
Inventory the seams before the next agent session. Search the tree for datetime.now and time.time. Search also for random, uuid4, and os.listdir. Search for locale.setlocale and bare open calls. Each hit is a place the world leaks inward. Record those hits in a short seam log. The log is a review artifact, not documentation theater.
A fixture contract then replaces ambient process state. Input files live under tests/fixtures and stay committed. JSON beats implicit factory graphs for agent-written code. The agent can read a fixture during a patch. The agent cannot rewrite that fixture in the same patch. Schema checks on fixtures belong to human review. If a required field is missing, the suite fails closed.
The next sample is a worked example only. It is not taken from a production incident report. The helper must name a clock. The tests must pin that clock.
# billing/window.py
from __future__ import annotations
from datetime import datetime, timedelta, timezone
from typing import Protocol
CLOSE_HOUR_UTC = 4
class Clock(Protocol):
def now(self) -> datetime:
...
class SystemUtcClock:
def now(self) -> datetime:
return datetime.now(timezone.utc)
def billing_window_id(clock: Clock) -> str:
now = clock.now()
if now.tzinfo is None:
raise ValueError("naive datetime rejected")
now_utc = now.astimezone(timezone.utc)
day = now_utc.date()
if now_utc.hour < CLOSE_HOUR_UTC:
day = day - timedelta(days=1)
return day.isoformat()
The production clock stays out of the test process. Each test constructs FrozenClock with one ISO timestamp. Two wall clocks must map to one window id. That is the contract the agent is not free to relax. A second fixture freezes invoice directory order. The helper must sort names. The test must not depend on inode luck.
# tests/conftest.py
from __future__ import annotations
import json
from datetime import datetime
from pathlib import Path
import pytest
FIXTURES = Path(__file__).parent / "fixtures"
class FrozenClock:
def __init__(self, iso: str) -> None:
self._now = datetime.fromisoformat(iso)
if self._now.tzinfo is None:
raise ValueError("fixture clock must be aware")
def now(self) -> datetime:
return self._now
@pytest.fixture
def clock_matrix() -> list[dict]:
payload = json.loads((FIXTURES / "clock_matrix.json").read_text())
required = {"id", "iso", "window"}
for row in payload:
missing = required - set(row)
if missing:
raise AssertionError(f"fixture {row!r} missing {missing}")
return payload
{
"comment": "tests/fixtures/clock_matrix.json",
"rows_are_in_the_array_below": true
}
Keep the real fixture as a JSON array. The comment object above is not the file. The committed file is only data. Humans edit that array. Agents do not.
[
{
"id": "utc_before_close",
"iso": "2026-03-09T03:59:59+00:00",
"window": "2026-03-08"
},
{
"id": "utc_at_close",
"iso": "2026-03-09T04:00:00+00:00",
"window": "2026-03-09"
},
{
"id": "eastern_after_spring_forward",
"iso": "2026-03-08T03:30:00-04:00",
"window": "2026-03-08"
}
]
# tests/test_billing_window.py
from __future__ import annotations
import os
from pathlib import Path
from billing.window import billing_window_id
from tests.conftest import FrozenClock
def test_window_id_follows_fixture_matrix(clock_matrix):
for row in clock_matrix:
clock = FrozenClock(row["iso"])
assert billing_window_id(clock) == row["window"], row["id"]
def test_naive_datetime_is_rejected():
class NaiveClock:
def now(self):
from datetime import datetime
return datetime(2026, 3, 9, 3, 0, 0)
try:
billing_window_id(NaiveClock())
except ValueError as exc:
assert "naive" in str(exc)
else:
raise AssertionError("naive clock must fail closed")
def test_invoice_names_are_sorted(tmp_path: Path):
(tmp_path / "b-17.json").write_text("{}")
(tmp_path / "a-02.json").write_text("{}")
names = sorted(p.name for p in tmp_path.iterdir())
assert names == ["a-02.json", "b-17.json"]
assert os.listdir(tmp_path) != names or names == sorted(os.listdir(tmp_path))
The last assertion looks odd on purpose. listdir order is not a contract. Sorted names are the contract. If the helper later prints os.listdir output, the suite should fail. Teach that rule in review, not in a prompt paragraph.
Run the seam scan before you run pytest. Then run pytest under two timezones. The freeze must still win. If TZ changes the window id, the helper still reads the wall clock.
rg -n "datetime\\.now|time\\.time|uuid\\.uuid4|random\\.|os\\.listdir|locale\\." billing tests
TZ=Pacific/Auckland pytest -q tests/test_billing_window.py
TZ=UTC pytest -q tests/test_billing_window.py
python - <<'PY'
from datetime import datetime
from zoneinfo import ZoneInfo
from billing.window import billing_window_id
from tests.conftest import FrozenClock
print(billing_window_id(FrozenClock("2026-03-08T03:30:00-04:00")))
print(datetime.now(ZoneInfo("US/Eastern")).isoformat())
PY
The last print is a contrast, not a test. Live now() has no place in the merge gate. Put it in a manual probe if operators need it. Keep it off CI.
Some teams generate the patch on a free model tier. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option. That split is the point of the freeze. The model proposes the helper. The server runs the hermetic suite. The laptop clock never becomes the source of truth. The same freeze layer belongs in the repo either way.
A short decision rule keeps reviews short. If the signal is clock drift, freeze it and block merge. If the signal is live network, stub it and keep it off the gate. If the output embeds a random UUID, pin the generator in tests. If the agent wants a new fixture field, a human lands that field first. If a test needs real time to pass, it is not a merge test. Call it a probe and skip it in CI.
This approach has sharp limits. A frozen UTC clock can hide timezone bugs. You still need an explicit matrix of aware timestamps. The three rows above are a start, not coverage. Stubs are not the network. Deterministic UUID tests will not catch collisions. Sorting directory names will not catch missing files on a half-written disk. The freeze also cannot save a helper that still calls datetime.now inside a nested utility. The seam log has to stay current after each patch.
Skip this workflow if live time is the product. A trading clock, a TOTP helper, and a cron parser need a different harness. Skip it if no human owns tests/fixtures. An agent that can edit the matrix can hide a billing change inside a fixture tweak. Skip it for safety-critical control software that needs formal methods. A JSON array is not a proof.
Agent patches make ambient tests look productive. They complete fast. They match the machine that wrote them. The bill shows up on Monday. Pin the clock. Pin the locale. Sort the names. Leave the weather outside the tunnel.
Top comments (0)