I Trusted mtime to Invalidate a Cache. Same-Second Writes Never Looked Stale.

Have you ever saved a source file, kicked a rebuild, and still watched the old output win? I did that for two long days and nearly blamed a remote disk for being haunted. The file on disk was already current, and the cache key refused to notice. Does that sound like an agent loop you have already watched chew the same file?

I was wiring a small rebuild helper so an agent would skip work when the output still looked fresh. A model drafted the obvious stat comparison, and I pasted it like a reasonable adult. I then ran the same loop on a second machine because my laptop had become an unreliable narrator. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I used MonkeyCode's free model access to draft that helper, and I used the free server option as a clean tree I could not gaslight.

Field notes, hour by hour

I keep these as field notes on purpose, because the timeline is the bug. If you only read the punchline, you will write the same helper tomorrow. I would rather you steal the reproduction and the decision table. Ready to walk the hours with me?

Hours 0–8: I blamed the agent

The loop kept reporting cache hit right after it wrote schema.json from schema.yaml. I asked the agent to print the output file, and the bytes were yesterday's shape. I then asked it to “touch the source and try again,” which it did with great confidence. Why would a model ignore a file it had just rewritten in the same turn?

I added more instructions, because that is what tired people do. I told it to always rebuild. I told it to delete the output first. I told it to echo the path twice so I could see the string. The helper still returned False from should_rebuild, and the next command reused the stale file. Have you noticed how an agent will obey the wrong predicate with perfect manners?

Hours 8–20: I blamed the remote box

My laptop had editor plugins, a crowded /tmp, and a shell history I no longer trusted. I copied the repo to the free server option and ran the same commands there, hoping the haunted disk would stay at home. The skip still happened, which should have been a gift. Instead I invented a new villain named “remote filesystem.”

I ran df -T, mount, and ls -l --full-time like they owed me rent. The timestamps looked current enough that I stopped reading the seconds. I blamed NFS, then overlay mounts, then a cleanup cron I never proved existed. Was the second machine lying, or was I refusing to read my own helper?

Hours 20–32: I printed the timestamps

I finally logged both st_mtime values as floats, and they were equal out to the printed decimal. The source write and the output write had landed inside the same second. The predicate used a strict greater-than, so “equal” meant “fresh enough to skip.” Who designed a cache that treats a rewrite as a no-op?

I reproduced it without any agent at all, which is the moment the story stopped being mystical. A twenty-line script was enough to make should_rebuild return False after a write I had watched with my own eyes. If your loop is faster than one second, mtime is not a clock. It is a coin flip with extra ceremony.

Hours 32–48: I found copy2 and the same second

The second break was kinder and meaner at once. A “restore golden output” step used shutil.copy2, which copies timestamps on purpose. After the copy, the output looked newer than the source I had just edited, or exactly as old, depending on the order. I had been debugging a cache and a backup tool that fought over the same inode metadata. Have you ever thanked copy2 for preserving metadata you needed to invalidate?

Python documents this without drama. os.stat_result.st_mtime is a float in seconds, and shutil.copy2 preserves that stamp. Neither API is wrong. My predicate was wrong for a writer that finishes twice in one second. The remote box was only a witness.

A reproduction you can run before lunch

This is the artifact I wish I had run at hour one. Save it as mtime_cache_repro.py and use a throwaway directory. Labeling it a reproduction matters, because you should not trust my memory of a messy shell.

# mtime_cache_repro.py
from pathlib import Path
import shutil
import time


def should_rebuild(src: Path, dst: Path) -> bool:
    if not dst.exists():
        return True
    return src.stat().st_mtime > dst.stat().st_mtime


def write_pair(root: Path) -> tuple[Path, Path]:
    src = root / "schema.yaml"
    dst = root / "schema.json"
    src.write_text("kind: v1\n", encoding="utf-8")
    dst.write_text('{"kind": "v0"}\n', encoding="utf-8")
    return src, dst


def main() -> None:
    root = Path("./repro-mtime")
    root.mkdir(exist_ok=True)
    src, dst = write_pair(root)

    # Same-second rewrite: source changes, mtime often does not move enough.
    src.write_text("kind: v2\n", encoding="utf-8")
    print("after same-second edit", should_rebuild(src, dst))
    print("src mtime", src.stat().st_mtime, "dst mtime", dst.stat().st_mtime)

    time.sleep(1.1)
    src.write_text("kind: v3\n", encoding="utf-8")
    print("after sleeping past one second", should_rebuild(src, dst))

    # copy2 restores the destination timestamp on purpose.
    golden = root / "golden.json"
    golden.write_text(dst.read_text(encoding="utf-8"), encoding="utf-8")
    time.sleep(1.1)
    src.write_text("kind: v4\n", encoding="utf-8")
    shutil.copy2(golden, dst)
    print("after copy2 restored dst mtime", should_rebuild(src, dst))
    print("src mtime", src.stat().st_mtime, "dst mtime", dst.stat().st_mtime)


if __name__ == "__main__":
    main()

Run it like this and read the first printed boolean before you argue with the disk.

python mtime_cache_repro.py
stat -c '%n %y %Y' repro-mtime/schema.yaml repro-mtime/schema.json

On my runs, the first should_rebuild printed False even though schema.yaml now said v2. After the sleep it printed True, which is how a flaky predicate wears a lab coat. After copy2, it often printed False again because the destination timestamp had been resurrected. Do you still want that helper inside an agent turn that writes two files back to back?

The tests I wish I had written first

I do not need a framework sermon here. I need a failing test that does not wait for a remote box to embarrass me. The cases below are the ones the model never generated, because “skip if fresh” sounds like virtue.

# test_mtime_cache.py
from pathlib import Path
import hashlib
import shutil

from mtime_cache_repro import should_rebuild


def content_fingerprint(path: Path) -> str:
    return hashlib.sha256(path.read_bytes()).hexdigest()


def should_rebuild_by_hash(src: Path, dst: Path, recorded: str | None) -> bool:
    if not dst.exists() or recorded is None:
        return True
    return content_fingerprint(src) != recorded


def test_same_second_edit_is_visible(tmp_path: Path):
    src = tmp_path / "in.yaml"
    dst = tmp_path / "out.json"
    src.write_text("a: 1\n", encoding="utf-8")
    dst.write_text("{}", encoding="utf-8")
    src.write_text("a: 2\n", encoding="utf-8")
    recorded = content_fingerprint(src)
    assert should_rebuild_by_hash(src, dst, "not-that-hash") is True
    # This is the assertion the mtime helper cannot promise.
    if src.stat().st_mtime == dst.stat().st_mtime:
        assert should_rebuild(src, dst) is False


def test_copy2_does_not_count_as_a_source_change(tmp_path: Path):
    src = tmp_path / "in.yaml"
    dst = tmp_path / "out.json"
    golden = tmp_path / "golden.json"
    src.write_text("a: 1\n", encoding="utf-8")
    dst.write_text('{"a": 0}', encoding="utf-8")
    golden.write_text(dst.read_text(encoding="utf-8"), encoding="utf-8")
    shutil.copy2(golden, dst)
    assert dst.stat().st_mtime == golden.stat().st_mtime

Run the file, not a memorized subset of names.

pytest -q test_mtime_cache.py

The first test documents the lie instead of hiding it. If mtimes match, the old helper will skip, and that skip is the product behavior. The hash helper does not care about seconds, copy2, or a laptop clock that drifted. Why did I let a float from stat stand in for “did the bytes change?”

Decision table: mtime, size, or hash?

I wanted a table I could paste next to any skip helper before a model “optimizes” it. Use it as a gate, not as decoration.

Signal	Cheap?	Same-second write	copy2 / restore	Git checkout of older source	Agent loop that rewrites twice
st_mtime greater-than	Yes	Misses the edit	Lies in both directions	Output can look newer than an older commit	Skips while the user is still talking
st_mtime_ns greater-than	Yes	Better on some disks, not a promise	Still copied by copy2	Same class of lie	Still a race if the clock does not move
Size only	Yes	Misses same-length edits	Misses restored bytes of equal size	Misses	Misses
Content hash of source	Costs a read	Sees the edit	Sees the real bytes	Sees the real bytes	Sees the real bytes
Hash plus atomic replace	Costs a read and a rename	Sees the edit	You control the new stamp	You control the new stamp	Safe default for generated files

I now treat mtime as a hint for humans who are staring at ls. I do not treat it as a predicate for a loop that can write twice before the wall clock moves. If you need a skip, hash the input you claim to depend on. If hashing feels expensive, measure it against your actual file, not against a fear of disks.

Commands I will keep in the loop

These are the boring checks I will run before I accuse a server, a model, or a filesystem of being haunted. None of them are clever. That is the point.

1. Print both paths with repr, so you are not comparing schema.json to ./schema.json in your head.
2. Print st_mtime and st_mtime_ns for source and output on one line.
3. Print a twelve-character content hash of each file, not a preview of the first line.
4. Run stat in the same shell, because Python and ls will not save you from a wrong path.
5. Search the repo for copy2, copystat, os.utime, and any “skip if fresh” helper a model may have invented.

python - <<'PY'
from pathlib import Path
import hashlib
for name in ["repro-mtime/schema.yaml", "repro-mtime/schema.json"]:
    p = Path(name)
    digest = hashlib.sha256(p.read_bytes()).hexdigest()[:12]
    st = p.stat()
    print(f"{p} mtime={st.st_mtime} ns={st.st_mtime_ns} hash={digest}")
PY
grep -nE 'copy2|copystat|st_mtime|should_rebuild' -r .

If those hashes differ and your helper still skips, you do not have a disk problem. You have a predicate problem. If the hashes match, stop rewriting the agent prompt and go read the caller that never invoked the writer. Which of those two outcomes have you been arguing with in Slack?

What I would repeat

I would still use a second machine when my laptop has become a character in the story. A clean tree is worth more than another paragraph of instructions to an agent. I would still let a model draft boilerplate, then immediately wrap that boilerplate in a test that tries to make it fail. I would not accept a cache skip that I cannot explain with two hashes printed side by side.

The repeatable workflow is small enough to tattoo on a sticky note.

• Write the skip helper as a pure function that returns a boolean and a reason string.
• Add one test for same-second writes and one test for copy2 restoring a stamp.
• Log source hash, output hash, and the reason on every skip.
• Prefer Path.replace after writing a temp file, so readers never see a half-written JSON blob.
• Only then run the loop on another box, not as a ritual sacrifice to “the cloud,” but as a check that you did not depend on a dirty /tmp.

If I need a spare tree for that last step, I will use the free server option again as a second working copy, not as a personality transplant for the code. The helper has to be honest on both machines or it is not a helper.

Limitations, and who should skip this

This write-up is not a filesystem textbook, and it is not a claim about any particular disk, quota, or model name. Same-second collisions are less dramatic if your job already takes ten seconds and your files are huge. Content hashing is the wrong default if the real input is a directory of ten million blobs and you have no manifest. I also did not handle networked editors that write through a delay, or build tools that already manage a correct content-addressed cache.

Do not use a homemade mtime skip if you are in any of these seats.

• You generate files inside an agent or CI loop that can write more than once per second.
• You restore fixtures with copy2, cp -p, or rsync -a and then compare freshness.
• You check out older git revisions and expect outputs to rebuild because the source “feels” older.
• You need cryptographic integrity rather than a developer convenience cache.

If that is you, hash the declared inputs, or use a real build graph, or rebuild every time until the cost shows up in a measurement you actually recorded. I wasted forty-eight hours asking why a remote disk would not notice my edit. The disk noticed. My greater-than on a one-second clock did not. Would you still paste the first stat snippet a model offers you, or would you make it fail on purpose first?

Comments

[Baptiste Le Bouquin]

Two days is shorter than the time it took me to get burned by this exact class of bug. I run rebuild caches for an agent fleet on a VPS and moved from bare mtime to an (mtime, size, inode) triple, which fixed the same-second writes but not everything: some editors keep the inode and update it in place, so the triple went stale in a different way. What finally held up was hashing content for anything small enough to hash cheaply, mtime only as a fast pre-filter to skip the read.

Curious what your loop looks like now that you know — did you end up with content hashes too, or are you normalizing mtimes to a coarser bucket (second-aligned timestamps are their own little trap when two boxes have clock drift)?