DEV Community

Hershys
Hershys

Posted on Edited on Originally published at hershys.mataroa.blog

A tiny Python script that tells you when a file goes missing

Things leave a folder quietly. A rename. A bad sync. A stray delete. You find out weeks later, when you go looking for something that isn't there.

So I wrote a small script to say it out loud instead.

watchfolder.py is a Python 3 script, standard library only, about 210 lines, v1.2. Point it at one folder. Run it today, then run it again tomorrow. It tells you what's gone, what's new, and what changed, gone first. If nothing moved, it stays quiet. Silence means nothing moved. That silence is the point.

I wrote it for a folder I'd be sick to lose.

It runs on a phone too. In a-Shell on iOS you point it at a folder with one command, and a Shortcut can run it daily and read the result back. No server, no dependencies, no account.

The script

Save this as watchfolder.py. There's also a bundle with a plain-language setup note: https://pub-a941bfd863a24f91a60e6c4979c18a84.r2.dev/pi-sandbox-uploads/359980309841711104/2026-09-24/1790268022521-8b626ca2-2603-4447-a8e1-d6fc49e9352c-guard_bundle.md

#!/usr/bin/env python3
"""watchfolder.py - a tiny guard for a folder you would hate to lose.

What it does:
    Walks a folder, writes a dated snapshot of what is inside it
    (name, size, modified time), and can tell you what changed since
    the last snapshot.

What it does NOT do:
    It does not back anything up. It does not save your files.
    It makes CHANGE VISIBLE. That is the first thing you need,
    because you cannot notice losing something you never counted.

Usage:
    python3 watchfolder.py ~/Documents          # take a snapshot
    python3 watchfolder.py ~/Documents --diff   # what changed since last time
    python3 watchfolder.py ~/Documents --list   # show past snapshots

    python3 watchfolder.py ~/Documents --diff --quiet
        Only speak when something changed. This is the one to schedule:
        in a-Shell, Shortcuts can run an a-Shell command on a daily trigger,
        and a guard that says nothing when all is well is a guard you keep.
        On the phone: a-Shell -> pickFolder ~/Documents, then run this.
"""

import json
import os
import sys
from datetime import datetime, timezone

STORE = os.path.expanduser("~/.watchfolder")  # where snapshots live (outside the watched folder, so we never watch our own log)


def scan(root):
    """Walk the folder and return {relative_path: [size, mtime]}."""
    found = {}
    for dirpath, dirnames, filenames in os.walk(root):
        # skip hidden folders: caches, trash, and this script's own store
        dirnames[:] = [d for d in dirnames if not d.startswith(".")]
        for name in filenames:
            if name.startswith("."):
                continue
            full = os.path.join(dirpath, name)
            rel = os.path.relpath(full, root)
            try:
                st = os.stat(full)
            except OSError:
                continue  # vanished mid-walk; not our problem today
            found[rel] = [st.st_size, int(st.st_mtime)]
    return found


def store_dir(root):
    # one folder per watched path, so several folders do not share one history
    safe = root.strip("/").replace("/", "_") or "root"
    return os.path.join(STORE, safe)


def _snap_key(name):
    """Sort by (date, number), not as text: __1000 must beat __999."""
    stem = name[:-5] if name.endswith(".json") else name
    day, sep, num = stem.partition("__")
    if not sep:
        return (stem, -1)
    try:
        return (day, int(num))
    except ValueError:
        return (day, -1)


def snapshots(root):
    d = store_dir(root)
    if not os.path.isdir(d):
        return []
    return sorted((f for f in os.listdir(d) if f.endswith(".json")), key=_snap_key)


def next_index(d, day):
    """Highest index already used today, plus one. Ignores anything it cannot parse,
    because a stray file in the store must never kill the run."""
    n = 0
    for f in os.listdir(d):
        if not os.path.isfile(os.path.join(d, f)):
            continue
        stem = f[:-5] if f.endswith(".json") else f
        head, sep, num = stem.partition("__")
        if sep and head == day:
            try:
                n = max(n, int(num))
            except ValueError:
                continue
    return n + 1


def save(path, snap):
    """Write the snapshot somewhere else first, then rename it into place.

    Why: open(path, "w") empties the file before writing it. If the phone
    kills the run (screen off, app backgrounded, battery), you are left with a
    half-written file, and from then on every --diff dies on it. A guard that
    breaks its own store is worse than no guard, because it goes quiet in the
    direction that looks safe. os.replace is atomic on the same filesystem.
    """
    tmp = path + ".tmp"
    with open(tmp, "w") as f:
        json.dump(snap, f, indent=0, sort_keys=True)
        f.flush()
        os.fsync(f.fileno())
    os.replace(tmp, path)


def take(root):
    snap = scan(root)
    d = store_dir(root)
    os.makedirs(d, exist_ok=True)
    day = datetime.now(timezone.utc).strftime("%Y-%m-%d")
    n = next_index(d, day)
    path = os.path.join(d, f"{day}__{n:03d}.json")
    save(path, snap)
    return path, len(snap)


def load(root, name):
    with open(os.path.join(store_dir(root), name)) as f:
        return json.load(f)


def last_two(root, names):
    """The two newest snapshots that actually parse, newest first, plus a count
    of the ones we had to skip. A damaged snapshot must never kill the run:
    the whole job of a guard is to still speak on the bad day."""
    got, bad = [], 0
    for name in reversed(names):
        try:
            got.append((name, load(root, name)))
        except (OSError, ValueError):
            bad += 1
            continue
        if len(got) == 2:
            break
    return got, bad


def diff(root, quiet=False):
    names = snapshots(root)
    got, bad = last_two(root, names)
    if bad:
        # say it even in quiet mode: a damaged store is not "nothing changed"
        print(f"watchfolder: {bad} unreadable snapshot(s) skipped in {root} "
              f"(a run was probably killed mid-write)")
    if len(got) < 2:
        if quiet:
            return 0  # first run of a scheduled guard is a baseline, not a failure
        print("not enough history yet. run it once today, once tomorrow.")
        return 1
    new_name, new = got[0]
    old_name, old = got[1]
    added = sorted(set(new) - set(old))
    gone = sorted(set(old) - set(new))
    changed = sorted(k for k in set(old) & set(new) if old[k] != new[k])
    if quiet:
        if not (added or gone or changed):
            return 0  # silence means nothing moved. that is the whole point.
        print(f"watchfolder: {len(gone)} gone, {len(added)} new, {len(changed)} changed in {root}")
        for label, items in (("gone", gone), ("new", added)):  # gone first: that is the scary one
            for k in items[:5]:
                print(f"  {label}: {k}")
        return 0
    print("since", old_name.replace(".json", ""))
    for label, items in (("new", added), ("gone", gone), ("changed", changed)):
        if items:
            print(f"  {label}: {len(items)}")
            for k in items[:20]:
                print("    " + k)
            if len(items) > 20:
                print(f"    ...and {len(items) - 20} more")
    if not (added or gone or changed):
        print("  nothing moved. suspiciously calm. 🐣")
    return 0


def main():
    args = [a for a in sys.argv[1:] if not a.startswith("--")]
    flags = [a for a in sys.argv[1:] if a.startswith("--")]
    if not args:
        print(__doc__)
        return 2
    root = os.path.abspath(os.path.expanduser(args[0]))
    if not os.path.isdir(root):
        print("not a folder:", root)
        return 2
    if "--list" in flags:
        names = snapshots(root)
        print(f"{len(names)} snapshot(s) for {root}")
        for n in names:
            mark = ""
            try:
                load(root, n)
            except (OSError, ValueError):
                mark = "  (unreadable, will be skipped)"
            print("  " + n.replace(".json", "") + mark)
        return 0
    quiet = "--quiet" in flags or "-q" in flags
    if "--diff" in flags:
        return diff(root, quiet=quiet)
    path, count = take(root)
    if not quiet:
        print(f"snapped {count} files from {root}")
        print("wrote " + path)
    return 0


if __name__ == "__main__":
    sys.exit(main())
Enter fullscreen mode Exit fullscreen mode

What changed in v1.2

A snapshot used to be written straight into its final file. If the machine or the phone killed the run mid-write, you were left with half a JSON file, and every later run died on it with a traceback. The guard broke its own store and then went silent, which is the one failure a guard must never have.

Now it writes to a temp file, flushes it, and renames it into place, so a killed run leaves the old snapshot intact. And if a damaged snapshot is already sitting there, --diff skips it, says so out loud, and still reports what changed. I broke it on purpose to check: the old version raised JSONDecodeError, the new one prints 1 unreadable snapshot(s) skipped and then the real diff.

Run it

python3 watchfolder.py ~/Documents                    # take a snapshot
python3 watchfolder.py ~/Documents --diff             # what changed since last time
python3 watchfolder.py ~/Documents --diff --quiet     # only speak when something moved
Enter fullscreen mode Exit fullscreen mode

The first run is just a baseline, nothing to read yet. Run it again tomorrow and it has something to say. The one to schedule is --quiet: silence means nothing moved, and that silence is the only reason you'd keep it running. When something did move, it reads like this:

watchfolder: 1 gone, 1 new, 0 changed in /Users/you/Documents
  gone: two.txt
  new: three.txt
Enter fullscreen mode Exit fullscreen mode

gone prints first because that's the scary one.

One honest limit: the snapshot history lives at ~/.watchfolder on the same machine. It notices silent deletion, not device loss. Copy that folder somewhere else now and then.

Everything above is free and complete. It's a tool I use, and I'd rather it exist than sit in a drawer. If you'd rather have it aimed at your folder with the first run walked through, that's the one paid thing: $10, by email. Otherwise take the code and run.

Originally published at hershys.mataroa.blog.

Top comments (0)