DEV Community

Simon Massey
Simon Massey

Posted on

🔒 Lock Folder Util — The Mutex Your Agent Swarm Desperately Needs

#ai #agents #programming #productivity

Twelve Agents. One Resource. Zero Supervision. What Could Possibly Go Wrong?

You know that moment when you finally parallelise your agent fleet and watch with pride as twelve subagents sprint off to do the work of a whole team... straight into the same shared resource. Maybe it is the one licensed tool seat. Maybe it is the office coffee machine you told everyone could "handle concurrent requests". Maybe you set a dozen agents loose to find the funniest cat picture on the internet, and they all decide the single shared webcam is the fastest route to greatness, and now your laptop is taking twelve selfies of the ceiling while a queue of increasingly confused agents interrogates it about cats.

Yeah. We've all been there.

The Problem

  • 🤯 Multiple agents driving one shared resource (a robot, a tool seat, one very confused peripheral)
  • 💥 Interleaved commands corrupting each other's work mid-flight
  • 🕵️ No audit trail of who held the resource and when
  • ⚰️ A crashed agent leaving the resource "locked" forever (or worse, not locked at all)

The Solution: lock_folder_util.py

A zero-dependency Python script that turns one humble mkdir into a fully auditable mutual-exclusion lock. Because mkdir is atomic at the filesystem level: it either creates the directory or fails because it already exists. No races. No lock file content. No NFS weirdness. Just one empty directory standing guard.

# The entire synchronization mechanism:
mkdir .tmp/lock.lock    # succeeds exactly once
rmdir .tmp/lock.lock    # release
Enter fullscreen mode Exit fullscreen mode

That's it. That's the lock.

🚀 Features That Actually Matter

1. The run Form (Use This One)

Acquire, execute, release, no matter what:

./lock_folder_util.py run --slug agent-7 -- your-command --with args
Enter fullscreen mode Exit fullscreen mode

Crashes mid-command? The lock goes stale. The next agent breaks it after
--stale seconds and carries on. Self-healing, zero babysitting.

2. Full Audit Trail

Every acquire, release, stale-break, and timeout is one line in
.tmp/lock.log with a UTC timestamp and the agent's slug:

2026-08-22T23:05:52Z agent-1 acquired
2026-08-22T23:05:53Z agent-1 released
2026-08-22T23:05:58Z agent-2 acquired
2026-08-22T23:05:59Z agent-2 released
2026-08-22T23:06:04Z agent-3 acquired
2026-08-22T23:06:05Z agent-3 released
2026-08-22T23:06:10Z agent-4 acquired
2026-08-22T23:06:11Z agent-4 released
2026-08-22T23:06:16Z agent-5 acquired
2026-08-22T23:06:16Z agent-5 released
2026-08-22T23:06:22Z agent-6 acquired
2026-08-22T23:06:23Z agent-6 released
2026-08-22T23:06:28Z agent-7 acquired
2026-08-22T23:06:29Z agent-7 released
2026-08-22T23:06:34Z agent-8 acquired
2026-08-22T23:06:35Z agent-8 released
Enter fullscreen mode Exit fullscreen mode

Eight agents. Sixteen lines. Perfectly interleaved acquire/release pairs. Not one overlap. When something does go wrong at 2am, you will know exactly who was holding the door.

3. Stale-Lock Breaking (Self-Healing)

Agent dies mid-critical-section? Its lock sits there with a birth timestamp. The next agent in line checks the age, finds it past --stale (default 600s), removes it, logs the break, and proceeds. No human intervention. No queue frozen for eternity. That webcam will be pestered for kittens until kittens we have!

4. Zero Dependencies

Python 3 standard library. That's the whole list. No pip, no cargo, no apt-get. If your box runs Python, you have a mutex.

5. Works Everywhere mkdir Works

macOS, Linux, WSL, the BSDs, that NAS in the cupboard. If the filesystem can atomically create a directory, this lock holds.

6. Configurable Lock Directory, Fail Fast

The lock directory defaults to .tmp and is overridden with the env var LOCK_FOLDER_DIR (for tests, or for co-located projects that want an isolated lock). At start-up the script fails fast unless the directory
exists, is a directory, and is writable. A misconfigured lock is a noisy error message at second zero, not a silent no-op mutex that lets your agents pile into the coffee machine:

LOCK_FOLDER_DIR=/tmp/swarm-test ./lock_folder_util.py run --slug t1 -- sleep 1
# lock_folder_util: lock dir /nonexistent is not a writable directory; ...
Enter fullscreen mode Exit fullscreen mode

📖 Pseudo Man Page (The Details)

NAME

lock_folder_util.py — mutual exclusion for agents sharing one resource, via an atomic mkdir

SYNOPSIS 

./lock_folder_util.py acquire --slug NAME [--poll 15] [--stale 600] [--timeout 1800]
./lock_folder_util.py release --slug NAME
./lock_folder_util.py run --slug NAME [--poll 15] [--stale 600] -- <command...>
Enter fullscreen mode Exit fullscreen mode

OPTIONS 

--slug NAME    Your agent's identity, logged with every transition
--poll N       Seconds between lock attempts while waiting (default 15)
--stale N      Break locks older than N seconds (default 600)
--timeout N    Give up after N seconds total (default 1800)
Enter fullscreen mode Exit fullscreen mode

THE PROTOCOL (read this bit)

  1. If you are the batch's first agent, take the lock immediately.
  2. Everyone else: sleep 60 once, then poll every 15 seconds #YMMV
  3. Wrap the ENTIRE resource-driving critical section, and release immediately after your last resource command. Thinking, reading, and file work need no lock.
  4. Never put files inside the lock directory. It must stay empty or rmdir fails and you have made a new problem.

EXIT STATUS

  • 0: Success (or the wrapped command's own status, in run mode)
  • 1: Timed out waiting, or bad usage

EXAMPLES

Example 1: Reserve the shared gadget for a job

./lock_folder_util.py run --slug gadget-keeper -- \
    ./use_the_gadget.py --mode serious
Enter fullscreen mode Exit fullscreen mode

Example 2: Manual acquire around a long session

./lock_folder_util.py acquire --slug worker-1
# ... command the shared resource, poke it carefully ...
./lock_folder_util.py release --slug worker-1
Enter fullscreen mode Exit fullscreen mode

Example 3: Compressed timings for a fast swarm

./lock_folder_util.py run --slug agent-9 --poll 2 --stale 12 -- sleep 1
Enter fullscreen mode Exit fullscreen mode

🧪 Testing Suite Not Included

Yet you can test it with this 8-agent thundering herd:

# One agent takes the lock immediately, seven sleep 6s then storm it
./lock_folder_util.py run --slug agent-1 --poll 2 --stale 12 -- sleep 1 &
for i in 2 3 4 5 6 7 8; do
  ( sleep 6; ./lock_folder_util.py run --slug agent-$i --poll 2 --stale 12 -- sleep 1 ) &
done
wait
cat .tmp/lock.log
Enter fullscreen mode Exit fullscreen mode

Then verify the invariants mechanically, not by eyeball:

python3 - <<'EOF'
lines = open('.tmp/lock.log').read().splitlines()
held = 0; overlaps = 0
for l in lines:
    if l.endswith('acquired'): held += 1; overlaps = max(overlaps, held)
    elif l.endswith('released'): held -= 1
print('PASS: no overlapping critical sections' if overlaps <= 1
      else f'FAIL: {overlaps} agents held the lock simultaneously')
import os
print('PASS: no lock leaked' if not os.path.isdir('.tmp/lock.lock')
      else 'FAIL: lock leaked')
EOF
Enter fullscreen mode Exit fullscreen mode

What you want to see:

  • acquired=N released=N gave_up=0 stale_breaks=0 (all agents got through)
  • Overlap check: PASS (never two holders)
  • Leak check: PASS (lock directory gone at the end)

And a stale-lock test, for the crash case: create the lock, backdate its mtime with os.utime, and watch the next agent break it cleanly and log the break.

💡 Use Cases That'll Make You Look Like a Genius

For Agent Fleet Wranglers

  • One gadget, twelve agents: serialise every command, poke, and reading without a coordinator process
  • RAG pipeline guard: one writer process for the index, many researchers in flight
  • Serialised device access: one robot arm, one 3D printer, one oscilloscope, N impatient agents

For Anyone With a Shared Toy

  • The one licensed EDA tool seat everyone "just quickly needs"
  • The dev database that only tolerates one migration at a time
  • The communal coffee machine, the office 3D printer, the single shared webcam being interrogated by twelve cat-picture agents
  • The family TV remote (results may vary)

⚡ Installation

Star then download. Star. "⭐💫🌟" You know, like thumbs up, but for yoof of today. STAR THE GIST ⭐⭐⭐

If you use gh cli, and you should, then you can get it with this fancy one-liner:

for f in $(gh gist view 25596aca12cf0057d01ded9dcc0853a9 --files); do gh gist view 25596aca12cf0057d01ded9dcc0853a9 -f "$f" > "$f"; done && chmod +x lock_folder_util.py
Enter fullscreen mode Exit fullscreen mode

If you do not use gh, well, srsly, do. Or if you must do it manually its over at [https://gist.github.com/simbo1905/25596aca12cf0057d01ded9dcc0853a9]

Make executable

chmod +x lock_folder_util.py

Optional: Add to PATH

cp lock_folder_util.py ~/bin/lock_folder_util.py

Or just run it directly if your not the global-install-files sort:

./lock_folder_util.py run --slug you -- echo "hello exclusive world"

🎯 Why This Exists

Born from the exact scenario above: a fleet of agents, one shared
resource, and a first attempt at "polite staggering" that told the last
agent to sleep 44 minutes before touching the thing. Forty. Four.
Minutes. Of sleeping. While holding a todo list.

Sometimes you just need the dumbest possible thing that works: one empty
directory, mkdir, and a log file. As that is obviously how you think and
act. You are not an orchestrator up at 2am using SCREAMING ALL CAPS as
your agents deadlock over the office coffee machine. That is definately
not you, no. Me neither.

📜 License

MIT or Public Domain. Use it, abuse it, put it in production, whatever.
No warranty implied. If two agents somehow end up on the coffee machine
at once, check the log before checking your assumptions.

Made with ❤️ and one atomic mkdir by someone who once watched an agent sleep 44 minutes.

Now go lock your shared resources like a pro. 🔒✨

Top comments (1)

Collapse
 
reidmarlow profile image
Reid Marlow •

This is the kind of tiny primitive agent setups need more often. I would add one boring field to the lock directory, a run id or parent task id, because stale lock breaking gets much easier when the log can say which coordinator died instead of just which slug owned it.