DEV Community

Matt Dodd
Matt Dodd

Posted on

Show DEV: We open-sourced a filesystem where parallel agents can’t overwrite each other

We're Matthew and Matt, two developers who got tired of a problem and built a thing to fix it. We just open-sourced it (dual MIT/Apache-2.0). This is the honest version.

It's open source, and you can self-host it

Run it yourself, locally or on your own box, or use our hosted service. Point git, git-lfs, or the desktop mount at it, and keep your blocks on local disk or any S3-compatible bucket (S3, R2, MinIO). The first full open-source release just went out: the self-hosted swarmfile-server, the swarmfile:// git remote helper, and the standalone tools (migrate, mirror, seed, doctor), plus the block/manifest format spec with conformance vectors.

You can clone our repo right now, anonymously, no account needed:

git clone swarmfile://swarmfile/swarmfile-lfs
Enter fullscreen mode Exit fullscreen mode

The bit we actually want feedback on: parallel agents on one branch

We run fleets of coding agents on a single branch, across local machines, remote boxes, and CI. The filesystem makes them read before they write. If an agent tries to save a byte range without having read its current version, the save is refused with ESTALE. It doesn't quietly overwrite whatever another agent just wrote. The agent sees the error, re-reads, and tries again.

We didn't set out to build this. Our own parallel agent sessions outgrew git worktrees and started stepping on each other, so we fixed it at the filesystem layer. Anthropic's recent Scion write-up lands on a lot of the same conclusions, which was good to see.

How it works

Swarmfile shows up as a normal drive. It only pulls the byte ranges you open, so a repo bigger than your disk mounts and opens in seconds and barely touches local storage. Blocks are content-addressed (FastCDC + BLAKE3) and move over an iroh/QUIC peer-to-peer swarm: your LAN first (a warm cache on a NAS or a colleague's machine can feed the whole office), then the wider swarm, then object storage. Reed-Solomon 10+4 erasure coding means losing a machine doesn't lose your data.

It talks git when you want git: git clone and fast-forward git push through the helper, stock git-lfs straight at it, so you can leave your code on GitHub and point LFS here. When you don't want git, there are native changesets, branches, locks, and hosted CI.

What doesn't work yet

Pushes are fast-forward only, no force push. Annotated tags aren't in. Git access is off for end-to-end-encrypted projects. It's early, and we keep the gaps on one page instead of hiding them: https://swarmfile.com/docs/reference/known-limitations

Mac, Windows, Linux. Self-serve, free tier, no card: https://swarmfile.com/?utm_source=devto&utm_medium=launch&utm_campaign=launch

If you've run agent fleets or shared a build cache off a mounted drive, I'd like to hear where our concurrency model holds up or falls over.

Top comments (0)