DEV Community

Cover image for Running Coding Agents in Parallel with Git Worktrees
Josep
Josep

Posted on

Running Coding Agents in Parallel with Git Worktrees

#git #ai #productivity #devops

Watch out for shared stashes and disk bloat

I kept hitting the same wall with coding agents. One Claude Code or Codex session in a repo works great. The moment I wanted two tasks moving at once - login in one terminal, payments in another - they started stepping on each other. Same working directory, same checked-out branch, two processes editing the same files. Chaos.

Traditional workflow: each developer on their own machine, one working copy, one task at a time; feature branches meet in develop and main via PRs

The fix turned out to be a Git feature that has been sitting there for years: git worktree. It gives you several working directories backed by the same repository. Each folder has its own checked-out branch, but all of them share the same objects, commits and branch list.

The setup

From your main checkout:

git worktree add ../integration -b integration main
git worktree add ../feature-login -b feature/login main
git worktree add ../feature-payments -b feature/payments main
Enter fullscreen mode Exit fullscreen mode

Which leaves you with something like:

project/
├── main/               → branch main
├── integration/        → branch integration
├── feature-login/      → branch feature/login
└── feature-payments/   → branch feature/payments
Enter fullscreen mode Exit fullscreen mode

Now every agent gets its own folder. One terminal per worktree, one agent per terminal, and nobody touches anybody else's files:

cd feature-login    # agent 1 works here
cd feature-payments # agent 2 works here, at the same time
Enter fullscreen mode Exit fullscreen mode

Workflow with agents and worktrees: one shared repo with main, integration and feature worktree folders, one agent per folder, all in parallel on a single machine

The part that surprised me: no push, no pull

My first instinct was: agent finishes login, pushes the branch, then I pull it into integration. That's the muscle memory from working in a team.

It's unnecessary here. All the worktrees belong to the same repository on the same machine, so Git already knows every branch locally. When agent 1 finishes:

cd feature-login
git add .
git commit -m "feat: implement login"
Enter fullscreen mode Exit fullscreen mode

...the integration worktree can merge it directly:

cd ../integration
git merge feature/login
git merge feature/payments
npm test
Enter fullscreen mode Exit fullscreen mode

No git push, no git pull. The directories are different, but feature/login and integration are branches of the same repo. When integration is green:

cd ../main
git merge integration
Enter fullscreen mode Exit fullscreen mode

You don't even have to wait for a worktree to be deleted before merging its branch. Both folders can exist at the same time. The only thing Git refuses is having the same branch checked out in two worktrees at once - which is exactly the protection you want when agents are involved.

Cleaning up

When a feature is done, resist the rm -rf:

git worktree remove ../feature-login
git branch -d feature/login
Enter fullscreen mode Exit fullscreen mode

git worktree remove deletes the folder and cleans Git's internal registry. If you already nuked the folder by hand, git worktree prune fixes the bookkeeping after the fact.

The full cycle for one task ends up being:

git worktree add ../feature-login -b feature/login main   # create
cd ../feature-login                                       # agent works, commits
cd ../integration && git merge feature/login && npm test  # integrate
git worktree remove ../feature-login                      # discard workspace
git branch -d feature/login                               # discard branch
Enter fullscreen mode Exit fullscreen mode

So where does GitHub fit?

For the agents to collaborate on one machine: nowhere. Git itself is the communication mechanism.

GitHub earns its place the moment humans enter the loop - PRs, review, CI, an audit trail:

cd feature-login
git push -u origin feature/login   # open a PR from this
Enter fullscreen mode Exit fullscreen mode

And if you want integration to merge exactly what's on the remote rather than the local branch:

git fetch origin
git merge origin/feature/login
Enter fullscreen mode Exit fullscreen mode

I now think of it as two separate planes. Locally: worktree → commit → merge into integration → tests. Remotely: push → PR → review → CI. The second plane is for people; the first one is where the agents actually work.

The detail that sold me on the whole thing: the integration agent never copies code from the other agents, never reads their folders, doesn't even know they exist. It only knows their branch names. Git was the message bus all along.

Bonus: when every agent wants port 3000

Worktrees isolate files and branches. They don't isolate runtime. The moment two agents each run npm run dev and a Playwright suite, they all reach for :3000, the same test database, and a surprising amount of RAM per headless browser.

The workaround I use today is boring: each worktree gets its own port through an env file written at creation time, and Playwright reads it for its baseURL:

echo "PORT=3101" > ../feature-login/.env.local
echo "PORT=3102" > ../feature-payments/.env.local
Enter fullscreen mode Exit fullscreen mode

But ports are only the visible part of the problem. Isolating test databases, keeping three browser suites from eating the machine, and deciding whether integration should be the only worktree allowed to run E2E at all - that deserves a post of its own. It's the next one I'm writing.

So in some cases is better to use isolated agents on VPS and i have another article that talks about that https://dev.to/servatj/the-ai-dev-environment-nobody-teaches-you-written-by-my-ai-agent-directed-by-me-p--1dg4

Top comments (8)

Collapse
 
reidmarlow profile image
Reid Marlow •

The other friction point with parallel worktrees on a single machine is untracked build artifacts and node_modules. If three worktrees each run their own full install, disk usage jumps immediately. Scoping local test databases directly to the assigned worktree port keeps the setup lightweight without needing separate VMs.

Collapse
 
servatj profile image
Josep •

Really good point thanks!

Collapse
 
vinhnguyenthanhdn profile image
Vinh Nguyen •

Worktrees isolate the working directory and the index, but not the stash — refs/stash is one stack in the shared repo. Tried it on git 2.50.1: git stash push in the feature/login worktree, then git stash list from feature/payments shows On feature/login: login wip, and popping there applies the login changes into the payments tree and drops the entry, so agent 1's work is gone from where it was left. Worth telling the agents to commit on their own branch rather than stash, since stash is one of the first things a coding agent reaches for when it wants to park something.

Collapse
 
servatj profile image
Josep •

Oh really good caveat to have in mind thanks!

Collapse
 
glenallen profile image
Glen Allen •

The interesting part isn't just the file isolation, but the fact that Git becomes the coordination boundary between agents. At IT Path Solutions, we’ve found that giving each agent an isolated workspace while keeping commits as the integration handoff makes parallel AI workflows much easier to inspect and manage. Each agent can work independently, while the integration layer still has a clean, reviewable history instead of relying on shared files or implicit communication.

Collapse
 
unitbuilds profile image
UnitBuilds •

The problem is what if it's not clean cut? Take finance for instance, you use a shared standard, eg. for Round(), you'd set a hard definition that's reproducible indefinitely. You only call that Round(), never maths.Round(). Or a shared GUI element, like a customized dialog. If 2 agents need to make changes to the same file, for different reasons and different outcomes, it's a mess to detangle it, because telling them to just 'solve the merge conflicts', leads to 1 side working and the other partially broken. That's why while for standalone additions, worktrees works great and multi-agent orchestration is effective, it falls apart in the real world when the codebase is more intertwined. Take the past week, I've been working on reports, while another person implemented a cargo split, which changed the base-class structure of that entire sector... So I had to essentially redo everything again for that entire section at 5pm, because their commit broke mine. It sounds easy, 'just let the AI handle it', till you calculate the token cost it takes to resolve. You need to take both worktrees' work into account, along with all related systems... That's alot... And it can very easily blow through your budget if it's widespread.

Collapse
 
jkming profile image
jkming •

Running two agents this way too. One gotcha we hit early: each worktree needs its own node_modules, otherwise one agent's npm install mid-run breaks the other's build. We ended up with a bootstrap step that installs on worktree creation. Curious how you handle the integration merge when both agents touched the same file - serialize the merges, or just let the conflict land on you?

Collapse
 
servatj profile image
Josep •

I ask the AI to solve those conflicts if they appear, then once everything is merge on integration branch and all the tests pass. Then the repo is deployed in an integration env where I test and check the acceptance criteria meets what I requested. I constantly ask the AI also to document, tests, create proofs like videos, screenshots etc , through ai pipelines, so I don't have to review everything all the time.