DEV Community

Dipankar Sarkar
Dipankar Sarkar

Posted on

Running five coding agents in parallel is easy. Not losing their state when one crashes is the hard part

Point one AI coding agent at your repo and it mostly works. Point five at it and the
problem stops being the agents and starts being the coordination. Who is working on
what. What happens when agent three's session dies mid-task. Where the state lives so
that a crash does not lose the plan. How the finished work merges back without
stepping on itself.

Most setups answer those questions with a JSON file in the working tree and a hope.
The file gets half-written when a process is killed. The working tree gets dirty and
pollutes code review. Two agents edit the same state and one wins silently.

brat is a Rust multi-agent harness built to answer those questions properly. The
core promise: even if agents crash, your coordination state is always recoverable and
auditable.

The core idea: coordination is an append-only event log

Brat does not keep a mutable state file. It is built on grite, an append-only event
log stored in git refs (refs/grite/wal). Every change to coordination state is an
event appended to that log. State is not stored, it is derived by replaying events.

That one decision is where the crash-safety comes from. An append-only log has no
half-written mutable record to corrupt. If a process dies mid-operation, you rebuild
deterministically from the last known-good point. And because the log lives in git
refs, not tracked files, your working tree stays clean. No coordination JSON showing
up in git status, no metadata noise in code review.

The problems it explicitly targets, from its own list:

Problem How Brat fixes it
Dirty working trees Metadata lives in refs/grite/*, never in tracked files
Silent failures All state changes recorded as events, fully observable
Crash recovery Append-only log enables deterministic rebuild from any point
Merge chaos The Refinery manages a queue with configurable policy

How it works: a small cast of roles

Brat models the work as a set of roles, and once you see them the mental model clicks:

  • Mayor is the AI orchestrator. It analyzes the codebase, breaks down the work, and creates convoys and tasks.
  • Convoy is a group of related tasks. Think sprint, epic, or feature branch.
  • Task is a single work item assigned to one coding agent.
  • Witness spawns and monitors the agent sessions.
  • Refinery manages the merge queue, runs CI checks, and handles integration.
  • Deacon is the background janitor: it cleans locks, syncs state, and detects orphaned sessions.

The flow reads top to bottom. The Mayor creates a Convoy that contains Tasks. The
Witness spawns agents against queued tasks. The Refinery merges finished work. The
Deacon keeps the whole thing from leaking locks and orphans.

Underneath, the substrate does the unglamorous correctness work. Events are immutable.
Each agent (actor) gets its own isolated data directory, so one agent's writes do not
clobber another's. Engine operations have bounded, configurable timeouts, so a hung
agent does not hang the harness. Resource coordination uses TTL-based lease locks, so
a crashed agent's lock eventually expires instead of deadlocking everyone.

And it is engine-agnostic. Brat drives your preferred coding tool through an adapter:

Engine Command
Claude Code claude
Aider aider
OpenCode opencode
Codex codex
Continue cn
Gemini gemini
GitHub Copilot gh copilot

You pick the engine in .brat/config.toml and Brat handles the orchestration around
it.

A concrete run

The loop is deliberately small. Initialize the substrate and the harness, start the
Mayor, ask it to analyze code, then let the Witness run the tasks it created.

cd your-project
grite init     # initialize the grite substrate
brat init      # initialize the Brat harness

# start the AI orchestrator and give it a job
brat mayor start
brat mayor ask "Analyze src/ and create tasks for any bugs you find"

# see what it created
brat status

# spawn agents for the queued tasks
brat witness run --once
brat status --watch
Enter fullscreen mode Exit fullscreen mode

Because coordination is events in git, brat status is not reading a fragile local
file, it is querying derived state that can be rebuilt from the log at any time.

You can also declare reusable workflows. A parallel convoy is just a set of legs that
fan out and a synthesis step that pulls the results together:

name: code-review
type: convoy
legs:
  - id: correctness
    title: "Review correctness"
  - id: security
    title: "Review security"
  - id: performance
    title: "Review performance"
synthesis:
  title: "Synthesize review findings"
Enter fullscreen mode Exit fullscreen mode

Three agents review three different concerns in parallel, and a fourth synthesizes.
That is the shape multi-agent work actually wants, and it maps cleanly onto convoys
and tasks.

Where it does not fit

Brat ships its own honest "what it does not solve" list, which is the main reason I
trust it. Repeating it, because the trade-offs are the point.

  • It does not fix engine reliability. API rate limits, auth failures, and vendor
    outages are outside Brat's control. If Claude or GPT is down, Brat cannot conjure a
    response. It can recover the coordination state around the failure, not the failure
    itself.

  • It does not resolve real merge conflicts. The Refinery manages the merge queue
    and policy. Genuine code conflicts still need human judgment. Brat orders and gates
    the merges, it does not understand your code well enough to reconcile two
    contradictory diffs.

  • It does not write your prompts. Brat orchestrates agents. Prompt quality is
    still your job. Point it at a vague task and you get vague work, coordinated
    cleanly.

  • It does not replace your CI. Brat integrates with your existing CI, it does not
    become it.

  • It is early and Rust-native. You need the Rust toolchain to build from source,
    and you install grite first as a prerequisite. This is infrastructure for people
    who want to run several agents seriously, not a one-click consumer app.

The pattern across that list: Brat is honest about being a coordination substrate, not
a magic wand. It makes the state crash-safe and auditable. It does not make the agents
good.

Takeaways

  • The hard part of multi-agent coding is not parallelism, it is state. An append-only event log in git refs makes that state crash-recoverable and keeps your working tree clean.
  • Roles (Mayor, Convoy, Task, Witness, Refinery, Deacon) give you a mental model that matches how the work actually decomposes.
  • Actor isolation, bounded timeouts, and TTL lease locks are the boring correctness details that decide whether a harness survives contact with a crash.
  • Believe a project more when it publishes what it does not solve. Brat does.

Code, the role docs, and the demo are here:
https://github.com/neul-labs/brat

If you are already running multiple coding agents, I want to know how you handle a
mid-task crash today. Kick the tyres, issues welcome.

Top comments (3)

Collapse
 
komo profile image
Reid Marlow

The git-ref event log is the right boring primitive here. The hard part I’d want surfaced is recovery confidence: what was rebuilt from the log, what agent context is gone, and which merges need human review before the convoy keeps moving. Do you expose that as a normal status state, or only when crash recovery happens?

Collapse
 
raju_dandigam profile image
Raju Dandigam

Putting coordination state in an append-only log instead of a mutable working-tree file is the important design choice here. Once multiple coding agents are involved, crash recovery and auditability matter more than the scheduler itself, and storing the WAL in git refs is a clever way to keep coordination metadata out of code review while still making replay deterministic.

That lines up with why we built agent-inspect around local-first execution receipts instead of conversational summaries. When an agent dies mid-run, you need something you can replay and inspect, not just what the orchestrator remembers saying about it.

How are you thinking about compaction once the event log gets large across long-lived repos?

Collapse
 
distilled profile image
Charles Solar

we resume an interrupted run from a checkpoint json that's written periodically. after a hard crash there's often no checkpoint at all, since a kill -9 or an OOM means the shutdown hook never ran, so the session gets rebuilt from the event log instead, parsed line by line so one corrupt record doesn't sink the rest.

that rebuild doesn't recover everything. which agents were still mid-task comes back; workflow execution state and the full context snapshot don't, and the resumed run is marked crash recovery rather than a clean stop.