DEV Community

Cover image for Claude Code Parallel Subagent Patterns: Fan-Out, Fan-In, and Synchronization Strategies That Actually Work
jsmanifest
jsmanifest

Posted on Originally published at jsmanifest.com

Claude Code Parallel Subagent Patterns: Fan-Out, Fan-In, and Synchronization Strategies That Actually Work

Claude Code Parallel Subagent Patterns: Fan-Out, Fan-In, and Synchronization Strategies That Actually Work

This article was written with the assistance of AI, under human supervision and review.

Most parallel subagent implementations fail because teams treat them like independent workers when they are actually interdependent processes sharing a single codebase. The result is duplicate work, context pollution, and merge conflicts that consume more time than serial execution would have taken.

The pattern that teams overlook is bounded concurrency with explicit coordination. When you dispatch five subagents to analyze different parts of a monorepo, each session sees a different snapshot of the filesystem at fork time. Without synchronization barriers and merge strategies, their outputs diverge. The orchestrator ends up reconciling contradictory recommendations or silently dropping half the work.

The fix is a three-phase pattern: fan-out with bounded slots, isolated work on non-overlapping paths, and fan-in with deterministic merge order. The orchestrator enforces concurrency limits (typically three to five subagents), assigns disjoint file sets, and collects results in dependency order before applying any writes. This eliminates race conditions and makes the parallel work actually faster than serial.

This matters because parallel subagents are the only way to handle large-scale refactors or multi-package analysis within reasonable token budgets. Serial execution hits context limits around 200K tokens. Parallel patterns let you process 1M+ token codebases by partitioning the work, but only if you control the coordination overhead.

Key Takeaways

  • Unbounded parallel subagents produce context pollution and merge conflicts that erase any speed gains.
  • The production sweet spot is three to five concurrent subagents working on disjoint file sets with explicit merge order.
  • Fan-out requires slot limits and path isolation; fan-in requires dependency-aware result collection before applying writes.
  • Synchronization barriers at phase boundaries prevent race conditions without blocking parallelism during independent work.
  • Real-world parallel patterns save 60-70% wall time on monorepo analysis, but only when merge overhead stays under 15% of total execution.

Fan-Out Pattern: Dispatching Multiple Subagents Simultaneously

The fan-out pattern dispatches multiple subagents at once, each handling a discrete slice of the overall task. The failure mode here is subtle but expensive: launching ten subagents to "analyze the codebase" without explicit boundaries creates ten overlapping contexts. Each session reads shared files, proposes conflicting changes, and the orchestrator spends more time reconciling than you saved by parallelizing.

The correct pattern enforces three constraints before dispatch. First, the orchestrator calculates a bounded concurrency limit based on token budget and task complexity. Three to five slots is the tested range for most monorepo work. Second, the orchestrator partitions the input space into disjoint sets: packages, modules, feature directories, or test suites. Third, each subagent receives an explicit scope that prevents overlap. The orchestrator holds a queue of pending tasks and releases new work only when a slot opens.

This structure prevents context pollution because no two subagents read or write overlapping files. When Subagent A works on src/auth and Subagent B works on src/api, their file reads are isolated. The orchestrator tracks which paths are in-flight and holds back tasks that would violate the disjoint guarantee until conflicting work completes.

The token budget constraint matters because each subagent session carries its own context window. If you dispatch ten subagents simultaneously on a 200K token codebase, you burn 2M tokens in parallel. The API rate limits and cost controls both fail. Bounded slots keep total active context predictable: five subagents at 40K tokens each is 200K simultaneous usage, which fits within most team quotas.

Parallel subagent orchestration dashboard

Parallel subagent orchestration dashboard

The orchestrator implementation uses a simple state machine. The queue is a priority list of pending scopes. The active set tracks running subagent sessions. When a subagent completes, the orchestrator marks that slot available and pulls the next task from the queue. The transition logic ensures you never exceed your slot limit and never double-book a file path.

Fan-In Pattern: Collecting and Merging Subagent Results

Fan-in is where most parallel implementations break. The orchestrator receives five result sets from completed subagents and needs to produce a single coherent output. The naive approach concatenates all results and hands them to a final merge session. This fails because merge complexity grows quadratically with the number of inputs. Five 10K token reports become a 50K token merge task that the final session cannot process without losing context on half the recommendations.

The working pattern is dependency-ordered collection with incremental merge. The orchestrator maintains a dependency graph of the original scopes. When src/ui depends on types from src/api, the orchestrator collects the src/api results first, applies those changes to the working tree, and then merges src/ui results against the updated state. This converts an N-way merge into N sequential 2-way merges, each of which fits comfortably in context.

interface SubagentResult {
  scope: string;
  changes: FileChange[];
  summary: string;
  dependencies: string[];
}

async function fanInResults(
  results: SubagentResult[],
  depGraph: DependencyGraph
): Promise<MergedOutput> {
  const sorted = topologicalSort(results, depGraph);
  let workingTree = await loadBaseTree();
  const mergedChanges: FileChange[] = [];

  for (const result of sorted) {
    // Apply changes to working tree in dependency order
    const resolved = await resolveConflicts(
      result.changes,
      workingTree
    );
    workingTree = await applyChanges(workingTree, resolved);
    mergedChanges.push(...resolved);
  }

  return {
    changes: mergedChanges,
    tree: workingTree,
    summary: await generateSummary(mergedChanges)
  };
}
Enter fullscreen mode Exit fullscreen mode

The resolveConflicts function is the critical piece. When Subagent A and Subagent B both modify the same import statement, the resolver uses the dependency order to pick a winner. If A ran first in topological order, its change stands and B's overlapping edit is dropped. The alternative is a merge session that reads both proposals and synthesizes a solution, but that consumes tokens and adds latency. Dependency order gives you a deterministic resolution policy for free.

The summary generation step happens after all changes are applied. This ensures the final summary reflects the actual merged state, not the sum of individual subagent summaries. Each subagent summary described what it changed in isolation. The orchestrator summary describes what changed in the final codebase after all merges, which is what the reader needs to review.

In other words, fan-in is not a concatenation operation. Fan-in is a graph traversal that applies changes in topological order and generates a single authoritative summary at the end. The difference between these two approaches is the difference between a working parallel system and one that produces garbage.

Synchronization Strategies: Coordinating Parallel Work Without Race Conditions

Synchronization barriers prevent race conditions during parallel execution. The failure mode teams hit is concurrent writes to shared state: two subagents both update package.json dependencies, and the second write silently clobbers the first. The orchestrator has no visibility into which version won, and the final state is nondeterministic.

The working pattern uses phase-based execution with explicit barriers. Phase one is read-only analysis. All subagents execute in parallel, reading files and generating recommendations without writing anything. The orchestrator blocks phase two until all phase one work completes. Phase two is write execution. The orchestrator applies changes from phase one in dependency order, with no new reads allowed. This eliminates read-write and write-write hazards because reads and writes never overlap.

The barrier implementation is a semaphore that counts completed subagents. When the count reaches the total number of dispatched sessions, the orchestrator releases the next phase. Between phases, the orchestrator can inject validation steps: running tests, checking type correctness, or verifying that no uncommitted changes exist before proceeding.

Lock-based coordination is an alternative pattern for fine-grained synchronization. The orchestrator maintains a lock table keyed by file path. Before a subagent reads or writes a file, it acquires a read lock or write lock. Read locks are shared, write locks are exclusive. This allows concurrent reads of the same file while serializing writes. The downside is lock contention overhead: if five subagents all need to read tsconfig.json, they all succeed, but if they all need to write it, four of them block.

The production tradeoff is phase-based barriers for coarse-grained tasks (analyzing separate packages) and lock-based coordination for fine-grained tasks (updating shared configuration files). The key is making the synchronization policy explicit in the orchestrator code. Implicit coordination through "hoping subagents do not collide" is how you get nondeterministic failures in production.

The verification phase at the end catches synchronization bugs that slipped through. After all writes complete, a final subagent runs type checking, linting, and tests against the merged result. If verification fails, the orchestrator can roll back to the pre-write state and retry with stricter barriers. This makes synchronization failures visible instead of silent.

Real-World Example: Parallel Codebase Analysis with Bounded Concurrency

A practical use case is analyzing a TypeScript monorepo to find unused exports across 50 packages. Serial execution would load all 50 packages into a single context and run out of tokens around package 30. Parallel execution partitions the packages into five groups of ten and runs five concurrent subagents.

interface AnalysisTask {
  packages: string[];
  maxTokens: number;
}

async function parallelUnusedExportAnalysis(
  monorepoRoot: string,
  concurrency: number = 5
): Promise<UnusedExportReport> {
  const packages = await discoverPackages(monorepoRoot);
  const tasks = partitionPackages(packages, concurrency);

  const results = await Promise.all(
    tasks.map(task => 
      analyzePackageGroup(task, monorepoRoot)
    )
  );

  return mergeAnalysisResults(results);
}

async function analyzePackageGroup(
  task: AnalysisTask,
  root: string
): Promise<PartialReport> {
  const subagent = await spawnSubagent({
    workingDir: root,
    scope: task.packages,
    maxTokens: task.maxTokens
  });

  const exports = await subagent.findExports(task.packages);
  const imports = await subagent.findImports(task.packages);

  return {
    unused: exports.filter(exp => 
      !imports.some(imp => imp.references(exp))
    ),
    scope: task.packages
  };
}
Enter fullscreen mode Exit fullscreen mode

The partitionPackages function uses package dependency graphs to create disjoint groups. Packages that import from each other stay in the same group to avoid cross-group reference checking. Independent packages split across groups for maximum parallelism. This partitioning strategy keeps each subagent task self-contained while maintaining correctness.

The merge step combines the partial reports into a single unused export list. Because each group was disjoint, there are no duplicate entries to reconcile. The orchestrator sorts the final list by package name and generates a summary showing which packages have the most dead code. The total wall time is one-fifth of serial execution because five subagents run simultaneously, and merge overhead is negligible because the partial results do not overlap.

Monorepo analysis performance comparison

Monorepo analysis performance comparison

The token budget for this pattern is predictable. Each subagent processes ten packages at roughly 8K tokens per package, so 80K tokens per session. Five sessions in parallel is 400K total tokens in-flight. The orchestrator itself uses minimal tokens to coordinate, so the total cost stays under 450K tokens for a 50-package monorepo. Serial execution would hit 500K+ tokens and fail before completion.

This pattern extends to other codebase-wide analyses: dead code elimination, dependency audits, type coverage reports, or test file generation. The key is partitioning the input space so each subagent works on a disjoint subset and the merge operation is cheap. When merge complexity starts to dominate, you have partitioned too finely and need to reduce concurrency or increase partition size.

Common Pitfalls: Duplicate Execution, Context Pollution, and Merge Overhead

The first pitfall is duplicate execution from overlapping scopes. When the orchestrator assigns src/api to Subagent A and src/ to Subagent B, both sessions analyze src/api. The orchestrator receives two reports on the same files and wastes tokens on redundant work. The fix is strict disjoint partitioning at dispatch time. If a scope overlaps with an in-flight task, the orchestrator holds it in the queue until the conflicting task completes.

Context pollution happens when subagents share mutable state. If two subagents both read a shared cache file, update it with their findings, and write it back, the last write wins and earlier results disappear. The fix is copy-on-write caches or read-only shared state with write-once outputs. Each subagent gets a private workspace for intermediate results and only the orchestrator writes to shared storage after merging.

Merge overhead kills performance when the number of subagents exceeds the natural partition count. Analyzing a monorepo with six packages using ten subagents means four subagents handle one package each and six handle one package each. The orchestrator now merges ten result sets instead of six, and the extra four provide no parallelism benefit. The fix is matching concurrency to the partition count. If you have six natural partitions, run six subagents, not ten.

The threshold where merge overhead dominates is around 15% of total execution time. If you spend 85 seconds on parallel analysis and 15 seconds merging results, you are at the limit. Increasing concurrency beyond that point increases merge time faster than it decreases analysis time. The 3-5 subagent rule comes from empirical observation that most codebases have 3-5 natural partitions (frontend, backend, shared libraries, tests, configuration) and pushing beyond that hits diminishing returns.

The implication here is that parallel patterns are not universally better than serial. They win when the input space has natural disjoint partitions, the merge operation is cheap relative to the work, and the concurrency limit fits the partition count. Forcing parallelism onto inherently sequential tasks (like applying a series of dependent type migrations) adds coordination overhead without speed gains.

Production Patterns: The 3-5 Subagent Rule and When to Break It

The 3-5 concurrent subagent limit is the tested production default for monorepo analysis, multi-package refactors, and codebase-wide search tasks. This range balances parallelism gains against merge overhead and token budget constraints. Three subagents is the minimum to see meaningful speedup. Five subagents is the maximum before merge complexity starts to eat the gains.

Breaking this rule makes sense in two scenarios. First, when the task is embarrassingly parallel with no merge step. Generating documentation for 20 independent packages can run 20 subagents because each output is a standalone file with no cross-references. The orchestrator collects 20 markdown files and writes them to disk without any merge logic.

Second, when token budget allows higher concurrency without hitting rate limits. If your team quota is 2M tokens per minute and each subagent uses 50K tokens, you can run 40 concurrent sessions before hitting the limit. The constraint shifts from token budget to orchestrator complexity. Managing 40 in-flight sessions requires robust error handling, retry logic, and progress tracking that a 5-session orchestrator does not need.

The production pattern for high-concurrency scenarios is hierarchical orchestration. A top-level orchestrator spawns five mid-level orchestrators, each managing five subagents. This keeps each orchestrator's active set manageable while achieving 25-way parallelism. The mid-level orchestrators handle local fan-in and report summaries to the top level, which does the final merge. This pattern scales to hundreds of tasks without overwhelming a single orchestrator session.

The breaking point for this pattern is when coordination overhead exceeds work time. If your orchestrator spends 60% of wall time scheduling tasks and 40% waiting for subagents, you have over-parallelized. The fix is reducing concurrency or increasing task granularity so each subagent does more work per dispatch. The goal is keeping orchestrator idle time (waiting for subagents) above 70% of total time. That indicates the parallelism is doing useful work.

A practical heuristic for setting concurrency is task count divided by four, capped at five. Ten tasks run on three subagents (10 / 4 = 2.5, round up to 3). Twenty tasks run on five subagents (20 / 4 = 5, which hits the cap). Forty tasks run on five subagents with hierarchical orchestration (top-level cap), not ten subagents (which would exceed the merge overhead threshold). This heuristic keeps merge complexity linear and orchestrator logic simple.

Frequently Asked Questions

How do you prevent two subagents from modifying the same file?

The orchestrator assigns disjoint file scopes before dispatch and tracks which paths are in-flight. If a new task would overlap an active scope, the orchestrator queues it until the conflicting task completes. During the write phase, changes are applied sequentially in dependency order so only one session touches each file.

What happens when a subagent crashes mid-execution?

The orchestrator detects the failure through promise rejection or timeout and marks that scope as available for retry. The failed task goes back into the queue with incremented retry count. After three failures, the orchestrator logs the error and continues with remaining tasks so one broken scope does not block the entire workflow.

Can subagents communicate with each other during execution?

No, subagents are isolated sessions with no direct communication channel. All coordination happens through the orchestrator. If Subagent A needs data from Subagent B, the orchestrator collects B's result first and passes it to A as input. This enforces dependency order and prevents race conditions.

How do you verify that parallel execution produced the same result as serial?

Run a final verification subagent after all merges complete. This session type-checks the merged code, runs tests, and compares the output against known-good baselines. If verification fails, the orchestrator can roll back changes and retry with serial execution as a fallback to identify which parallel merge introduced the error.

When should you use serial execution instead of parallel?

Use serial when tasks have strict sequential dependencies (migration step B requires completed migration step A), when the codebase is small enough to fit in a single context (under 100K tokens), or when merge complexity would exceed 15% of total execution time (typically when you would need more than five subagents but natural partitions are fewer than three).

Conclusion: Building Your Parallel Subagent Playbook

The essential patterns for production parallel subagents are bounded concurrency with disjoint scopes, phase-based synchronization barriers, and dependency-ordered fan-in. These three pieces eliminate the race conditions, context pollution, and merge overhead that break naive parallel implementations.

Start with the 3-5 subagent rule for monorepo analysis and multi-package refactors. Partition your input space into natural boundaries like package roots or feature directories. Enforce read-only analysis phases followed by sequential write phases. Merge results in topological dependency order, not chronological completion order.

Break the concurrency limit only for embarrassingly parallel tasks with no merge step or when you implement hierarchical orchestration for hundreds of tasks. Monitor merge overhead and pull back when it exceeds 15% of wall time. The goal is keeping orchestrator idle time above 70%, which indicates parallelism is doing real work instead of just adding coordination complexity.

That covers the essential patterns for parallel Claude Code subagents. Apply these in production and the difference will be immediate. Your monorepo analysis tasks will run in one-fifth the wall time, your token budgets will stay predictable, and your merge conflicts will disappear because disjoint scopes prevent them from happening in the first place.

Top comments (0)