DEV Community

Devanshu Biswas
Devanshu Biswas

Posted on

Orchestrator–Workers: let a lead model plan the subtasks at runtime instead of hardcoding them

#ai #llm #promptengineering #beginners

The previous pattern I built, parallelization, fans one input out to a fixed set of subtasks you wrote in advance — headline, features, FAQ, tweet, always those four. But you don't always know the pieces until you see the input: a bug fix wants diagnose → test → patch, a comparison wants one worker per competitor you decide is relevant, a trip wants a track per day. Orchestrator–Workers makes the decomposition itself a model call. I built an interactive walkthrough of the loop; here's the idea and the whole thing in six short pieces.

The fixed fan-out you're upgrading

First, what almost works: parallelization with a hardcoded subtask list. You wrote the slots before you ever saw the input, and every task gets the same ones — fine, until the task needs two subtasks, or seven, or entirely different ones. The decomposition is frozen in your source.

// ❌ Fixed: you hardcode the subtasks BEFORE seeing the task.
const SUBTASKS = ["headline", "features", "faq", "tweet"];  // always these 4
const parts = await Promise.all(SUBTASKS.map(s => llm(promptFor(s) + input)));
return parts.join("\n\n");
Enter fullscreen mode Exit fullscreen mode

The orchestrator plans the subtasks at runtime

The one new idea. Instead of a constant list, ask a lead model to produce the plan from the task, as structured JSON. The model decides how many subtasks and which — a role and a goal for each. Decomposition becomes a call, so it adapts to the input.

const plan = await llm(`
  You are an orchestrator. Break this task into the RIGHT
  number of independent subtasks — no more, no fewer.
  Return JSON: { "subtasks": [ { "role": "...", "goal": "..." } ] }
  TASK: ${task}
`);
// the model decides HOW MANY and WHICH — not you, not up front.
Enter fullscreen mode Exit fullscreen mode

Parse and guard — the plan is data you don't control

A model-authored plan is untrusted data, so guard it before acting on it. Parse the JSON, reject an empty plan, and cap the worker count so a runaway orchestrator can't fan out fifty calls. After this, N is whatever the task genuinely needed.

const { subtasks } = JSON.parse(plan);
if (!Array.isArray(subtasks) || subtasks.length === 0)
  throw new Error("orchestrator returned no plan");
const capped = subtasks.slice(0, MAX_WORKERS);   // don't fan out 50
Enter fullscreen mode Exit fullscreen mode

Dispatch the workers — parallel fan-out

Now it's just parallelization: fan the planned subtasks out with Promise.all, so they run at once and the wall clock is the slowest one. Each worker is a focused specialist that gets its role, its goal, and the shared context. The mechanic is identical to the fixed version; the only change is that the list came from the model.

const results = await Promise.all(
  capped.map(st => llm(`Role: ${st.role}. Goal: ${st.goal}. Context: ${task}`))
);
Enter fullscreen mode Exit fullscreen mode

The synthesizer reconciles — it doesn't just concatenate

Fixed sectioning could get away with join() because you designed the slots to fit together. A dynamic plan can't — the workers didn't know about each other, so their outputs overlap, contradict, or jar. The synthesizer is a final model pass that stitches them into one coherent whole: resolve overlaps, fix transitions, keep one voice.

const final = await llm(`
  Combine these worker outputs into one coherent result.
  Resolve overlaps, fix transitions, keep one voice.
  ${results.map((r, i) => `#${i + 1}: ${r}`).join("\n")}
`);
return final;   // more than concat — it stitches N pieces into a whole.
Enter fullscreen mode Exit fullscreen mode

Put it together, and know the cost

Three moves: plan → dispatch → synthesize.

async function orchestrate(task) {
  const plan   = await planSubtasks(task);      // 1 lead call, dynamic N
  const pieces = await runWorkers(plan, task);  // N worker calls, parallel
  return await synthesize(pieces, task);        // 1 synthesis call
}
Enter fullscreen mode Exit fullscreen mode

The demo makes the payoff concrete: the orchestrator plans a different-sized plan for each task — three workers for a bug fix, four for an app comparison, five for a trip, two for a tiny refactor — while a rigid 4-slot template invents empty "Examples"/"Summary" slots for the small job and comes up short for the big one. You pay for the flexibility: it costs N + 2 calls (the plan and the synthesis are the price of being dynamic), and the orchestrator can over- or under-decompose, so cap N and keep the plan reviewable. The workers also can't see each other, so truly dependent steps still belong in a chain.

Reach for it when you genuinely can't know the subtasks up front — coding agents, deep research, multi-file edits. When you can, a fixed fan-out is cheaper. Next I'll close the loop with an evaluator that grades the output and sends it back for another pass.

Pick a task and watch the orchestrator size the plan to it — then flip to the fixed split to see it misfit:
https://dev48v.infy.uk/prompt/day38-orchestrator-workers.html

Top comments (0)