DEV Community

Said Olano
Said Olano

Posted on

Orchestrator-Workers with Spring AI: Let the Model Decide the Subtasks

Orchestrator-Workers with Spring AI: Let the Model Decide the Subtasks

Most LLM workflows I have built start the same way. I know the steps before I write the code. I call the model once to extract something, once to transform it, once to check it, and the control flow is hard-coded in Java. That works well until the input shape changes and the steps change with it. A product description needs a different breakdown than a legal summary, and a bug report needs a different breakdown than a feature request.

The orchestrator-workers pattern addresses exactly that gap. Instead of fixing the subtasks at design time, you ask a central model to look at the task and decide what pieces of work are needed. Specialized worker calls then execute those pieces, and the results are gathered into one answer. Anthropic describes this pattern in its research on building effective agents, and the sample project in the spring-ai-examples repository implements it with Spring AI.

In this article I walk through that sample, explain the moving parts, point out where the sample is simpler than the pattern in production, and show the trade-offs. I will quote from the code where it helps and describe the rest in my own words, because the code is the source of truth and I want you to be able to check every claim against it.

Where the pattern sits

Anthropic's taxonomy of agentic workflows separates fixed workflows, where the developer writes the control flow, from agents, where the model chooses its next action. Orchestrator-workers sits between them. The shape of the flow is fixed: one orchestrator call, then a set of worker calls, then assembly of the output. The content of the flow is dynamic: the orchestrator decides what the workers should do.

That distinction matters when you choose a pattern. If you can list the subtasks before you see the input, a simple chain or a parallelization is cheaper, easier to test, and more predictable. Orchestrator-workers earns its cost when the subtasks depend on the input in ways you cannot enumerate. Coding tasks are the classic example: the number of files to edit and the kind of edit depend entirely on the request. Research tasks are another: the sources you need depend on the question.

If you have read the other articles in this series, you will recognize the family. Parallelization runs a fixed set of calls concurrently. Routing classifies the input and sends it to one specialized prompt. Orchestrator-workers combines the two ideas: the orchestrator plays the role of a planner, and the workers play the role of specialists. The difference from parallelization is that the list of workers is not known in advance.

The sample project

The project lives in agentic-patterns/orchestrator-workers. It is a Spring Boot application with spring.main.web-application-type=none, so it runs once from the command line and exits. It depends on the Anthropic starter, and the parent POM pins Spring Boot 4.1.1 and Spring AI 2.1.0-SNAPSHOT through the Spring AI BOM. The Java version is 17.

The code is split into two files. Application wires a ChatClient and runs the pattern once with a sample task. OrchestratorWorkers contains the pattern itself, including three records and two prompt templates. That separation is deliberate: the pattern class does not know about Spring Boot, so you can construct it in a test with a mocked ChatClient.

The sample task is simple on purpose: write a product description for an eco-friendly water bottle. The point is to see the shape of the flow, not to build a useful product.

The three records

The pattern is described by three small types. Reading them first makes the rest of the class easier to follow.

public static record Task(String type, String description) {
}

public static record OrchestratorResponse(String analysis, List<Task> tasks) {
}

public static record FinalResponse(String analysis, List<String> workerResponses) {
}
Enter fullscreen mode Exit fullscreen mode

OrchestratorResponse is the structured output of the planning step. It holds an analysis, a free-text explanation of how the model understood the task, and a list of Task objects. Each Task has a type, which is a label such as "formal", and a description, which tells the worker what to produce. FinalResponse is what the caller gets back: the analysis plus the list of worker outputs.

Notice what is missing. There is no separate synthesizer class. The sample returns the worker outputs as a list and leaves merging to the caller. The Javadoc and the README both mention a synthesizer as one of three components, so treat this as a simplification: in a real system you would add a final call that merges the worker outputs, and I come back to that below.

The orchestrator call

The orchestrator is a single ChatClient call whose response is mapped to a Java record:

OrchestratorResponse orchestratorResponse = this.chatClient.prompt()
        .user(u -> u.text(this.orchestratorPrompt)
                .param("task", taskDescription))
        .call()
        .entity(OrchestratorResponse.class);
Enter fullscreen mode Exit fullscreen mode

The .entity(...) call is where Spring AI does a lot of work for you. It appends format instructions to the prompt, asks the model for JSON that matches the record, and deserializes the answer. You do not write a parser. You do still need to handle the case where the model returns something unexpected, which I discuss in the failure section.

The default orchestrator prompt is written so that the model returns exactly that structure. It asks the model to break the task into two or three distinct approaches, and it gives two example entries, one formal and one conversational. In other words, the default prompt does not ask the model to decompose the task into engineering steps. It asks the model to propose variations on the output. That is a valid use of the pattern, and it is the easiest one to see working, but it is worth knowing before you copy the sample into a coding assistant. A coding orchestrator would need a prompt that asks for file-level or step-level subtasks.

The worker calls

Each subtask goes to a worker. The worker prompt is a small template with three parameters: the original task, the type of the subtask, and its description.

DEFAULT_WORKER_PROMPT = """
        Generate content based on:
        Task: {original_task}
        Style: {task_type}
        Guidelines: {task_description}
        """;
Enter fullscreen mode Exit fullscreen mode

The worker receives the original task so it does not lose context, and it receives its own type and description so it knows what to focus on. The worker returns plain text through .content(). In the sample, the calls are made one after another:

List<String> workerResponses = orchestratorResponse.tasks().stream().map(task -> this.chatClient.prompt()
        .user(u -> u.text(this.workerPrompt)
                .param("original_task", taskDescription)
                .param("task_type", task.type())
                .param("task_description", task.description()))
        .call()
        .content()).toList();
Enter fullscreen mode Exit fullscreen mode

This is an important detail. The README says the workers process subtasks in parallel, and the Javadoc describes the same behavior. The code, however, uses a sequential stream. Each worker waits for the previous one to finish. The sample is correct, but it is slower than the documentation suggests. If your orchestrator produces three subtasks and each takes four seconds, the workers take about twelve seconds in total, not four.

Making the workers concurrent is straightforward, and it is the same technique I used in the parallelization article: a fixed thread pool, CompletableFuture.supplyAsync for each task, and join on the futures in input order so the result list keeps the same order as the subtasks. The order matters because the orchestrator's type values are meaningful to the reader. I would not change the sample itself, since the sample is a teaching artifact, but in production you should parallelize the worker stage and make the order explicit.

Assembling the result

The last step is building the response:

return new FinalResponse(orchestratorResponse.analysis(), workerResponses);
Enter fullscreen mode Exit fullscreen mode

The caller receives the orchestrator's analysis and the list of worker outputs. For a demo that is enough. For a product, the assembly step is where you decide what the user actually sees. You might ask a final model call to merge the outputs into one answer, you might let the user choose among the variations, or you might run a validation pass that checks each output against a schema before anything is shown. All three are legitimate, and all three are small additions to this class.

Where the sample is simpler than production

I want to be direct about the gaps, because they are the parts that cost time when you move this pattern into a real service.

First, there is no failure handling. If the orchestrator returns JSON that does not match the record, .entity(...) throws, and the whole run fails. If one worker call times out, the sequential stream throws, and the other workers never run. In production you want a retry policy for the orchestrator call, a bounded number of tasks so a bad plan cannot fan out into dozens of calls, and a decision about what to do when one worker fails: drop it, retry it, or fail the whole run.

Second, there is no limit on the number of subtasks. The default prompt asks for two or three, but the model is not bound by that. Since each subtask is a model call, an unbounded plan is an unbounded bill. Validate the size of tasks() after the orchestrator call and reject plans that exceed your budget.

Third, there is no synthesizer. As noted above, the caller gets a list. That is fine for variations, but for a multi-file coding task you would need a step that merges the edits without conflicts, and that step is a hard problem on its own.

Fourth, the System.out.println calls print the orchestrator output and the worker output to standard output. That is useful when you are learning the pattern and noisy when you are not. Replace them with a logger and log the plan and the worker count, not the full text, if the outputs contain user data.

A parallel worker stage

Here is a sketch of the worker stage running concurrently. It keeps the same contract as the sample: the orchestrator's tasks go in, and the outputs come back in the same order.

ExecutorService pool = Executors.newFixedThreadPool(4);
try {
    List<CompletableFuture<String>> futures = orchestratorResponse.tasks().stream()
            .map(task -> CompletableFuture.supplyAsync(() -> this.chatClient.prompt()
                    .user(u -> u.text(this.workerPrompt)
                            .param("original_task", taskDescription)
                            .param("task_type", task.type())
                            .param("task_description", task.description()))
                    .call()
                    .content(), pool))
            .toList();

    List<String> workerResponses = futures.stream().map(CompletableFuture::join).toList();
} finally {
    pool.shutdown();
}
Enter fullscreen mode Exit fullscreen mode

Two details matter here. The pool size is a limit you choose, so a plan with many tasks does not open an unbounded number of connections to the model provider. And join is called in the order of the futures, not the order in which the calls finish, so the output list always lines up with the orchestrator's task list. If one worker throws, join rethrows that exception, which is the place to add the failure policy discussed above.

Testing the pattern

The pattern class is easy to test because it takes a ChatClient in its constructor. You can create a ChatClient backed by a stub ChatModel that returns a fixed orchestrator JSON and fixed worker strings, then assert on the result. That lets you test the control flow without spending tokens:

  • The number of worker calls equals the number of tasks in the plan.
  • The order of workerResponses matches the order of tasks.
  • Empty or blank task descriptions are rejected by the Assert calls.
  • A malformed orchestrator response fails in a predictable way.

Testing the prompts is a separate problem. The default prompts are tuned by hand, and the way to check them is with a small set of representative inputs and a review of the plans the orchestrator produces. Keep those inputs in version control, and rerun them when you change a prompt, the model, or the model version.

When to choose this pattern

I use three questions to decide.

Can I list the subtasks before I see the input? If yes, use a chain or a parallelization. They are simpler and cheaper.

Does the right breakdown change with each input in a way that matters to the output? If yes, orchestrator-workers is a good fit.

Can I afford the extra calls and the extra latency? The orchestrator adds one call before any work begins, and the workers add one call per subtask. If the task is short and the answer must be fast, a single well-written prompt may beat the pattern.

There is also a middle option that I often prefer: use a router to pick between two or three fixed plans, and only use an orchestrator inside the branch that really needs dynamic planning. That keeps the common path cheap and keeps the flexible path for the cases that need it.

Running the sample

To run the sample you need Java 17 or later, Maven or the included wrapper, and an Anthropic API key in the ANTHROPIC_API_KEY environment variable. The application reads the key through spring.ai.anthropic.api-key=${ANTHROPIC_API_KEY}. Start it with the Maven wrapper, and the orchestrator will print its analysis and the planned tasks, then the worker outputs, then exit. Change the sample task in Application to try a different kind of request. A task that naturally splits into several perspectives, such as "explain the trade-offs of event sourcing for a payments service", will show the pattern more clearly than the product description.

Key takeaways

  • Orchestrator-workers lets a model decide the subtasks at runtime, while the code still controls the overall flow.
  • In Spring AI, the orchestrator is a structured call mapped to a record with .entity(...), and each worker is a plain text call with .content().
  • The sample runs workers sequentially even though its documentation describes parallel execution. Parallelize the worker stage in production.
  • The sample has no failure handling, no limit on subtasks, and no synthesizer. Add all three before you rely on it.
  • Choose the pattern when the right subtasks depend on the input. Choose a simpler pattern when you can list them in advance.

The full sample is in the spring-ai-examples repository under agentic-patterns/orchestrator-workers. The source of the pattern is Anthropic's research on building effective agents, which describes the orchestrator-workers workflow and the other patterns I have covered in this series.

Top comments (0)