Multi-Tool Execution: When One Tool Isn't Enough

Part 7 of the "From Zero to AI Agent: My Journey into Java-based Intelligent Applications" series

In our previous post, we built a query processor that could analyze natural language and execute single tools. But here's what I discovered while testing it: real users don't think in single-tool terms.

They say things like:

• "Get the weather in Denver,CO and save it to a file"
• "Create a backup folder and move all my documents there"
• "Check prices on three websites and tell me the cheapest"

Each requires multiple tools working together. Today, we're solving this by building a multi-tool orchestration system that can coordinate complex workflows automatically.

The Evolution: From Single to Multi

Let's start with what we had in Part 5. Our SimpleInference could handle this:

// ✅ This worked fine
"What's the weather in NYC?" → Single tool → weather-server:weather_query

But it couldn't handle this:

// ❌ This required hardcoding or failed entirely  
"Get NYC weather and save it to weather.txt" → ??? → Two tools needed

The user wants one fluid action, but we need two separate tool calls with coordination between them.

The MultiToolOrchestrator:

Instead of trying to cram multi-tool logic into SimpleInference, we created a dedicated orchestrator that specializes in coordinating multiple tools:

public class MultiToolOrchestrator {
    private final LLMClient llmClient;
    private final MCPService mcpService;
    private final Map<String, ToolResult> stepResults = new HashMap<>();

    public ToolResult executePlan(MultiToolPlan plan) {
        return switch (plan.getPlanType()) {
            case SEQUENTIAL -> executeSequential(plan);
            case PARALLEL -> executeParallel(plan);
            case CHAINED -> executeChained(plan);
            case COMPETITIVE -> executeCompetitive(plan);
            // Note: CONDITIONAL and ITERATIVE are partially implemented
        };
    }
}

Key insight: Rather than having one complex method that tries to handle everything, we have specialized execution strategies for different types of multi-tool scenarios.

The Four Main Execution Patterns (Currently Implemented)

1. Sequential: "Do This, Then That"

When to use: Steps must happen in a specific order, but each step is independent (no data flow between them).

Example: "Send an email to John about the meeting, then set a calendar reminder for tomorrow"

private ToolResult executeSequential(MultiToolPlan plan) throws Exception {
    List<Step> steps = plan.getSteps();

    for (Step step : steps) {
        // Check if this step's dependencies are satisfied
        if (!areDependenciesSatisfied(step)) {
            return ToolResult.error("Dependencies not satisfied for step: " + step.id());
        }

        // Execute the step
        Map<String, Object> resolvedParams = resolveParameters(step);
        ToolResult result = mcpService.callTool(step.serverId(), step.toolName(), resolvedParams);

        // Store result for later steps to use
        stepResults.put(step.id(), result);

        if (!result.success()) {
            return result; // Stop on first failure
        }
    }

    return aggregateResults(new ArrayList<>(stepResults.values()), plan.getAggregationPrompt());
}

Why this pattern: The folder must exist before we can save a file to it. Sequential execution ensures proper ordering.

2. Parallel: "Do These All At Once"

When to use: Operations are independent and can run simultaneously.

Example: "Check the weather in Austin,TX; Chicago,IL and Atlanta,GA"

private ToolResult executeParallel(MultiToolPlan plan) throws Exception {
    List<Step> independentSteps = plan.getIndependentSteps();
    ExecutorService executor = getExecutorService();

    // Launch all independent steps simultaneously
    List<CompletableFuture<Void>> futures = independentSteps.stream()
        .map(step -> CompletableFuture.runAsync(() -> {
            try {
                Map<String, Object> resolvedParams = resolveParameters(step);
                ToolResult result = mcpService.callTool(step.serverId(), step.toolName(), resolvedParams);
                synchronized (stepResults) {
                    stepResults.put(step.id(), result);
                }
            } catch (Exception e) {
                synchronized (stepResults) {
                    stepResults.put(step.id(), ToolResult.error("Parallel execution failed: " + e.getMessage()));
                }
            }
        }, executor))
        .toList();

    // Wait for all to complete
    CompletableFuture.allOf(futures.toArray(new CompletableFuture[0])).join();

    return aggregateResults(new ArrayList<>(stepResults.values()), plan.getAggregationPrompt());
}

Why this pattern: Getting weather for different cities doesn't depend on each other, so we can do them all at once for better performance.

3. Chained: "Output Becomes Input"

When to use: Steps must happen in a specific order, but each step is independent (no data flow between them)

Example: "Get the weather in NYC and save it to weather.txt in documents"

private ToolResult executeChained(MultiToolPlan plan) throws Exception {
    List<Step> steps = plan.getSteps();

    for (int i = 0; i < steps.size(); i++) {
        Step step = steps.get(i);
        Map<String, Object> resolvedParams = resolveParameters(step);

        // Automatically pipe previous result as input
        if (i > 0 && !stepResults.isEmpty()) {
            ToolResult previousResult = stepResults.get(steps.get(i - 1).id());
            if (previousResult != null && previousResult.success()) {
                resolvedParams.putIfAbsent("input", previousResult.content());
            }
        }

        ToolResult result = mcpService.callTool(step.serverId(), step.toolName(), resolvedParams);
        stepResults.put(step.id(), result);

        if (!result.success()) {
            return result;
        }
    }

    return aggregateResults(new ArrayList<>(stepResults.values()), plan.getAggregationPrompt());
}

Why this pattern: Each step transforms the data from the previous step, creating a processing pipeline.

4. Competitive: "Race for the Best Result"

When to use: Multiple tools can do the same thing, and you want the best/fastest result.

Example: "Check laptop prices on three sites and recommend the cheapest"

private ToolResult executeCompetitive(MultiToolPlan plan) throws Exception {
    ExecutorService executor = getExecutorService();

    // Run all steps in parallel and race them
    List<CompletableFuture<ToolResult>> futures = plan.getSteps().stream()
        .map(step -> CompletableFuture.supplyAsync(() -> {
            try {
                Map<String, Object> resolvedParams = resolveParameters(step);
                return mcpService.callTool(step.serverId(), step.toolName(), resolvedParams);
            } catch (Exception e) {
                return ToolResult.error("Competitive execution failed: " + e.getMessage());
            }
        }, executor))
        .toList();

    // Return first successful result
    List<ToolResult> results = futures.stream()
        .map(CompletableFuture::join)
        .toList();

    for (ToolResult result : results) {
        if (result.success()) {
            return result; // First success wins
        }
    }

    return ToolResult.error("All competitive executions failed");
}

Why this pattern: Sometimes multiple tools can accomplish the same goal, and we want the fastest or best result.

The Smart Planning: From Natural Language to Execution Plan

The deal happens in how we determine which pattern to use. We enhanced our SimpleInference with this analysis:

private String executeMultiTool(QueryAnalysis analysis, String originalQuery) throws Exception {
    if (orchestrator == null) {
        return "Multi-tool execution not available - orchestrator not configured";
    }

    // Generate a plan using specialized AI analysis
    MultiToolPlan plan = analyzeMultiToolQuery(originalQuery);

    if (!plan.isValid()) {
        return "Could not create valid execution plan for this query.";
    }

    // Execute the plan using the appropriate pattern
    ToolResult result = orchestrator.executePlan(plan);

    if (result.success()) {
        return generateToolResponse(originalQuery, "multi-tool plan", result.content());
    } else {
        // Fallback to direct answer if execution fails
        String fallbackPrompt = PromptTemplates.getFallbackPrompt(originalQuery);
        return llmClient.send(fallbackPrompt);
    }
}

The Variable Resolution

One of the interesting features is how steps can reference results from previous steps:

private String resolveVariables(String value) {
    Pattern pattern = Pattern.compile("\\$\\{([^.]+)\\.([^}]+)\\}");
    Matcher matcher = pattern.matcher(value);

    StringBuffer resolved = new StringBuffer();
    while (matcher.find()) {
        String stepId = matcher.group(1);
        String field = matcher.group(2);

        ToolResult stepResult = stepResults.get(stepId);
        if (stepResult != null && stepResult.success()) {
            String replacement = extractField(stepResult, field);
            matcher.appendReplacement(resolved, Matcher.quoteReplacement(replacement));
        }
    }
    matcher.appendTail(resolved);

    return resolved.toString();
}

This allows us to create plans like:

{
  "steps": [
    {
      "id": "get_weather",
      "toolName": "weather_query",
      "parameters": {"location": "NYC"}
    },
    {
      "id": "save_weather", 
      "toolName": "write_file",
      "parameters": {
        "path": "weather.txt",
        "content": "${get_weather.result}"  // ← Magic happens here
      },
      "dependencies": ["get_weather"]
    }
  ]
}

One Example:

Let's trace through a complete example:

User: "Get the weather in NYC and save it to weather.txt in documents"

1. Analysis Phase - LLM determines:

QueryAnalysis[
   execution=MULTI_TOOL, 
   details=CHAINED, 
   parameters={toolsInfo=weather-server:get-forecast:
     Retrieve weather data for NYC, filesystem-
     server:write_file: Save the weather data to    
     weather.txt in documents, 
     reasoning=The query requires getting the weather in NYC and saving it to a file, which involves two distinct actions: retrieving weather data and writing to a file. The output of getting the weather becomes the input for saving it to a file., 
     planName=CHAINED}, 
     multiToolPlan=Optional.empty]

2. Planning Prompt:

Create a detailed execution plan for this multi-tool 
query:"Get the weather in NYC and save it to weather.txt in documents"

         Available tools:
         - filesystem-server:read_file 
         - filesystem-server:read_multiple_files 
         - filesystem-server:write_file - 
         - .....
         - weather-server:get-forecast 

         Analyze the query and return a JSON plan with:
         - planType: "SEQUENTIAL", "PARALLEL", "CHAINED", "CONDITIONAL", "COMPETITIVE", or "ITERATIVE"
         - steps: array of objects with {"id": "unique_id", "serverId": "server_id", "toolName": "tool_name", "parameters": {}, "dependencies": []}
         - conditionPrompt: (optional) for conditional/iterative logic
         - aggregationPrompt: (optional) for result consolidation

         Guidelines:
         - Use SEQUENTIAL for dependent operations
         - Use PARALLEL for independent operations
         - Use CHAINED when output of one becomes input of next
         - Include dependencies array with step IDs this step depends on
         - Use variable substitution like ${previous_step_id.result} in parameters
         - For file paths, use RELATIVE paths only (e.g., "documents/file.txt" NOT "/documents/file.txt")
         - All paths must be relative to the current working directory
         - Do not use leading slashes (/) in file paths

 CRITICAL CONSTRAINTS:
- ONLY use tools from the "Available tools" list above - do NOT invent tools
- Use your complete knowledge base to provide any required parameters (coordinates, timezones, conversions, etc.)
- Match tool names and parameter names exactly as specified in tool descriptions
- If multiple independent operations of same type, use PARALLEL planType for efficiency

         Return ONLY the JSON plan:

3. Plan Generation:

{
  "planType": "SEQUENTIAL",
  "steps": [
    {
      "id": "step1",
      "serverId": "weather-server",
      "toolName": "get-forecast",
      "parameters": {
        "latitude": 40.7128,
        "longitude": -74.0060
      },
      "dependencies": []
    },
    {
      "id": "step2",
      "serverId": "filesystem-server",
      "toolName": "write_file",
      "parameters": {
        "path": "documents/weather.txt",
        "content": "${step1.result}"
      },
      "dependencies": ["step1"]
    }
  ]
}

4. Execution:

• Step 1: Get weather

  mcpService.callTool("weather-server", "get-forecast", {
      "latitude": 40.7128, 
      "longitude": -74.006
  })

→ Returns: "Sunny, 22°C in NYC"

• Variable resolution:

  orchestrator.resolveVariable("${step1.result}") 

→ "${step1.result}" becomes "Sunny, 22°C in NYC"

• Step 2: Save file with resolved content

  mcpService.callTool("filesystem-server", "write_file", {
      "path": "documents/weather.txt",
      "content": "Sunny, 22°C in NYC"  // resolved from ${step1.result}
  })

→ Returns: "Successfully wrote to documents/weather.txt"

• Final response generation:

  generateToolResponse(
      originalQuery: "Get the weather in NYC and save it to weather.txt in documents",
      toolName: "multi-tool plan", 
      toolResult: "Result 1: Successfully wrote to documents/weather.txt\n\nResult 2: Forecast for 40.7128, -74.006:\n\nTonight:\nTemperature: 63°F..."
  )

→ Result: "I've got the weather forecast for NYC. Tonight it's partly cloudy with a temperature of 63°F, and tomorrow is partly sunny with a high of 81°F, but there's a slight chance of rain showers. I've saved the full 7-day forecast to a file called weather.txt in your documents folder. You can check it out for more details on the rest of the week."

Chain execution flow:
weather-server:get-forecast → variable substitution → filesystem-server:write_file → response synthesis

Integration is Seamless

Our existing ChatInterface doesn't need any changes:

public ChatInterface(MCPService mcpService, LLMClient llmClient, MultiToolOrchestrator orchestrator) {
    this.mcpService = mcpService;
    this.inference = new SimpleInference(mcpService, llmClient);
    this.inference.setOrchestrator(orchestrator); // Just inject the orchestrator
    this.scanner = new Scanner(System.in);
}

Current Limitations (Being Honest)

While we've implemented the core patterns, some features are still evolving:

Partially Implemented:

• Conditional execution currently falls back to sequential
• Iterative execution has basic loop logic but limited condition checking

Testing Status:

• CHAINED and SEQUENTIAL: Thoroughly tested and working reliably
• PARALLEL: Basic implementation tested with simple scenarios
• COMPETITIVE, CONDITIONAL, ITERATIVE: Limited testing - prompt adjustments may be needed based on real-world usage patterns
• Complex multi-step scenarios: Some edge cases in variable resolution and dependency handling may require prompt refinement

Future Enhancements:

• More sophisticated error recovery
• Dynamic plan modification during execution
• Performance monitoring and metrics
• Prompt optimization based on comprehensive testing of all plan types across diverse query patterns

Note: The LLM-driven planning approach means that prompt engineering is an ongoing process. As we encounter more varied queries, we may need to refine the analysis and planning templates to handle edge cases and improve classification accuracy.

What's Next?

We've built it! An AI agent that can take natural language instructions and orchestrate workflows across multiple tools. The core architecture contains query analysis, multi-tool orchestration and some execution patterns.

But there's so much more to explore! This is just the foundation. We could add:

• Advanced error recovery strategies
• Dynamic plan modification during execution
• Performance monitoring and optimization
• More sophisticated LLM prompt engineering
• Integration with additional MCP servers
• Real-time streaming responses
• Agent memory and learning capabilities

Want to contribute?

The project is open source and we'd love your help!
The complete source code is available in our GitHub repository.

---

This is part 7 of "From Zero to AI Agent: My Journey into Java-based Intelligent Applications" series. The complete series shows you how to build AI MCP client in Java from scratch!

Follow @gazolla for more practical AI development insights!