DEV Community: Venkata

Orchestrating Multi-Agents: Unifying Fragmented Tools into Coordinated Workflows

Venkata — Mon, 30 Jun 2025 06:18:36 +0000

Previous posts on MCP and Agents:

Fragmented Tools

Development teams are deploying specialized AI tools across different vendors, architectures, and environments. These tools exist in silos, creating operational complexity and limiting their collective potential.

As AI adoption accelerates and the number of deployed agents multiplies, a new challenge emerges: how do we coordinate these specialized tools to work together effectively?

Agent Orchestrator

The answer lies in agent orchestration - Agent orchestration coordinates multiple specialized AI agents within a unified system to efficiently achieve shared objectives. It also helps in enabling collaboration with third-party agents to solve complex problems and aid decision-making. Think of Agent Orchestrator as a senior engineer routing work to team members with right skills to get the job done.

Use Cases

Scenarios where agent orchestration delivers impact include:

Autonomous Incident Coordination

When a complex deployment issue arises, the orchestrator might coordinate between a monitoring agent that identifies the problem, a diagnostic agent that analyzes logs, and an automation agent that implements the fix—all without human intervention.

Multi-Cloud Disaster Recovery/Notification

When a primary cloud region experiences an outage, the orchestrator coordinates between a health monitoring agent that detects the failure, a backup verification agent that confirms data integrity, a DNS routing agent that redirects traffic, and a notification agent that alerts stakeholders—executing a complete failover strategy across multiple cloud providers seamlessly.

Building the Orchestration Framework

Implementing effective agent orchestration requires three foundational components:

Contextual Workflow Orchestration

The orchestrator manages task dependencies and execution flow, ensuring each agent hands off contextually relevant output to the next. This coordination enables seamless, efficient progression through complex workflows.

Intelligent Data Pipelines for Agents

Seamless data flow is essential for orchestrated agents. The orchestrator allows agents to access shared data sources, exchange information, and maintain consistency across operations.

Open Source Orchestration Technologies

Open-source orchestration frameworks provide scalable, interoperable building blocks for coordinating agents. These technologies offer standardized protocols for agent communication, monitoring capabilities for orchestrated workflows. An example framework for agent-to-agent communication is Google’s A2A protocol.

Main Components of the Orchestrator

Most of the orchestrators are composed of the below core components

Orchestrator

Central coordinator managing interactions between classifiers, agents, storage, and retrievers. It processes user input, directs agent workflows, and handles errors and fallbacks.

Classifier

Analyzes user input, agent metadata, and conversation history to select the most suitable agent.

Agents

Perform tasks based on classification. Includes prebuilt, customizable, and fully custom agents tailored to specific needs.

Conversation Storage

Stores conversation history at both classifier and agent levels.

Retrievers

Fetch relevant external context to enhance agent performance.

Agent Orchestrator Architecture

To illustrate how these components work together, let's trace through a typical deployment scenario. When a developer requests "Deploy new API version and monitor for anomalies," the orchestration system coordinates multiple specialized agents through a structured workflow:

The orchestrator manages this through a sequential communication flow between agents:

Parse Request - The orchestrator's classifier analyzes the developer's request
Deploy API - Routes to the Deployment Agent for CI/CD pipeline execution.
Start Monitoring - Automatically triggers the Monitor Agent for metrics
Analyze Logs - If anomalies detected, activates the Log Agent for analysis
Report Status - Provides consolidated status updates back to the developer

This A2A (Agent-to-Agent) communication ensures seamless handoffs between specialized agents while maintaining context throughout the entire workflow.

High-Level Implementation

Below is a high-level implementation demonstrating how an orchestrator can coordinate tasks across multiple agents.

import json
from typing import List, Dict, Any

class Agent:
    def __init__(self, name: str, description: str, tools: List, model: str):
        self.name = name
        self.description = description
        self.tools = tools
        self.model = model

    def execute_task(self, task_input: str) -> str:
        # Agent-specific task execution logic
        for tool in self.tools:
            if tool.can_handle(task_input):
                return tool.execute(task_input)
        return f"{self.name} processed: {task_input}"

class DevOpsOrchestrator:
    def __init__(self, agents: List[Agent]):
        self.agents = {agent.name: agent for agent in agents}
        self.context_history = []

    def classify_intent(self, user_input: str) -> Dict[str, Any]:
        """Use LLM to determine which agent should handle the request"""
        context = "\n".join(self.context_history[-5:])

        available_tools = {", ".join([f"* {name}: {agent.description}" for name, agent in self.agents.items()])}

        expected_structure = {"agent": "", "task": "", "follow_up": ""}

        prompt = f"""
        You are a specialized workflow coordinator for infrastructure operations.
        Leverage the conversation history to determine optimal task routing.

        Previous Conversations:
        {context}

        Registered tools with capabilities: {available_tools}

        Current Request: {user_input}

        ###Instructions###
        - Examine the request and identify the best-suited agent
        - Transform the request into clear instructions for the chosen agent
        - For complex workflows requiring multiple agents, specify the initial agent and indicate continuation needs
        - When no agent matches or task is complete, use "user_response" as the agent
        - Output must be valid JSON matching: {expected_structure}

        """

        # Mock LLM response (replace with actual LLM call)
        llm_response = self.query_llm(prompt)
        return json.loads(llm_response)

    def query_llm(self, prompt: str) -> str:
        """Mock LLM query - replace with actual implementation"""
        # This would call your actual LLM (OpenAI, Anthropic, etc.)
        return '{"action": "deployment_agent", "input": "deploy new version", "next_action": "monitoring_agent"}'

    def orchestrate(self, user_input: str) -> str:
        self.context_history.append(f"User: {user_input}")

        # Classify intent and route to appropriate agent
        routing_decision = self.classify_intent(user_input)
        agent_name = routing_decision["agent"]
        task = routing_decision["task"]

        if agent_name in self.agents:
            result = self.agents[agent_name].execute_task(task)
            self.context_history.append(f"{agent_name}: {result}")
            return result

        return "No suitable agent found for this request"

# Usage Example
deployment_agent = Agent(
    name="deployment_agent",
    description="Handles application deployments and releases",
    tools=[DeploymentTool()],
    model="gpt-4o-mini"
)

monitoring_agent = Agent(
    name="monitoring_agent", 
    description="Monitors system health and processes alerts",
    tools=[MetricsTool(), AlertTool()],
    model="gpt-4"
)

# Create orchestrator
orchestrator = DevOpsOrchestrator([deployment_agent, monitoring_agent])

# Execute coordinated workflow
result = orchestrator.orchestrate("Deploy the new API version and monitor for errors")

Considerations and Challenges

While agent orchestration offers significant benefits, implementation teams should consider few key challenges

Complex Coordination: Managing workflows across multiple agents increases complexity and risk of failure.
High Costs: More agents mean higher compute and integration overhead, often leading to unclear ROI.
Context Management: Maintaining consistent state across agents is difficult due to token limits and fragmented memory.
Security Risks: Inter-agent communication and API exposure widen the attack surface and raise privacy concerns.

In Summary

Agent orchestration isn’t just about automation—it’s about designing intelligent, adaptive systems where specialized agents work together seamlessly. By coordinating these agents in real time, teams can unlock the following benefits:

Eliminate manual handoffs through automated, intent-driven task delegation.
Streamline complex workflows via a unified, orchestrated interface.
Scale with ease by integrating new agents as capabilities evolve

As agent-based architectures continue to mature, orchestration will be the key to unlocking their full potential.

References and Frameworks

MCP Client Agent: Architecture & Implementation, Integration with LLMs

Venkata — Sun, 18 May 2025 14:14:27 +0000

This is a follow up article after a brief introduction to MCP (Model Context Protocol).

In this post we will go much deeper into an overall Architecture and MCP Client flow as well as implement an MCP Client Agent.

…and hopefully provide a clarity on ‘What happens when you submit your request to MCP powered with LLMs”

There are a bunch of articles/posts on how to implement MCP Servers, for reference here is an official example from MCP website. In this article we will only focus on implementing an MCP Client agents that can programmatically connect to the MCP servers.

High-Level MCP Architecture

Architecture image from modelcontextprotocol.io

MCP Components

Host: AI Code editors (like Claude Desktop or Cursor) that users directly interact with, serving as the main interface and system manager.
Clients: Intermediaries that maintain connections between hosts and MCP servers, handling communication protocols and data flow.
Servers: Components that provide specific functionalities, data sources, and tools to AI models through standardized interfaces

Without delaying further lets get to the core of this article.

What are MCP Client Agents?

Custom MCP Clients: Programmatically invoking MCP Servers

Most of the use cases we have seen so far are about using MCP in an AI powered IDE. Users configure MCP servers in the IDEs and use its chat interface to interact with MCP Servers, here the chat interface is the MCP Client/Host.

But what if you would want to programmatically invoke these MCP servers from your services? This is the real advantage of MCPs, which is a standardized way to provide context and tools to your LLMs, and so we don’t have to start implementing code to integrate with all External APIs, Resources or files, and instead start providing the context and tools and send them to LLM for intelligence.

MCP Client Agent Flow w/Multi MCP Servers

The diagram illustrates how MCP Custom Clients/AI agents process user requests through MCP servers. Below is a step-by-step breakdown of this interaction flow:

Step 1: User Initiates Request

User asks a query or submits a request either through an IDE, or browser or terminal
Query is received by the Custom MCP Client/Agent interface.

Step 2: MCP Client & Server Connection

MCP Client connects to the MCP Server. It can connect to multiple servers at a time and requests for tools from these servers
Servers send back the supported list of tools and functions.

Step 3: AI Processing

Both user query and tools list are sent to the LLM (e.g., OpenAI)
LLM analyzes the request and suggests appropriate tool and input parameters and sends back response to MCP Client

Step 4: Function Execution

MCP Client calls the selected function in MCP Server with the suggested parameters.
MCP Server receives the function call and processes the request, depending on the request the corresponding tool in a specific MCP Server will get called. Please note to make sure the tool names across your MCP servers are different to avoid LLM hallucination and non-deterministic responses.
Server may interact with databases, external APIs, or file systems to process the request

Step 5: (Optional) Improve Response using LLM

MCP Server returns the function execution response to MCP Client.
(Optional)
- MCP Client can then forward that response to LLM for refinement
- LLM converts technical response to natural language or creates a summary

Step 6: Respond to User

Final processed response is sent back to the user through the client interface
User receives the answer to their original query

Custom MCP Client Implementation / Source Code

Connect to MCP Servers: As described above an MCP client can connect to multiple MCP servers, and we can simulate the same in Custom MCP Client.
Note: To avoid over hallucination and get fixed results it is recommended to not have collision among tools across these multiple servers.
MCP Servers 2 types of transport selection: STDIO (for local processes), SSE (for http/websocket requests)

Connecting to STDIO transport

async def connect_to_stdio_server(self, server_script_path: str):
        """Connect to an MCP stdio server"""
        is_python = server_script_path.endswith('.py')
        is_js = server_script_path.endswith('.js')
        if not (is_python or is_js):
            raise ValueError("Server script must be a .py or .js file")
        command = "python" if is_python else "node"
        server_params = StdioServerParameters(
            command=command,
            args=[server_script_path],
            env=None
        )
        stdio_transport = await self.exit_stack.enter_async_context(stdio_client(server_params))
        self.stdio, self.write = stdio_transport
        self.session = await self.exit_stack.enter_async_context(ClientSession(self.stdio, self.write))
        await self.session.initialize()
        print("Initialized stdio...")

Connecting to SSE transport

async def connect_to_sse_server(self, server_url: str):
        """Connect to an MCP server running with SSE transport"""
        # Store the context managers so they stay alive
        self._streams_context = sse_client(url=server_url)
        streams = await self._streams_context.__aenter__()

        self._session_context = ClientSession(*streams)
        self.session: ClientSession = await self._session_context.__aenter__()

        # Initialize
        await self.session.initialize()
        print("Initialized SSE...")
Get Tools and Process User request with LLM & MCP Servers
Once the Servers are initialized, we can now fetch tools from all available servers and process user query, processing user query will follow the steps as described above

# get available tools from the servers
stdio_tools = await std_server.list_tools()
sse_tools = await sse_server.list_tools()

Process user request

async def process_user_query(self, available_tools: any, user_query: str, tool_session_map: dict):
        """
        Process the user query and return the response.
        """
        model_name = "gpt-35-turbo"
        api_version = "2022-12-01-preview"

        # On first user query, initialize messages if empty
        self.messages = [
            {
                "role": "user",
                "content": user_query
            }
        ]

        # Initialize your LLM - e.g., Azure OpenAI client
        openai_client = AzureOpenAI(
            api_version=api_version,
            azure_endpoint=<OPENAI_ENDPOINT>,
            api_key=<API_KEY>,
        )

        # send the user query to the LLM along with the available tools from MCP Servers
        response = openai_client.chat.completions.create(
            messages=self.messages,
            model=model_name,
            tools=available_tools,
            tool_choice="auto"
        )

        llm_response = response.choices[0].message

        # append the user query along with LLM response
        self.messages.append({
            "role": "user",
            "content": user_query
        })
        self.messages.append(llm_response)

        # Process respose and handle tool calls
        if azure_response.tool_calls:

            # assuming only one tool call suggested by LLM or keep in for loop to go over all suggested tool_calls
            tool_call = azure_response.tool_calls[0]

            # tool call based on the LLM suggestion
            result = await tool_session_map[tool_call.function.name].call_tool(
                tool_call.function.name,
                json.loads(tool_call.function.arguments)
            )

            # append the response to messages
            self.messages.append({
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": result.content[0].text
            })

            # optionally send the response to LLM to summarize
            azure_response = openai_client.chat.completions.create(
                messages=self.messages,
                model=model_name,
                tools=available_tools,
                tool_choice="auto"
            ).choices[0].message

Hopefully this has provided enough guidance to get you started with implementing MCP Clients, in the later posts we will learn more about hosting MCPs for remote access using Kubernetes / Docker.

Here is a sample source code with MCP Client Agent and Server implementation.

MCP Servers: Plugging AI into Your Developer Toolkit

Venkata — Sun, 30 Mar 2025 18:17:47 +0000

Part 1: The USB-C Moment for AI Development - Accelerating Developer Workflows

Introduction

The Model Context Protocol (MCP) solves a critical challenge in today's AI landscape: how to enable AI models to effectively communicate with diverse software tools. As AI capabilities expand, MCP provides a standardized interface that eliminates custom integration work, allowing models to seamlessly interact with applications through a common language.

What is an MCP Server?

An MCP server functions as a bridge between AI models and software applications. It exposes tools and services to AI models through a standardized request-response protocol that operates over standard I/O or command interfaces. Language-agnostic by design, MCP servers maintain security boundaries while enabling type-safe interactions with external services.

As quoted from the Model Context Protocol documentation:

MCP is an open protocol that standardizes how applications provide context to LLMs. Think of MCP like a USB-C port for AI applications. Just as USB-C provides a standardized way to connect your devices to various peripherals and accessories, MCP provides a standardized way to connect AI models to different data sources and tools.

Why MCP?

MCP creates a universal communication standard between AI models and software applications, eliminating complex integration requirements. Key advantages include:

Standardized messaging format across all connected applications
Automatic translation of natural language to specific application commands
Unified access management for both local and cloud-based services
Seamless multi-tool workflows without custom coding
Integration over standard I/O

The Power of Standardization

Universal Communication

MCP uses a consistent JSON-based message format, providing several key benefits:

Consistency

Uniform error handling
Standardized response formats
Predictable behavior

Flexibility

Language-agnostic implementation
Easy tool addition/removal
Scalable architecture

Security

Built-in permission models
Request validation
Audit trails

Key MCP Integrations

Development Tools

GitHub / GitLab: Repository management and API integration.
Artifactory: Binary management and API integration.
Jira: Issue retrieval and analysis.

Productivity & Communication

Slack: Channel management and messaging.
Google Maps: Location services and directions.

Data & File Systems

PostgreSQL / SQLite: Database querying with schema inspection.
Google Drive: File access and search.

Community Highlights

Docker: Container management.
Kubernetes: Orchestrate pods and services.
Snowflake: Database interaction.

This article is part of a series on MCP. Stay tuned for our next piece on going over architecture of MCP where we'll explore the intricate details of how MCP components work together