Multi-Agent Orchestration with Semantic Kernel
Part 3 of 5: Building Intelligent AI Systems with Azure AI Agent Service
In Part 1, we explored the fundamentals of Azure AI Agent Service and built our first intelligent agent. Part 2 dove deeper into tool integration and conversation management. Now it's time to tackle something more ambitious: what happens when one agent isn't enough?
Real-world AI applications often require specialized expertise. A customer support system might need one agent for technical issues, another for billing questions, and a third for general inquiries. A software development assistant might combine a code reviewer, a security analyst, and a documentation writer. This is where multi-agent orchestration shines.
In this article, we'll explore how Semantic Kernel's AgentGroupChat enables sophisticated multi-agent conversations, and how to combine Azure AI Agent Service agents with Semantic Kernel's native agents for powerful hybrid systems.
Why Multi-Agent Orchestration?
Before diving into code, let's understand why you'd want multiple agents:
- Specialization: Each agent can excel at a specific domain, with focused system prompts and tools
- Separation of Concerns: Cleaner architecture with well-defined responsibilities
- Scalability: Add new capabilities by introducing new agents without modifying existing ones
- Debate and Validation: Agents can review each other's work, catching errors and improving quality
- Complex Workflows: Model real-world processes that naturally involve multiple roles
Think of it like a team meeting—different experts contribute their perspectives to solve problems collaboratively.
Introducing AgentGroupChat
Semantic Kernel provides AgentGroupChat as the primary abstraction for multi-agent conversations. It manages a shared conversation history and coordinates which agent responds when.
Here's the basic structure:
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Agents;
using Microsoft.SemanticKernel.Agents.Chat;
// Create your agents (we'll cover this in detail)
var codeReviewer = CreateCodeReviewerAgent();
var securityAnalyst = CreateSecurityAnalystAgent();
var documentationWriter = CreateDocumentationWriterAgent();
// Create the group chat with all agents
var groupChat = new AgentGroupChat(codeReviewer, securityAnalyst, documentationWriter)
{
ExecutionSettings = new()
{
SelectionStrategy = new SequentialSelectionStrategy(),
TerminationStrategy = new MaximumIterationTerminationStrategy(10)
}
};
// Add user input to kick off the conversation
groupChat.AddChatMessage(new ChatMessageContent(
AuthorRole.User,
"Please review this code for quality, security, and documentation needs:\n\n" + codeToReview
));
// Let the agents collaborate
await foreach (var message in groupChat.InvokeAsync())
{
Console.WriteLine($"[{message.AuthorName}]: {message.Content}");
}
The magic happens in InvokeAsync()—agents take turns responding based on your selection strategy until a termination condition is met.
Creating Specialized Agents with ChatCompletionAgent
Semantic Kernel's ChatCompletionAgent is the workhorse for creating specialized agents. Each agent gets its own personality, instructions, and optional tools:
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Agents;
using Microsoft.SemanticKernel.Connectors.AzureOpenAI;
public class AgentFactory
{
private readonly Kernel _kernel;
public AgentFactory(string endpoint, string apiKey, string deploymentName)
{
var builder = Kernel.CreateBuilder();
builder.AddAzureOpenAIChatCompletion(deploymentName, endpoint, apiKey);
_kernel = builder.Build();
}
public ChatCompletionAgent CreateCodeReviewer()
{
return new ChatCompletionAgent
{
Name = "CodeReviewer",
Instructions = """
You are an expert code reviewer with 15 years of experience in C# and .NET.
Your responsibilities:
- Analyze code for clarity, maintainability, and adherence to best practices
- Identify potential bugs, edge cases, and error handling gaps
- Suggest concrete improvements with code examples
- Rate code quality on a scale of 1-10
Be constructive but thorough. Developers appreciate specific, actionable feedback.
When you've completed your review, clearly state "CODE REVIEW COMPLETE".
""",
Kernel = _kernel
};
}
public ChatCompletionAgent CreateSecurityAnalyst()
{
return new ChatCompletionAgent
{
Name = "SecurityAnalyst",
Instructions = """
You are a security specialist focused on identifying vulnerabilities in code.
Your responsibilities:
- Check for OWASP Top 10 vulnerabilities
- Identify injection risks, authentication weaknesses, and data exposure
- Review authorization logic and access controls
- Flag hardcoded secrets, insecure configurations
- Suggest security improvements with remediation code
Prioritize findings as Critical, High, Medium, or Low severity.
When you've completed your analysis, clearly state "SECURITY ANALYSIS COMPLETE".
""",
Kernel = _kernel
};
}
public ChatCompletionAgent CreateDocumentationWriter()
{
return new ChatCompletionAgent
{
Name = "DocumentationWriter",
Instructions = """
You are a technical writer who creates clear, helpful documentation.
Your responsibilities:
- Suggest XML documentation comments for public APIs
- Identify undocumented or poorly documented code
- Write README sections explaining functionality
- Create usage examples that developers can copy-paste
Good documentation is concise but complete. Include edge cases and gotchas.
When you've completed your suggestions, clearly state "DOCUMENTATION REVIEW COMPLETE".
""",
Kernel = _kernel
};
}
}
Notice how each agent has a distinct personality and clear completion signal. These signals become important for termination strategies.
Selection Strategies: Who Speaks Next?
The selection strategy determines which agent responds at each turn. Semantic Kernel provides several built-in options, and you can create custom strategies.
Sequential Selection
The simplest approach—agents take turns in order:
var groupChat = new AgentGroupChat(agent1, agent2, agent3)
{
ExecutionSettings = new()
{
SelectionStrategy = new SequentialSelectionStrategy()
}
};
Good for: Structured workflows where each agent has a defined role in sequence.
Kernel Function Selection (AI-Driven)
Let an AI model decide which agent should respond based on the conversation context:
var selectionFunction = KernelFunctionFactory.CreateFromPrompt(
"""
Analyze the conversation and determine which agent should respond next.
Available agents:
- CodeReviewer: Handles code quality, best practices, and maintainability
- SecurityAnalyst: Handles security vulnerabilities and compliance
- DocumentationWriter: Handles documentation and examples
Recent conversation:
{{$history}}
Consider:
1. Has each agent had a chance to contribute?
2. Did the last message raise concerns another agent should address?
3. Is there a logical next step in the review process?
Respond with only the agent name that should speak next.
""");
var groupChat = new AgentGroupChat(codeReviewer, securityAnalyst, documentationWriter)
{
ExecutionSettings = new()
{
SelectionStrategy = new KernelFunctionSelectionStrategy(selectionFunction, _kernel)
{
InitialAgent = codeReviewer, // Who starts the conversation
HistoryVariableName = "history",
ResultParser = (result) => result.GetValue<string>()?.Trim() ?? "CodeReviewer"
}
}
};
Good for: Dynamic conversations where the optimal next speaker depends on context.
Custom Selection Strategy
For complex logic, implement your own:
public class PrioritySelectionStrategy : SelectionStrategy
{
private readonly Dictionary<string, int> _priorities;
private readonly HashSet<string> _agentsWhoHaveSpoken = new();
public PrioritySelectionStrategy(Dictionary<string, int> priorities)
{
_priorities = priorities;
}
protected override Task<Agent> SelectAgentAsync(
IReadOnlyList<Agent> agents,
IReadOnlyList<ChatMessageContent> history,
CancellationToken cancellationToken = default)
{
// Track who has spoken
foreach (var message in history)
{
if (!string.IsNullOrEmpty(message.AuthorName))
_agentsWhoHaveSpoken.Add(message.AuthorName);
}
// Find highest priority agent who hasn't spoken yet
var nextAgent = agents
.Where(a => !_agentsWhoHaveSpoken.Contains(a.Name))
.OrderByDescending(a => _priorities.GetValueOrDefault(a.Name ?? "", 0))
.FirstOrDefault();
// If everyone has spoken, pick based on last message content
if (nextAgent == null)
{
var lastMessage = history.LastOrDefault()?.Content ?? "";
// Route security concerns to security analyst
if (lastMessage.Contains("vulnerability", StringComparison.OrdinalIgnoreCase) ||
lastMessage.Contains("security", StringComparison.OrdinalIgnoreCase))
{
nextAgent = agents.First(a => a.Name == "SecurityAnalyst");
}
else
{
nextAgent = agents.First();
}
}
return Task.FromResult(nextAgent);
}
}
Termination Strategies: When to Stop
Equally important is knowing when the conversation should end. Semantic Kernel provides several options:
Maximum Iterations
Simple but effective—stop after N turns:
ExecutionSettings = new()
{
TerminationStrategy = new MaximumIterationTerminationStrategy(10)
}
Keyword-Based Termination
Stop when a specific phrase appears:
public class CompletionKeywordTerminationStrategy : TerminationStrategy
{
private readonly string[] _completionKeywords;
private readonly HashSet<string> _agentsCompleted = new();
public CompletionKeywordTerminationStrategy(params string[] keywords)
{
_completionKeywords = keywords;
}
protected override Task<bool> ShouldAgentTerminateAsync(
Agent agent,
IReadOnlyList<ChatMessageContent> history,
CancellationToken cancellationToken = default)
{
var lastMessage = history.LastOrDefault();
if (lastMessage?.Content != null)
{
foreach (var keyword in _completionKeywords)
{
if (lastMessage.Content.Contains(keyword, StringComparison.OrdinalIgnoreCase))
{
_agentsCompleted.Add(lastMessage.AuthorName ?? "");
break;
}
}
}
// Terminate when all agents have signaled completion
return Task.FromResult(_agentsCompleted.Count >= Agents.Count);
}
public IReadOnlyList<Agent> Agents { get; set; } = Array.Empty<Agent>();
}
Aggregated Termination
Combine multiple strategies:
var terminationStrategy = new AggregatedTerminationStrategy(
new MaximumIterationTerminationStrategy(15), // Safety limit
new CompletionKeywordTerminationStrategy(
"REVIEW COMPLETE",
"ANALYSIS COMPLETE",
"ALL DONE"
)
);
Combining Azure AI Agent Service with Semantic Kernel
Here's where it gets interesting. Azure AI Agent Service provides powerful managed agents with built-in tools (code interpreter, file search, etc.), while Semantic Kernel offers flexible orchestration. You can use both together!
The AzureAIAgent Adapter
Semantic Kernel provides AzureAIAgent to wrap Azure AI Agent Service agents:
using Azure.AI.Projects;
using Azure.Identity;
using Microsoft.SemanticKernel.Agents.AzureAI;
public class HybridAgentOrchestrator
{
private readonly AIProjectClient _projectClient;
private readonly Kernel _kernel;
public HybridAgentOrchestrator(string connectionString)
{
_projectClient = new AIProjectClient(connectionString, new DefaultAzureCredential());
var builder = Kernel.CreateBuilder();
builder.AddAzureOpenAIChatCompletion(
"gpt-4o",
Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT")!,
new DefaultAzureCredential()
);
_kernel = builder.Build();
}
public async Task<AzureAIAgent> CreateDataAnalystAgentAsync()
{
// Create an Azure AI Agent Service agent with code interpreter
var agentDefinition = await _projectClient.GetAgentsClient()
.CreateAgentAsync(
model: "gpt-4o",
name: "DataAnalyst",
instructions: """
You are a data analyst who excels at analyzing datasets and creating visualizations.
Use the code interpreter to:
- Process and analyze data files
- Create charts and graphs
- Perform statistical analysis
- Generate insights from data
Always explain your methodology and findings clearly.
""",
tools: new List<ToolDefinition>
{
new CodeInterpreterToolDefinition()
}
);
// Wrap it for use with Semantic Kernel
return new AzureAIAgent(agentDefinition, _projectClient.GetAgentsClient());
}
public ChatCompletionAgent CreateReportWriterAgent()
{
// A Semantic Kernel native agent for writing reports
return new ChatCompletionAgent
{
Name = "ReportWriter",
Instructions = """
You are a business analyst who transforms technical analysis into executive-friendly reports.
Your responsibilities:
- Summarize findings in clear, non-technical language
- Highlight key insights and actionable recommendations
- Structure information for busy executives
- Create compelling narratives from data
When the report is complete, state "REPORT COMPLETE".
""",
Kernel = _kernel
};
}
}
Putting It All Together: A Complete Example
Let's build a code review pipeline that combines multiple agents:
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Agents;
using Microsoft.SemanticKernel.Agents.Chat;
using Microsoft.SemanticKernel.ChatCompletion;
public class CodeReviewPipeline
{
private readonly AgentFactory _factory;
public CodeReviewPipeline(string endpoint, string apiKey, string deploymentName)
{
_factory = new AgentFactory(endpoint, apiKey, deploymentName);
}
public async Task<CodeReviewResult> ReviewCodeAsync(string code, string language = "C#")
{
// Create our specialized agents
var codeReviewer = _factory.CreateCodeReviewer();
var securityAnalyst = _factory.CreateSecurityAnalyst();
var documentationWriter = _factory.CreateDocumentationWriter();
// Create a coordinator agent that synthesizes feedback
var coordinator = new ChatCompletionAgent
{
Name = "Coordinator",
Instructions = """
You are the review coordinator. After all specialists have provided feedback:
1. Synthesize the findings into a unified report
2. Prioritize issues by severity and impact
3. Provide an overall quality score (1-10)
4. List the top 3 most important changes to make
Format your response as a structured summary.
End with "REVIEW PIPELINE COMPLETE".
""",
Kernel = _factory.GetKernel()
};
// Set up the group chat
var groupChat = new AgentGroupChat(
codeReviewer,
securityAnalyst,
documentationWriter,
coordinator
)
{
ExecutionSettings = new()
{
SelectionStrategy = new SequentialSelectionStrategy(),
TerminationStrategy = new AggregatedTerminationStrategy(
new MaximumIterationTerminationStrategy(8),
new KeywordTerminationStrategy("REVIEW PIPELINE COMPLETE")
)
}
};
// Start the review
groupChat.AddChatMessage(new ChatMessageContent(
AuthorRole.User,
$"""
Please conduct a comprehensive review of this {language} code:
```
{% endraw %}
{language.ToLower()}
{code}
{% raw %}
```
Each specialist should provide their analysis, then the coordinator
will synthesize the findings into a final report.
"""
));
// Collect all responses
var responses = new List<AgentResponse>();
await foreach (var message in groupChat.InvokeAsync())
{
responses.Add(new AgentResponse
{
AgentName = message.AuthorName ?? "Unknown",
Content = message.Content ?? "",
Timestamp = DateTime.UtcNow
});
// Log progress
Console.WriteLine($"\n{'=',-50}");
Console.WriteLine($"[{message.AuthorName}]");
Console.WriteLine($"{'=',-50}");
Console.WriteLine(message.Content);
}
return new CodeReviewResult
{
Responses = responses,
FinalReport = responses.LastOrDefault(r => r.AgentName == "Coordinator")?.Content ?? "",
TotalAgentInteractions = responses.Count
};
}
}
public record AgentResponse
{
public required string AgentName { get; init; }
public required string Content { get; init; }
public DateTime Timestamp { get; init; }
}
public record CodeReviewResult
{
public required IReadOnlyList<AgentResponse> Responses { get; init; }
public required string FinalReport { get; init; }
public int TotalAgentInteractions { get; init; }
}
Usage Example
var pipeline = new CodeReviewPipeline(
Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT")!,
Environment.GetEnvironmentVariable("AZURE_OPENAI_KEY")!,
"gpt-4o"
);
var codeToReview = """
public class UserService
{
private readonly string _connectionString;
public UserService()
{
_connectionString = "Server=prod;Database=Users;User=admin;Password=admin123";
}
public User GetUser(string id)
{
var query = $"SELECT * FROM Users WHERE Id = '{id}'";
// Execute query...
return null;
}
public void DeleteUser(string id)
{
var query = $"DELETE FROM Users WHERE Id = '{id}'";
// Execute query...
}
}
""";
var result = await pipeline.ReviewCodeAsync(codeToReview);
Console.WriteLine("\n\n========== FINAL REPORT ==========");
Console.WriteLine(result.FinalReport);
Console.WriteLine($"\nTotal agent interactions: {result.TotalAgentInteractions}");
Best Practices for Multi-Agent Systems
After building several multi-agent systems, here are lessons learned:
1. Clear Agent Boundaries
Each agent should have a well-defined scope. Overlapping responsibilities lead to redundant work and conflicting advice.
2. Explicit Completion Signals
Design agents to clearly indicate when they've finished their task. This makes termination strategies more reliable.
3. Conversation History Awareness
Agents should acknowledge previous contributions. Prompt them to build on each other's work, not repeat it.
4. Error Handling
Agents can fail or produce unexpected output. Always have maximum iteration limits as a safety net:
TerminationStrategy = new AggregatedTerminationStrategy(
new MaximumIterationTerminationStrategy(20), // Always have a ceiling
yourCustomStrategy
)
5. Cost Consciousness
Multiple agents mean multiple API calls. For development, use smaller models. For production, consider caching and batching strategies.
6. Observability
Log agent interactions for debugging and improvement:
await foreach (var message in groupChat.InvokeAsync())
{
_logger.LogInformation(
"Agent {AgentName} responded with {CharCount} characters",
message.AuthorName,
message.Content?.Length ?? 0
);
// Track in telemetry
_telemetry.TrackEvent("AgentResponse", new Dictionary<string, string>
{
["AgentName"] = message.AuthorName ?? "Unknown",
["ConversationId"] = conversationId
});
}
What's Next?
You now have the foundation for building sophisticated multi-agent systems. We've covered:
-
AgentGroupChatas the orchestration primitive - Selection strategies for controlling conversation flow
- Termination strategies for knowing when to stop
- Combining Azure AI Agent Service with Semantic Kernel agents
- A complete code review pipeline example
In Part 4, we'll explore advanced tool integration—building custom tools, connecting to external APIs, and giving your agents real-world superpowers. We'll create agents that can query databases, call REST APIs, and interact with your existing systems.
Resources:
Found this helpful? Follow for Part 4, where we'll give our agents superpowers through custom tool integration!
Tags: #dotnet #azure #ai #semantickernel #agents
Top comments (0)