Semantic Kernel from First Principles: Understanding the Architecture

If you've built applications with the OpenAI or Azure OpenAI APIs directly, you've probably noticed the pattern: create an HTTP client, construct messages, handle retries, parse responses, manage conversation history. It works, but it quickly becomes tedious as your application grows.

Microsoft's Semantic Kernel (SK) solves this elegantly. It's not just another SDK wrapper—it's a complete orchestration framework that fundamentally changes how you architect LLM-powered applications. In this first article of our five-part series, we'll build a deep understanding of SK's core concepts and architecture.

Why Semantic Kernel Over Raw API Calls?

Before we dive into code, let's understand what problems SK actually solves:

1. Service Abstraction: Your code doesn't care whether you're using Azure OpenAI, OpenAI, Hugging Face, or a local Ollama instance. SK abstracts the provider completely.

2. Plugin Architecture: Functions become first-class citizens. The LLM can discover and invoke your C# methods naturally through function calling.

3. Memory Management: Built-in patterns for semantic memory, vector stores, and retrieval-augmented generation (RAG).

4. Prompt Engineering: Template systems, prompt functions, and execution settings that scale beyond simple string concatenation.

5. Observability: Filters, telemetry hooks, and middleware patterns for production monitoring.

Let's see these concepts in action.

Building Your First Kernel

The Kernel is the central orchestrator in Semantic Kernel. Think of it as a dependency injection container specialized for AI services. Here's how to build one:

using Microsoft.SemanticKernel;
using Azure.Identity;

var builder = Kernel.CreateBuilder();

// Add Azure OpenAI chat completion
builder.AddAzureOpenAIChatCompletion(
    deploymentName: "gpt-4o",
    endpoint: Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT")!,
    credential: new DefaultAzureCredential());

// Add embedding service for memory/RAG scenarios
builder.AddAzureOpenAITextEmbeddingGeneration(
    deploymentName: "text-embedding-3-large",
    endpoint: Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT")!,
    credential: new DefaultAzureCredential());

var kernel = builder.Build();

Notice we're using DefaultAzureCredential. This is the recommended pattern for Azure services—it automatically handles managed identities in production and developer credentials locally. Avoid hardcoding API keys in your source code.

Understanding the Service Layer

Once your kernel is built, you can access the underlying services directly when needed:

using Microsoft.SemanticKernel.ChatCompletion;
using Microsoft.SemanticKernel.Embeddings;

// Get the chat completion service
var chatService = kernel.GetRequiredService<IChatCompletionService>();

// Get the embedding service
var embeddingService = kernel.GetRequiredService<ITextEmbeddingGenerationService>();

// Use chat service directly
var history = new ChatHistory();
history.AddSystemMessage("You are a helpful assistant.");
history.AddUserMessage("What is Semantic Kernel?");

var response = await chatService.GetChatMessageContentAsync(history);
Console.WriteLine(response.Content);

// Generate embeddings directly
var embeddings = await embeddingService.GenerateEmbeddingsAsync(new[] 
{
    "Semantic Kernel is an AI orchestration framework",
    "It supports multiple LLM providers"
});

Console.WriteLine($"Generated {embeddings.Count} embeddings of dimension {embeddings[0].Length}");

This direct access is useful when you need fine-grained control, but most of the time you'll work through the kernel's higher-level APIs.

KernelFunction: The Building Block

Everything in Semantic Kernel revolves around KernelFunction. A function represents a unit of work the kernel can execute—either a prompt sent to an LLM or native C# code.

Creating Functions from Prompts

The simplest way to create a function is from a prompt template:

var summarizeFunction = kernel.CreateFunctionFromPrompt(
    promptTemplate: """
        Summarize the following text in {{$style}} style.
        Keep the summary under {{$maxWords}} words.

        Text to summarize:
        {{$input}}

        Summary:
        """,
    functionName: "Summarize",
    description: "Summarizes text in a specified style");

var result = await kernel.InvokeAsync(summarizeFunction, new KernelArguments
{
    ["input"] = longArticleText,
    ["style"] = "bullet points",
    ["maxWords"] = "100"
});

Console.WriteLine(result.GetValue<string>());

Notice the {{$variableName}} syntax—this is the default Semantic Kernel template format. Variables are passed through KernelArguments, which is essentially a dictionary.

Understanding FunctionResult

When you invoke a function, you get a FunctionResult object:

FunctionResult result = await kernel.InvokeAsync(summarizeFunction, arguments);

// Get the raw value
string summary = result.GetValue<string>()!;

// Access metadata
Console.WriteLine($"Function: {result.Function.Name}");
Console.WriteLine($"Rendered prompt tokens: {result.Metadata?["Usage"]?["PromptTokens"]}");
Console.WriteLine($"Completion tokens: {result.Metadata?["Usage"]?["CompletionTokens"]}");

The metadata contains provider-specific information like token usage, model name, and finish reason—critical for cost tracking and debugging.

Native Functions: C# as AI Tools

While prompt functions send text to an LLM, native functions execute regular C# code. This is where Semantic Kernel becomes truly powerful—your LLM can call your application's business logic.

public class TextAnalysisPlugin
{
    [KernelFunction("count_words")]
    [Description("Counts the number of words in the provided text")]
    public int CountWords(
        [Description("The text to analyze")] string text)
    {
        return text.Split(' ', StringSplitOptions.RemoveEmptyEntries).Length;
    }

    [KernelFunction("extract_emails")]
    [Description("Extracts all email addresses from the provided text")]
    public string[] ExtractEmails(
        [Description("The text to search for email addresses")] string text)
    {
        var emailPattern = @"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}";
        return Regex.Matches(text, emailPattern)
            .Select(m => m.Value)
            .Distinct()
            .ToArray();
    }

    [KernelFunction("calculate_reading_time")]
    [Description("Estimates reading time in minutes based on word count")]
    public double CalculateReadingTime(
        [Description("The text to analyze")] string text,
        [Description("Words per minute reading speed (default: 200)")] int wordsPerMinute = 200)
    {
        var wordCount = CountWords(text);
        return Math.Round((double)wordCount / wordsPerMinute, 1);
    }
}

// Register the plugin
kernel.Plugins.AddFromType<TextAnalysisPlugin>();

// Now these functions are available for the LLM to call

The [Description] attributes are crucial—they're included in the function schema sent to the LLM, helping it understand when and how to use each function.

Registering Plugins

Semantic Kernel offers several ways to register plugins:

// 1. From a type (creates instance internally)
kernel.Plugins.AddFromType<TextAnalysisPlugin>();

// 2. From an instance (useful for dependency injection)
var orderService = serviceProvider.GetRequiredService<IOrderService>();
var orderPlugin = new OrderPlugin(orderService);
kernel.Plugins.AddFromObject(orderPlugin, "Orders");

// 3. From functions directly
var functions = new List<KernelFunction>
{
    kernel.CreateFunctionFromPrompt("Translate to French: {{$input}}", "TranslateToFrench"),
    kernel.CreateFunctionFromPrompt("Translate to Spanish: {{$input}}", "TranslateToSpanish")
};
kernel.Plugins.AddFromFunctions("Translation", functions);

// 4. Accessing registered plugins
foreach (var plugin in kernel.Plugins)
{
    Console.WriteLine($"Plugin: {plugin.Name}");
    foreach (var function in plugin)
    {
        Console.WriteLine($"  - {function.Name}: {function.Description}");
    }
}

Prompt Template Engines

Semantic Kernel supports multiple template syntaxes. Choose based on your needs:

Default SK Format

Simple and lightweight—good for most cases:

var skTemplate = """
    You are a {{$role}}.

    User: {{$input}}

    Response:
    """;

Handlebars Format

More powerful with conditionals and loops:

using Microsoft.SemanticKernel.PromptTemplates.Handlebars;

var handlebarsTemplate = """
    You are a helpful assistant.

    {{#if context}}
    Use this context to answer:
    {{context}}
    {{/if}}

    {{#each examples}}
    Example: {{this}}
    {{/each}}

    Question: {{question}}
    """;

var function = kernel.CreateFunctionFromPrompt(
    new PromptTemplateConfig
    {
        Template = handlebarsTemplate,
        TemplateFormat = "handlebars",
        Name = "AnswerWithContext"
    },
    new HandlebarsPromptTemplateFactory());

Liquid Format

If you're coming from a Ruby/Jekyll background:

using Microsoft.SemanticKernel.PromptTemplates.Liquid;

var liquidTemplate = """
    {% if premium_user %}
    Welcome back, premium member!
    {% endif %}

    Your question: {{ question }}
    """;

Execution Settings

Control how the LLM processes your prompts:

using Microsoft.SemanticKernel.Connectors.AzureOpenAI;

var settings = new AzureOpenAIPromptExecutionSettings
{
    MaxTokens = 1000,
    Temperature = 0.7,  // Higher = more creative
    TopP = 0.95,
    FrequencyPenalty = 0.0,
    PresencePenalty = 0.0,
    ResponseFormat = "json_object",  // Force JSON output
    Seed = 42  // For reproducible outputs
};

var result = await kernel.InvokePromptAsync(
    "List three random fruits as JSON",
    new KernelArguments(settings));

Different providers have different settings classes (OpenAIPromptExecutionSettings, AzureOpenAIPromptExecutionSettings, etc.), but they share common properties.

The Request Flow

Understanding how a request flows through Semantic Kernel helps with debugging and optimization:

1. InvokeAsync called with function + arguments
        ↓
2. Template engine renders the prompt
        ↓
3. Pre-invocation filters execute (logging, validation)
        ↓
4. Request sent to AI service
        ↓
5. Response received
        ↓
6. Post-invocation filters execute (caching, telemetry)
        ↓
7. FunctionResult returned to caller

You can hook into this flow with filters:

public class LoggingFilter : IPromptRenderFilter
{
    private readonly ILogger<LoggingFilter> _logger;

    public LoggingFilter(ILogger<LoggingFilter> logger) => _logger = logger;

    public async Task OnPromptRenderAsync(
        PromptRenderContext context, 
        Func<PromptRenderContext, Task> next)
    {
        _logger.LogDebug("Rendering prompt for {Function}", context.Function.Name);

        await next(context);

        _logger.LogDebug("Rendered prompt: {Prompt}", context.RenderedPrompt);
    }
}

// Register the filter
builder.Services.AddSingleton<IPromptRenderFilter, LoggingFilter>();

Dependency Injection Integration

Semantic Kernel integrates seamlessly with .NET's dependency injection:

// In Program.cs or Startup.cs
builder.Services.AddKernel()
    .AddAzureOpenAIChatCompletion(
        deploymentName: "gpt-4o",
        endpoint: config["AzureOpenAI:Endpoint"]!,
        credential: new DefaultAzureCredential())
    .AddAzureOpenAITextEmbeddingGeneration(
        deploymentName: "text-embedding-3-large",
        endpoint: config["AzureOpenAI:Endpoint"]!,
        credential: new DefaultAzureCredential());

// Register your plugins
builder.Services.AddSingleton<OrderPlugin>();
builder.Services.AddSingleton<TextAnalysisPlugin>();

// In your service
public class AiAssistantService
{
    private readonly Kernel _kernel;

    public AiAssistantService(Kernel kernel)
    {
        _kernel = kernel;

        // Plugins registered via DI are automatically available
    }

    public async Task<string> ProcessQueryAsync(string query)
    {
        return await _kernel.InvokePromptAsync<string>(query);
    }
}

What's Next

In this article, we've built a solid foundation:

• Kernel: The central orchestrator that manages services and plugins
• Services: Abstracted AI capabilities (chat, embeddings)
• Functions: Prompt-based or native C# units of work
• Plugins: Collections of related functions
• Templates: Multiple syntax options for complex prompts
• Execution Settings: Fine-grained control over LLM behavior

In Part 2, we'll dive deep into plugins—native functions, importing OpenAPI specifications, and the game-changing Model Context Protocol (MCP) integration that's reshaping how we build AI applications in 2025.

---

This is Part 1 of a 5-part series on Semantic Kernel. Next up: Plugins Deep Dive: From Native Functions to MCP Integration