GenAI with LLMs in .NET using Microsoft.Extensions.AI

Luân Trương — Sun, 02 Feb 2025 14:55:16 +0000

I. Introduce

The rise of Generative AI (GenAI) has redefined how developers build intelligent applications. From chatbots that mimic human conversation to code generators that accelerate development, Large Language Models (LLMs) like GPT-4, Llama 3, and Claude 3 are unlocking unprecedented possibilities. But Why focus on GenAI in .NET, specifically?

1. GenAI is Reshaping Software Development
Modern applications demand contextual awareness, natural language interactions, and adaptive decision-making. GenAI bridges this gap by enabling systems to:

Automate repetitive tasks(e.g., documents summarization, code suggestions).
Deliver hyper-personalized user experiences(e.g., dynamic content generation)
Solve complex problems with human-like reasoning(e.g., data analysis, troubleshooting)

2. NET's Enterprise-Grade Ecosytem
While Python dominates AI research, .NET powers mission-critical systems across industries(finance, healthcare, logistics). Integrating GenAI into .NET applications allows enterprise to:

Leverage exsiting C#/F# codebases and .NET libraries.
Maintain scalability, security, and complicance standards.
Unify AI capabilities with Microsoft's cloud ecosystem(Azure OpenAI, Semantic Kernel).

3. The Emergence of Microsoft.Extension.AI

Microsoft's new Microsoft.Extensions.AI library simplifies LLM integration into .NET app using familiar patterns:

Standardized APIs: Interact with multiple LLM providers (OpenAI, Hugging Face, Azure) through a unified interface.
Dependency Injection (DI): Seamlessy inject AI services into .ASP .NET core pipelines.
Semantic Kernel Integration: Build modular, AI-driven workflows with C#.

What's next?
This article is a result of what I researched and worked on GenAI with .NET apps

.NET app with GenAI

Source code for these scenarios can be found at https://github.com/LuanTruongPTIT/GenAI-Micro
I use Ollama for local development(saving cost), and in a higher environment.
The business use cases for these scenarios are semantic search and chat completion (text summary for data seeding actually)

Getting Started

Before diving into the examples, here's what you need to run LLMs locally:

Docker running on your machine
Ollama container running with the llama3.2:1b model:


# Pull the Ollama container
docker run --gpus all -d -v ollama_data:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

# Pull the llama3.2:1b model
docker exec -it ollama ollama pull llama3.2:1b

A few NuGet packages (I built this using a .NET 9 console application):

Install-Package Microsoft.Extensions.AI # The base AI library
Install-Package Microsoft.Extensions.AI.Ollama # Ollama provider implementation
Install-Package Microsoft.Extensions.Hosting # For building the DI container

Semantic search with GenAI

The technologies used to implement semantic search in this scenario are pgvector, and its .NET packages. We use the cosine distance searching which is supported by pgvector extension.

Supported distance functions are:

<-> - L2 distance
<#> - (negative) inner product
<=> - cosine distance
<+> - L1 distance (added in 0.7.0)
<~> - Hamming distance (binary vectors, added in 0.7.0)
<%> - Jaccard distance (binary vectors, added in 0.7.0) And we use <=> - cosine distance for this scenario

SELECT p.id, p.description, p.embedding, p.price, p.type, p.updated, p.embedding <=> @__vector_0 AS "Distance"
FROM item.products AS p
ORDER BY p.embedding <=> @__vector_0

The code below processes data query:

internal class ItemTypesQueryHandler(ProductDbContext dbContext, IProductItemAI productItemAI, ILogger<ItemTypesQueryHandler> logger) : IRequestHandler<ItemTypesQuery, IEnumerable<ItemTypeDto>>
    {
        public async Task<IEnumerable<ItemTypeDto>> Handle(ItemTypesQuery request, CancellationToken cancellationToken)
        {
            ArgumentNullException.ThrowIfNull(request);

            request.Text = request.Text ?? "coffee";

            var vector = await productItemAI.GetEmbeddingAsync(request.Text);

            var itemsWithDistance = await dbContext.Items
                .Select(c => new { Item = c, Distance = c.Embedding.CosineDistance(vector) })
                .OrderBy(c => c.Distance)
                .ToListAsync();

            logger.LogDebug("Results from {text}: {results}", request.Text, string.Join(", ", itemsWithDistance.Select(i => $"{i.Item.Type} => {i.Distance}")));

            return await Task.FromResult(itemsWithDistance.Select(x => new ItemTypeDto
            {
                Name = x.Item.Type.ToString(),
                ItemType = x.Item.Type
            }).Distinct());
        }

Chat completion (text summary)

Based on the previous example, we can create a conversation to create data at the beginning of the project:

public class ProductDbContextSeeder(
    IProductItemAI catalogAI,
    IChatClient chatClient,
    ILogger<ProductDbContextSeeder> logger
    ) : IDbSeeder<ProductDbContext>
{
    private readonly SemaphoreSlim _semaphore = new(1);

    public async Task SeedAsync(ProductDbContext context)
    {
        await _semaphore.WaitAsync();

        try
        {

            context.Database.OpenConnection();
            ((NpgsqlConnection)context.Database.GetDbConnection()).ReloadTypes();

            if (!context.Items.Any())
            {
                await context.Items.ExecuteDeleteAsync();

                var catalogItems = new ItemV2Data();

                if (catalogAI.IsEnabled)
                {
                    logger.LogInformation("Generating {NumItems} embeddings", catalogItems.Count);
                    for (int i = 0; i < catalogItems.Count; i++)
                    {
                        var prompt = $"Generate the description of {catalogItems[i].Type} in max 20 words";
                        var response = await chatClient.CompleteAsync(prompt);
                        catalogItems[i].SetDescription(response.Message?.Text);
                        catalogItems[i].Embedding = await catalogAI.GetEmbeddingAsync(catalogItems[i]);
                    }
                }

                await context.Items.AddRangeAsync(catalogItems);
                logger.LogInformation("Seeded catalog with {NumItems} items", context.Items.Count());
                await context.SaveChangesAsync();
            }

            await context.SaveChangesAsync();
        }
        finally
        {
            _semaphore.Release();
        }
    }
}

public static class ChatCompletionServiceExtensions
{
    public static void AddChatCompletionService(this IHostApplicationBuilder builder, string serviceName)
    {
        var pipeline = (ChatClientBuilder pipeline) => pipeline
            .UseFunctionInvocation()
            .UseOpenTelemetry(configure: c => c.EnableSensitiveData = true);

        if (builder.Configuration["ai:Type"] == "openai")
        {
            builder.AddOpenAIChatClient(serviceName, pipeline);
        }
        else
        {
            builder.AddOllamaChatClient(serviceName, pipeline);
        }
    }
    // remove for brevity
    // ...
}

Tracing chat completion to summary text with