Local AI apps with C#, Semantic Kernel and Ollama

Welcome to my first post ever! Enough talk. Let's get started.

Impatient? GitHub.

Make sure that you have the following things with you in your local:

• dotnet 8.0 and above
• local ollama instance with an SLM like llama3.2

Project Setup

This is going to be a quick console app. Make sure to install all the nugets mentioned below.

dotnet new console -n sk-console -o sk-console
cd sk-console\
dotnet add package Microsoft.SemanticKernel
dotnet add package Microsoft.SemanticKernel.Connectors.Ollama --prerelease

*At the time of this writing, Semantic Kernel's Ollama connector is still in preview. So you might want to update the package command.

Coding Time

In your program.cs file

• use the necessary packages
• create a builder for Semantic Kernel using the Kernel class and inject the AddOllamaChatCompletion() service.

using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;

var builder = Kernel.CreateBuilder();
var uri = new Uri("http://localhost:11434");

#pragma warning disable SKEXP0070 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
builder.Services.AddOllamaChatCompletion("llama3.2", uri);
#pragma warning restore SKEXP0070 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.

• Once after building the kernel get the chatCompletionService using which the chat invoked.

var kernel = builder.Build();
var chatCompletionService = kernel.GetRequiredService<IChatCompletionService>();

try
{
    ChatMessageContent chatMessage = await chatCompletionService
                                    .GetChatMessageContentAsync("Hi, can you tell me a dad joke");
    Console.WriteLine(chatMessage.ToString());
}

For my next post, I’ll implement use the same implementation in a WebAPI project.

That's it. Good day!