DEV Community: Kamil Riyas

Function Calling with Semantic Kernel in AST.NET Core Web API

Kamil Riyas — Sun, 16 Mar 2025 08:26:05 +0000

Impatient? GitHub

In my last post we saw how to integrate Semantic Kernel and invoke its chat service in a very basic example. Not we'll take our idea to the next level by implementing Function Calling.

In this context, if we need the LLM to know something on demand, we enable it to call the actual code(which can be a library or business logic or a wrapper to an external API), thus providing up to date information to the user. This opens up new possiblities and transforms your LLM from a chatbot to an AI Agent.

I ran this App using only a laptop CPU. So don't worry about the hardware and install the following to get started:

dotnet 8.0 and above
local ollama instance with an SLM like llama3.2.

Implementation

Create a Plugins class calling LightsPlugin.cs. What differentiates this from a regular domain classes is the presence of the [KernelFunction("get_lights")] and [Description("Gets a list of lights and their current state")] annotations, that describes that behavior of each methods. The LLM understands these annotations and decides to whether use its existing knowledge or invoke the plugins to provide the answers to the users.

public class LightsPlugin
{
    private readonly List<LightModel> _lights;
    private readonly ILogger<LightsPlugin> _logger;

    public LightsPlugin(ILogger<LightsPlugin> logger)
    {
        _lights = new()
           {
              new LightModel { Id = 1, Name = "Table Lamp", IsOn = false, Brightness = Brightness.Medium, Color = "#FFFFFF" },
              new LightModel { Id = 2, Name = "Porch light", IsOn = false, Brightness = Brightness.High, Color = "#FF0000" },
              new LightModel { Id = 3, Name = "Chandelier", IsOn = true, Brightness = Brightness.Low, Color = "#FFFF00" }
           };

        _logger = logger;
    }

    [KernelFunction("get_lights")]
    [Description("Gets a list of lights and their current state")]
    public async Task<List<LightModel>> GetLightsAsync()
    {
        _logger.LogInformation("getting List of Lights");
        return _lights;
    }

    [KernelFunction("change_state")]
    [Description("Changes the state of the light")]
    public async Task<LightModel?> ChangeStateAsync(LightModel changeState)
    {
        // Find the light to change
        var light = _lights.FirstOrDefault(l => l.Id == changeState.Id);

        // If the light does not exist, return null
        if (light == null)
        {
            return null;
        }

        // Update the light state
        light.IsOn = changeState.IsOn;
        light.Brightness = changeState.Brightness;
        light.Color = changeState.Color;

        return light;
    }
}

Inject the Kernel, Chat Completion Service and the Plugin. I have used an extension to clean up my Program.cs file.

// Add Kernel and Chat Service
services.AddKernel()
    .AddOllamaChatCompletion("llama3.2", httpClient);
// Add Plugins
services.AddSingleton<KernelPlugin>(sp => KernelPluginFactory.CreateFromType<LightsPlugin>(serviceProvider: sp));

In your controller, use the injected Kernel to invoke prompt.

public class ChatController : ControllerBase
{
    private readonly Kernel _kernel;
    private readonly ILogger<ChatController> _logger;

    public ChatController(Kernel Kernel, ILogger<ChatController> logger)
    {
        _kernel = Kernel;
        _logger = logger;
    }

    [HttpPost]
    public async Task<string> GetResponse(UserChatRequest chatRequest)
    {
        if (chatRequest.query != null)
        {
            PromptExecutionSettings settings = new() { FunctionChoiceBehavior = FunctionChoiceBehavior.Auto() };
            var chatResult = await _kernel.InvokePromptAsync(chatRequest.query, new(settings));
            Console.WriteLine(chatResult.ToString());
            return chatResult.ToString();
        }
        else
        {
            return null;
        }
    }
}

Execution

This is the fun part. Now instead of just asking a general question, we can ask the LLM to Get the list of lights for which we expect it to call the plugin to get the list from native c# code.

@sk_functional_calling_webapi_HostAddress = http://localhost:5279

POST {{sk_functional_calling_webapi_HostAddress}}/api/chat/
Content-Type: application/json
{
  "query": "Get the list of lights"
}

###

Output

Status: 200 OKTime: 10333.09 msSize: 184 bytes
FormattedRawHeadersRequest
Body
text/plain; charset=utf-8, 184 bytes
Here is the list of lights:

1. Table Lamp - Not on, Medium brightness, White color
2. Porch light - Not on, High brightness, Red color
3. Chandelier - On, Low brightness, Yellow color

Thus we can see how we can integrate the LLMs into our Apps in a way that actually makes sense. This feature is what inspired me to dive deep into LLM integration. Hope this has the same effect on you.

Have a good day.

Local AI WebAPI with Semantic Kernel and Ollama

Kamil Riyas — Mon, 20 Jan 2025 15:50:42 +0000

Impatient? GitHub

In my last post we saw how to get started with local SLM using Ollama and Semantic Kernel where we called Llama3.2 model from a console applicaition.

In this write-up we'll see how to integrate Semantic Kernel with Asp.net WebAPI.

Please note that this is just a barebone demo, not the standard way to use Semantic Kernel with WebAPI. I'm planning to showcase that in a future post.

Make sure that you have the following things with you in your local:

dotnet 8.0 and above
local ollama instance with an SLM like llama3.2.

Project Setup

We'll initialize a bare bone Asp.Net WebAPI application and install the below packages as well.

dotnet new webapi -n sk-webapi -o sk-webapi
cd sk-webapi\
dotnet add package Microsoft.SemanticKernel
dotnet add package Microsoft.SemanticKernel.Connectors.Ollama --prerelease

*At the time of this writing, Semantic Kernel's Ollama connector is still in preview. So you might want to update the package command.

Coding Time

In your program.cs file

Just like any other WebAPI apps, create the WebApplication builder.
Create an HttpClient object with the local ollama instance uri.
Inject AddOllamaChatCompletion("llama3.2", httpClient) to the service collection.

var builder = WebApplication.CreateBuilder(args);

// Add services to the container.
builder.Services.AddControllers();

var httpClient = new HttpClient() { 
        BaseAddress = new Uri("http://localhost:11434")
};

#pragma warning disable SKEXP0070 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
builder.Services.AddOllamaChatCompletion("llama3.2", httpClient);
#pragma warning restore SKEXP0070 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.        

var app = builder.Build();

Endpoint Setup

Create a controller and like any other service that you'd inject, invoke the chat completion service using IChatCompletionService.

public class ChatController : ControllerBase
    {
        public readonly IChatCompletionService _chatCompletionService;
        public ChatController(IChatCompletionService chatCompletionService)
        {
            _chatCompletionService = chatCompletionService;
        }

        [HttpGet]
        public async Task<string?> GetCharResponseAsync(string input)
        {
            if (input != null)
            {
                var chatResult = await _chatCompletionService.GetChatMessageContentsAsync(input);
                return chatResult[0].ToString();
            }
            else
            {
                return null;
            }
        }
    }

That's it. For my next post i'll be implementing a sample showcasing Function Calling.

Local AI apps with C#, Semantic Kernel and Ollama

Kamil Riyas — Wed, 01 Jan 2025 13:24:14 +0000

Welcome to my first post ever! Enough talk. Let's get started.

Impatient? GitHub.

Make sure that you have the following things with you in your local:

dotnet 8.0 and above
local ollama instance with an SLM like llama3.2

Project Setup

This is going to be a quick console app. Make sure to install all the nugets mentioned below.

dotnet new console -n sk-console -o sk-console
cd sk-console\
dotnet add package Microsoft.SemanticKernel
dotnet add package Microsoft.SemanticKernel.Connectors.Ollama --prerelease

*At the time of this writing, Semantic Kernel's Ollama connector is still in preview. So you might want to update the package command.

Coding Time

In your program.cs file

use the necessary packages
create a builder for Semantic Kernel using the Kernel class and inject the AddOllamaChatCompletion() service.

using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;

var builder = Kernel.CreateBuilder();
var uri = new Uri("http://localhost:11434");

#pragma warning disable SKEXP0070 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
builder.Services.AddOllamaChatCompletion("llama3.2", uri);
#pragma warning restore SKEXP0070 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.

Once after building the kernel get the chatCompletionService using which the chat invoked.

var kernel = builder.Build();
var chatCompletionService = kernel.GetRequiredService<IChatCompletionService>();

try
{
    ChatMessageContent chatMessage = await chatCompletionService
                                    .GetChatMessageContentAsync("Hi, can you tell me a dad joke");
    Console.WriteLine(chatMessage.ToString());
}

For my next post, I’ll implement use the same implementation in a WebAPI project.

That's it. Good day!