Chatbot with Semantic Kernel - Part 1: Setup and first steps 👣

Welcome to the very first post in a series of blogs about my journey building a chatbot with Semantic Kernel. In particular, I will work with the new experimental Agent framework.

This side project serves two main purposes. First, it's a learning opportunity to gain hands-on experience with Semantic Kernel, providing a place to test new features, models, and experiment with plugin development. Second, it serves as a rapid prototyping platform. With a ready-made User Interface integrated with Semantic Kernel, creating quick prototypes for new use cases becomes straightforward.

Before we dive into it, here are some important notes about this series.

• These posts assume basic knowledge of Generative AI. We won't cover fundamental concepts like Large Language Models (LLMs) or embeddings 🎓
• While this project uses Python, Semantic Kernel also supports .NET and Java. Feel free to experiment with your preferred language ⭐
• The chat User Interface implementation details won't be covered. For those interested, I'm using the NiceGUI Python library 🗪
• In order to make something practical, we will develop a Librarian chatbot. Throughout the different chapters, we will add new features, such as similarity search, abstract summarization, or book curation 📖
• Initially, the chatbot is integrated with Azure OpenAI. Support for OpenAI and Hugging Face will be added in future posts 🤝
• You can find a working version of the chatbot in my GitHub repository 🐍 PyChatbot for Semantic Kernel 👨‍💻

Let's begin this exciting journey together!

What is Semantic Kernel?

According to the official documentation:

Semantic Kernel is a lightweight, open-source development kit that lets you easily build AI agents and integrate the latest AI models into your C#, Python, or Java codebase. It serves as an efficient middleware that enables rapid delivery of enterprise-grade solutions.

In essence, Semantic Kernel is an SDK that simplifies AI agent development. The chatbot will be based on the new experimental Agent Framework, which is still in development phase.

Installation

To begin, install the semantic-kernel package using pip:

pip install semantic-kernel

For C# or Java implementations, you can refer to the official Semantic Kernel quickstart guide.

The Kernel

The Kernel is the core of Semantic Kernel. It is basically a Dependency Injection container that manages the services and plugins used for an AI application.

# Init kernel
kernel = Kernel()

In this chapter, we start by creating the most common AI service, a chat completion. This service type generates responses in conversational contexts, where the model not only use the last user message isolated, but within a context (the conversation history) so the response is coherent and relevant to the conversation.

# Add chat completion service
kernel.add_service(AzureChatCompletion(
    base_url='base_url' # For example, an Azure OpenAI instace url
    api_key='api_key', # The Api Key associated to the previous instace
    deplyoment_name='deployment_name' # A chat model like gpt-4o-mini
))

Alternatively, if the settings are not provided explicitly on the constructor, Semantic Kernel will try to load them from the environment based on predefined names. For example, Azure OpenAI related settings are always prefixed with AZURE_OPEN_AI (e.g: AZURE_OPENAI_BASE_URL, AZURE_OPENAI_API_KEY, AZURE_OPENAI_CHAT_DEPLOYMENT_NAME).

Once a service is added to the Kernel, it can be retrieved later by its type.

chat_service = kernel.get_service(type=ChatCompletionClientBase)

The agent

For this series of blogs, I will build a book assistant. Feel free to experiment with your preferred theme for your chatbot.

To start with it, we create a book_assistant.py file. In the constructor, we initialize the Kernel and the corresponding AI services.

from semantic_kernel import Kernel
from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion

class BookAssistant:
    def __init__(self):
        # Initalize the kernel and the AI Services
        self.kernel = Kernel()
        self.chat_service = AzureChatCompletion(
            service_id="chat_completion"
        )

        # Add AI Services to the kernel
        self.kernel.add_service(self.chat_service)

For this conversational agent, we'll be using the Agent Framework from Semantic Kernel. We define it as a ChatCompletionAgent providing the AI Services through the Kernel. Semantic Kernel automatically selects the available ChatCompletion services from the Kernel. In our case, it will leverage the AzureChatCompletion service to reply to the user questions.

from semantic_kernel.agents import ChatCompletionAgent

class BookAssistant:
    def __init__(self):
        ... # More code

        self.agent = ChatCompletionAgent(
            service_id='chat_completion',
            name='BookAssistant',
            kernel=self.kernel
        )

Agent Framework is experimental, the code used here might not be compatible with future versions of the library.

Chat history

The chat history tracks and maintains the record of messages throughout a chat session, enabling context preservation and continuity of conversation.

For now, we just initialize it on the __init__ assistant method:

from semantic_kernel.contents import ChatHistory

class BookAssistant:
    def __init__(self):
        ... # More code

        self.history = ChatHistory()

Calling the model

Once we have the ChatCompletionAgent and the ChatHistory initialized, we are ready to interact with the agent. We add a new async method call in our BookAssistant class. The method will:

1. Receive the last user_message as an argument.
2. Add it to the ChatHistory as a user message.
3. Invoke the agent with the invoke method passing the ChatHistory.
4. The ChatCompletionAgent uses the model to generate a reply to the last user_message guided by the context provided in the ChatHistory.
5. Add the response to the ChatHistory as an assistant message.
6. Return back the response to the caller so it's shown in the chat interface.

from semantic_kernel.contents.utils.author_role import AuthorRole

async def call(self, user_message: str) -> str:
    self.history.add_message(ChatMessageContent(role=AuthorRole.USER, content=user_message))

    async for response in self.agent.invoke(self.history):
        self.history.add_message(response)
        return str(response)

With this simple piece of code, we already have an easy way of having a conversation with the chatbot. However, the agent does not act as a book assistant yet, it is just a generic one. For example, if we ask what kind of things can do, it replies with a vague and generic response:

Let's see how can we customize the behavior of the agent to act closer to a book assistant.

Adding instructions

We use the System Prompt to instruct the agent who/what it should be, how it should behave and how it should respond. In previous version of Semantic Kernel, System Prompt was defined using the Persona. However, as we are using the new experimental Agent Framework, the System Prompt is now provided on the Agent initialization:

self.agent = ChatCompletionAgent(
    service_id='chat_completion',
    name='BookAssistant',
    kernel=self.kernel,
    instructions="""
        You are a knowledgeable book assistant who helps readers explore and understand literature. You provide thoughtful analysis of themes, characters, and writing styles while avoiding spoilers unless explicitly requested.

        Your responses are concise but insightful, and you're careful to ask clarifying questions when needed to better understand readers' preferences and needs. When uncertain about details, you openly acknowledge limitations and present literary interpretations as possibilities rather than absolutes.
    """
)

With these simple instructions, we have adjusted the agent's tone, specifying its purpose, and adding some remarks about how we expect it to act under some circumstances. If we now repeat the previous question, the agent replies with a more concrete and precise answer.

If you want to know more about how to define a good system prompt, there are many free courses and blogs about the topic. For example, you can check out the Microsoft Prompt engineering techniques documentation.

Summary

In this chapter, we have accomplished the first steps on the development of a chatbot using Semantic Kernel with the new experimental Agent Framework. We have gone through some basic concepts, and provide some "personality" to the agent.

Remember that all the code is already available on my GitHub repository 🐍 PyChatbot for Semantic Kernel.

In the next chapter, we will add specific Librarian skills to our Agent through Plugins.