DEV Community: Maho Pacheco

OpenAI and .NET tutorial with Semantic Kernel - Part 1

Maho Pacheco — Sun, 31 Mar 2024 22:20:00 +0000

If you prefer video, there is a YouTube version of this blog-post.

Since the release of LLMs, I have been extremely enthusiastic about exploring such a fun tool to play with. I won't delve into discussions about how transformative it is or will be, but rather focus on the enjoyable aspect of using this technology.

In the broader context of ML/AI and data science, Python stands out as the most commonly used programming language. Despite my open-source enthusiasm, I have always found C# to be remarkable. With the introduction of Mono.NET and later dotnet core, I have happily used it in Linux environments. It's no surprise that when working on projects related to ML/AI, C# and .NET are my preferred choices.

Over the past few months, I've transitioned from being a mere user to someone who shares what I've learned. I find the topics of evaluating models and Responsible AI particularly fascinating. I've created some code samples that I'm excited to share.

However, in line with my approach to sharing knowledge in other topics, I prefer to establish a common ground or baseline understanding. Therefore, I've decided to start from the ground up, sharing and teaching how to build a small tool based on OpenAI or other LLMs. I'll then progress to unit testing, evaluating models, and observability – all in C# and dotnet. I acknowledge that there are already numerous Python projects in this space, and it is my desire to include the dotnet community.

So, buckle up! This will be a 4-5 post series, and I may even record a couple of videos for the first time in my life.

Prerequisites

Knowledge

Understanding of what OpenAI is and the concept of a LLM.
Optional but highly recommended basic knowledge of Azure (primarily for deploying an Azure OpenAI model).
Familiarity with C# and .NET (expertise is not necessary; basic understanding will suffice).
Know how to Google/Bing/search the internet. If any section does not have detailed instructions is on purpose, there is maybe plenty of articles already out there of how to do that specific step.

Tech stuff

A computer with Mac, Linux or Windows
dotnet 8.0 installed, or, a docker enabled machine, or, a github account
This tutorial will use VSCode, but any editor (vim, visual studio code, notepad++, wherever) should suffice.

The Idea Overview

I have decided we will build a short post content generator. Think of something to generate tweets/toots, mastodon posts, or LinkedIn posts, but very short (100-200 characters). Feel free to build something different, I actually encourage to try something different, this tutorial should be still helpful.

We will provide 3 inputs: style, topic, and persona field. The prompt would look like this:

You are a [persona] expert. Generate a short tweet in a [style] style about [topic].

For example:

You are a marketing expert. Generate a short tweet in a sarcastic style about the challenges of creating a viral campaign in summer.

You may be asking, what is the different between creating something
from scratch than only using a ChatGPT agent? Well, you could integrate it with other products, platforms by doing your own API. You could start improving it with your own examples, you could even use it with a very small non-OpenAI local LLM to run it in a non-cloud environment. And of course, you can just have fun learning. There is a lot of benefits.

Step 0

If you already have dotnet installed skip to Step 1. If you don't, you can use a ready to use devcontainer from this repo, just clone the repo, open it in vscode and start the remote container. If you don't have a docker enabled machine, you could always use GitHub Codespaces.

This is one of those things where there are many posts out there that explain this part better than me, so I am on purpose not adding instructions.

Step 1

So, we will start by creating a new command line project, we will open a terminal (bash or powershell should work) and execute:

dotnet new console -n PostGenerator

this will generate a Program.cs and postgenerator.csproj

Now we will add a couple of packages, the first one is Semantic Kernel. Semantic Kernel is one of my favorite libraries, to play with OpenAI. So we enter the folder PostGenerator and execute:

dotnet add package Microsoft.SemanticKernel

We will also add a couple of extensions that are useful from the beginning, the first is about Dependency Injection, and the second logging:

dotnet add package Microsoft.Extensions.DependencyInjection
dotnet add package Microsoft.Extensions.Logging
dotnet add package Microsoft.Extensions.Logging.Console

You may argue, why we need this from day zero or with a prototype.
With the time and experience you start answering to each question with "depends" and at the same time getting more opinionated in the way of work. This is one of those topics where there is no right or wrong (depends), but I have found very useful to use DI and Logging extensions from the beginning to avoid refactor my stuff later. You may agree or not, and that is fine.

The next one is more controversial, it is a library to handle the arguments in my command-line tool. I think (opinion) that is the same effort to handle the arguments array (and ugly effort), than implement it with the right devx with a utility library. So let's install, System.CommandLine, which is in preview, hence the need of the --prerelease option.

dotnet add package System.CommandLine --prerelease

Let's verify that our program still builds:

dotnet build
...
Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:04.13

Amazing.

Step 2

Now to the fun part, let's scallfold the command line utility. We will start by adding 3 parameters: --persona, --topic, and --style:

using System.CommandLine.Parsing;
using System.CommandLine;

var personaOption = new Option<string>("--persona")
{
    IsRequired = true,
};

var topicOption = new Option<string>("--topic")
{
    IsRequired = true,
};

var styleOption = new Option<string>("--style")
{
    IsRequired = true,
};

We will also add a "root" command, and the "create-post" command.

var rootCommand = new RootCommand();

var createPostCommand = new Command("create-post", "Create a new post");

createPostCommand.AddOption(personaOption);
createPostCommand.AddOption(topicOption);
createPostCommand.AddOption(styleOption);

rootCommand.AddCommand(createPostCommand);

Note: Fun fact, if you don't add the AddOption, it will still work, but things like IsRequired will be ignored.

We finally execute the command and return the result. Notice how we don't need to specify the main void static args, but we will use the args variable. Cool, right?

var result = await rootCommand.InvokeAsync(args);

return result;

Let's test it:

$ dotnet run
Required command was not provided.

Description:

Usage:
  PostGenerator [command] [options]

Options:
  --version       Show version information
  -?, -h, --help  Show help and usage information

Commands:
  create-post  Create a new post

And if we provide the create-post command, we get a blank response:

$dotnet run -- create-post --persona "software developer" --topic "unit tests" --style "sarcastic"

This is because there is no handler for the command, let's add one just before adding the createPost command to the root command.

createPostCommand.SetHandler((persona, topic, option) =>
{
    Console.WriteLine($"Command requested for {persona} {topic} {option}");

    throw new NotImplementedException($"Command not implemented");
}, personaOption, topicOption, styleOption);

The other of personaOption, topicOption, styleOption, does not really matter, the user can provide them in any order, we just need to make sure it matches the order of the arguments.

Step 3 - DI & Logging

Let's implement Depencency Injection with Logging. Observability has 3 pilars: logging, metrics, and traces. Logging is basically all messages that would be useful to troubleshoot an issue later, but also helpful to send messages to the user. Dotnet provides an excellent logging mechanism, and several extensions. We can include these two:

using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;

Dependency Injection is just a fancy repository for objects that will be used on the application. In long running systems, it is more clear the advantage of it, for example in web systems we can reuse the same http client and avoid instantiating a new object in each request and stagnating memory.

There are many ways to store/instantiate these objects, the two extremes is to create one each time we require one from the service provider, and the other is having a singleton, the same exact object.

We will create a new method at the end to configure these services. As it becomes more complex it can be their own class, or even extensions, but for now a method will be enough.

static void ConfigureServices(ServiceCollection serviceCollection, string[] args)
{

}

You would want to call this as one of the first things in your script, to avoid doing nasty things and not using it, so we add this, just after the usings.

var serviceCollection = new ServiceCollection();

ConfigureServices(serviceCollection, args);

using var serviceProvider = serviceCollection.BuildServiceProvider();

Now we will configure our logging using one of the built-in extensions:

static void ConfigureServices(ServiceCollection serviceCollection, string[] args)
{
    serviceCollection.AddLogging(configure =>
        {
            configure.AddSimpleConsole(options => options.TimestampFormat = "hh:mm:ss ");

            if (args.Any("--debug".Contains))
            {
                configure.SetMinimumLevel(LogLevel.Debug);
            }
        })
}

If you want to play with the formatters just check this.

As you may notice we are hijacking the --debug argument here, we need to add it to all the commands as well:

var debugOption = new Option<bool>(new[] { "--debug" }, "Enables debug logging");

rootCommand.AddGlobalOption(debugOption);

Finally we will replace that nasty Console.WriteLine.

createPostCommand.SetHandler((persona, topic, option) =>
{
    var logger = serviceProvider.GetRequiredService<ILogger>();

    logger?.LogDebug($"Command requested for {persona} {topic} {option}");

    throw new NotImplementedException($"Command not implemented");
}, personaOption, topicOption, styleOption);

Beautiful. We could abstract the handler to a different class, but for now we should be happy with the result. Let's test it:

dotnet run -- create-post --debug --persona "influencer" --topic "software dev" --style "sarcastic"

And with debug:

dotnet run -- create-post --debug --persona "influencer" --topic "software dev" --style "sarcastic" --debug

Amazing, we are ready to start adding LLMs stuff, you can find the our full code in my repo for this LLM series.

Bringing your site to the Fediverse: A practical guide for static sites - Part 1

Maho Pacheco — Sat, 16 Mar 2024 22:17:00 +0000

I spent the last couple of weeks getting this blog onto the social web (aka fediverse), investing an hour here and there (I hope you are reading this from Mastodon, btw). It was fun and a very exciting learning experience, and I want to share not only my reflections but also create a guide so others can do the same, to move the needle a little bit in the right direction.

First of all, ActivityPub is a very nice idea; I love how much it has grown and the adoption that is growing day by day. I am not an expert in ActivityPub; I am far from being one. My only technical experience can be counted in literally ten days since this first post. But I have grown fond of it, and I am now #TeamActivityPub. So much so that I want to explain to others how to achieve the same on their own sites.

But before explaining how to do it, I want to share my vision of why you should do it. (If you have already decided to do it, you may skip this part).

You may have heard all these nice things about the indieweb, the bad corporations, and privacy. And I agree with almost all of them, but there is also the aspect of innovation which not many times is covered. ActivityPub, the Fediverse, or the SocialWeb are enablers of innovation, and we should make sure others see it. If you want to create a new app, a new social network, or a new product, you don't need to start from zero to build a user base (or what is harder, a community); you can (and should) just focus on building that thing that adds value.

Bear with me; this is important to explain how to implement ActivityPub on your website. (Because at the end we are not going to implement the whole ActivityPub protocol, just the parts that make sense for you).

As you already know, Mastodon is a microblogging platform, a clone of Twitter, but the underlying protocol/technology, ActivityPub, is much more than that. It describes concepts that repeat in almost all social networks:

You can create content, described as a note.
You have an identity, or user.
You can follow and unfollow other users or people.
You can "like" and "reply" to other content.

There are others, but these base 4 concepts repeat themselves in each social network, no matter if it's MySpace, LinkedIn, or TikTok. These features have been implemented over and over, since Google+ and Orkut where a thing.

That is why ActivityPub is the base of other implementations besides microblogging; take as examples Lemmy (a Reddit clone) and PixelFed (an Instagram clone). They look very different from Mastodon, but they are interconnected (aka federated). This means that I can follow a community from Lemmy, not only with my Mastodon account, but with my Mastodon app. It may be that when I favorite a Lemmy-created link in Mastodon means no more than a thumbs up (for Mastodon), but for Lemmy (remember, a Reddit clone) it can be used to upvote a post. Or a photo of a cat on Mastodon could just be yet another post, but it could be beautifully presented on Pixelfed. I personally want to see the GitHub feed to federate in the socialweb (maybe someone can help to it in GitLab).

But also you do not need to reimplement ActivityPub to create some new instance on a server; you can just consume the APIs on the client side. I can create my own For You algorithm, an app for weather alerts, or a news aggregator (like Google News). Flipboard is a content curation site, for example, and it is interconnected with the socialweb.

And this leads me to your blog, a newsletter, or news site, which also:

Publishes content.
Has an identity or multiple (for example, a news site could have @tech@nytimes.com, @politics@wsj.com, etc.).
You can subscribe to a blog or newsletter (follow/unfollow).
You can "like" or "comment" (reply).

Which btw, there is nothing on ActivityPub that says you cannot make your subscriptors (followers) paid-only, a follow request can be accepted after you verify any sort of payment. But the point is that imagine that you won't need to create yet another account on each newspaper to be able to discuss or comment on their content. Or that you don't need to create an account on Substack to follow an author. Or that you don't even need to create a user on Google or Facebook to like a photo.

That is what ActivityPub, the Fediverse, and the Social Web bring: interoperability. This can go further, as in distributed dentity verification (you'll be able to own your identity across instances), but that is another topic. For now, I want to convince you that if you have a site that publishes content, it should be in the Fediverse and Social Web. If not for principles, for numbers: there is already 2 million active users every month can read, consume, and engage with your content. You don't need to create a user base, the user base already exists.

It appears that WordPress, Medium, Threads, Flipboard understood this, and with their actions, they are helping to convince others that this is no longer just an ideological matter but a practical one. There are some people who said this may not succeed, but what I have seen is that it has already done.

It may not go mainstream to replace Twitter, but that is not the purpose of the Fediverse. That is not the real question. The question is: how can we harness the power of interoperability to enhance accessibility, engagement, and inclusivity within our online communities and content distribution platforms? By integrating with ActivityPub and embracing the principles of the Fediverse and the Social Web, we open doors to a vast network of users who can seamlessly interact with our content without barriers such as mandatory account creation or platform restrictions.

Moreover, we contribute to the democratization of online spaces, allowing individuals to maintain control over their digital identities and fostering a more decentralized and open internet ecosystem. This not only benefits users but also offers significant advantages for content creators and publishers, as it expands their reach and facilitates meaningful interactions with their audience.

In essence, the question is not whether the Fediverse will replace mainstream platforms but rather how we can leverage its potential to create more inclusive, user-centric online experiences that prioritize accessibility and collaboration. This is the promise of the world-wide-web, interconnected websites, but also interconnected people. And if you site is not part of it, well, then keep reading.

You can find the series here or in my substack and follow the blog (@blog@maho.dev) in the Fediverse/SocialWeb!