Implementing Simple RAG in local environment /w .NET (C#).

Rudolf Šimo — Fri, 25 Apr 2025 14:06:18 +0000

This tutorial demonstrates how to build a simple Retrieval-Augmented Generation (RAG) solution using:

Ollama LLM (Gemma 3)
mxbai large embedding model
PostgreSQL with the pgvector extension
Pokémon dataset

Overview

This solution demonstrates how to:

Generate embeddings for Pokémon data using the mxbai large embedding model.
Store embeddings in a PostgreSQL database with pgvector.
Perform similarity searches to retrieve relevant Pokémon data.
Use Ollama LLM to generate responses based on user input and retrieved data.

Prerequisites

Environment Setup:
- Install .NET 8 SDK.
- Install PostgreSQL with the pgvector extension enabled.
- Install Ollama from: https://ollama.com/
- Install LLM model through Ollama, I used gemma3 model: https://ollama.com/library/gemma3
- Install Embedding model through Ollama, I used mxbai-embed-large: https://ollama.com/library/mxbai-embed-large
Dependencies:

Add the following NuGet packages to your project:
- Microsoft.SemanticKernel
- Microsoft.SemanticKernel.Connectors.Postgres
- IdeaTech.SemanticKernel.Connectors.Ollama
- Npgsql
Database Setup:

Ensure your PostgreSQL database is running and has the pgvector extension installed. Use the connection string:

   Host=localhost;Port=5433;Username=postgres;Password=admin;Database=postgres;

Step 1: Define Pokémon Data

Create a Data class to hold Pokémon records and generate sample data.

Why?

This step creates the foundational data structure for the Pokémon dataset. The Pokemon class implements the IEmbeddable interface, which ensures compatibility with the embedding and vector search services. The DescriptionEmbedding property will store the vector representation of the Pokémon's description.

internal class Data
{
    private static List<Pokemon> myRecords;

    public Data()
    {
        myRecords = new List<Pokemon>();
        CreateRecords();
    }

    private void CreateRecords()
    {
        myRecords = new List<Pokemon>
        {
            new Pokemon { Id = 1, Name = "Bulbasaur", Description = "A Grass/Poison type Pokémon." },
            new Pokemon { Id = 2, Name = "Charmander", Description = "A Fire type Pokémon." },
            new Pokemon { Id = 3, Name = "Squirtle", Description = "A Water type Pokémon." },
            // Add more Pokémon as needed
        };
    }

    public List<Pokemon> RetrieveRecords()
    {
        return myRecords;
    }
}

internal class Pokemon : IEmbeddable
{
    [VectorStoreRecordKey(StoragePropertyName = "id")]
    public long Id { get; set; }

    [VectorStoreRecordData(StoragePropertyName = "name")]
    public string Name { get; set; }

    [VectorStoreRecordData(StoragePropertyName = "description")]
    public string Description { get; set; }

    [VectorStoreRecordVector(Dimensions: 1024, DistanceFunction.CosineDistance, StoragePropertyName = "descriptionembedding")]
    public ReadOnlyMemory<float> DescriptionEmbedding { get; set; }
}

Step 2: Initialize the Database

Create a DatabaseInitializer class to set up the PostgreSQL database.

For the embedding VECTOR(1024) size, in this case 1024, can be found as embedding_length, embedding_size, vector size, output_dim, hidden size...

For mxbai-embed-large Embedding Model, the dimensions were found here:

Why?

This step ensures that the database is ready to store Pokémon data and their embeddings. The VECTOR(1024) column is specifically designed to store embeddings generated by the mxbai-embed-large model.

internal class DatabaseInitializer
{
    private readonly string _connectionString;

    public DatabaseInitializer(string connectionString)
    {
        _connectionString = connectionString;
    }

    public async Task InitializeDatabaseAsync<T>() where T : IEmbeddable
    {
        using var connection = new NpgsqlConnection(_connectionString);
        await connection.OpenAsync();

        string createTableQuery = @"
            CREATE TABLE IF NOT EXISTS pokemons (
                id BIGINT PRIMARY KEY,
                name TEXT,
                description TEXT,
                descriptionembedding VECTOR(1024)
            );";

        using var command = new NpgsqlCommand(createTableQuery, connection);
        await command.ExecuteNonQueryAsync();
    }
}

Step 3: Generate and Store Embeddings

Use the TextEmbedding class to generate embeddings for Pokémon descriptions and store them in the database.

Why?

This step converts Pokémon descriptions into vector embeddings, which are essential for performing similarity searches. The embeddings are then stored in the database for future queries.

internal class TextEmbedding
{
    public async Task GenerateEmbeddingAndUpsertAsync<T>(
        ITextEmbeddingGenerationService textEmbeddingGenerationService,
        IVectorStoreRecordCollection<long, T> vectorStoreRecordCollection,
        T objectToEmbed) where T : IEmbeddable
    {
        string description = objectToEmbed.Description;

        ReadOnlyMemory<float> searchEmbedding = await textEmbeddingGenerationService.GenerateEmbeddingAsync(description);
        objectToEmbed.DescriptionEmbedding = searchEmbedding;

        await vectorStoreRecordCollection.UpsertAsync(objectToEmbed);
    }
}

Step 4: Perform Similarity Search

Retrieve the most relevant Pokémon data based on user input.

Why?

This step enables the system to find Pokémon whose descriptions are semantically similar to the user's query. It uses vector embeddings to perform the search.

public async Task<string> SearchAsync<T>(
    ITextEmbeddingGenerationService textEmbeddingGenerationService,
    IVectorStoreRecordCollection<long, T> vectorStoreRecordCollection,
    string query) where T : IEmbeddable
{
    ReadOnlyMemory<float> queryEmbedding = await textEmbeddingGenerationService.GenerateEmbeddingAsync(query);
    VectorSearchResults<T> results = await vectorStoreRecordCollection.VectorizedSearchAsync(queryEmbedding);
    var resultItems = await results.Results.ToListAsync();

    var topResults = resultItems.Take(3).ToList();

    StringBuilder resultString = new StringBuilder();
    foreach (var result in topResults)
    {
        if (result.Record is Pokemon pokemon)
        {
            resultString.AppendLine($"{pokemon.Name}: {pokemon.Description}");
        }
    }

    return resultString.ToString();
}

Step 5: Integrate Ollama LLM

Use Ollama LLM to generate responses based on user input and retrieved Pokémon data.

Why?

This step integrates the Ollama LLM to generate natural language responses based on the retrieved Pokémon data and user input.

internal class LLMCommunicator
{
    private TextEmbedding myTextEmbedding;
    private Kernel myKernel;
    private PostgresVectorStoreRecordCollection<long, Pokemon> myVectorStoreRecordCollection;

    public LLMCommunicator()
    {
        myKernel = InitializeKernel();
        myTextEmbedding = new TextEmbedding();
    }

    public Kernel InitializeKernel()
    {
        IKernelBuilder kernelBuilder = Kernel.CreateBuilder();
        kernelBuilder.Services
            .AddOllamaChatCompletion("gemma3", new Uri("http://localhost:11434"))
            .AddOllamaTextEmbeddingGeneration("mxbai-embed-large", new Uri("http://localhost:11434"));

        return kernelBuilder.Build();
    }

    public async Task AddRecords()
    {
        var ollamaEmbedding = myKernel.Services.GetService<ITextEmbeddingGenerationService>();
        Data data = new Data();

        var databaseInitializer = new DatabaseInitializer("Host=localhost;Port=5433;Username=postgres;Password=admin;Database=postgres;");
        await databaseInitializer.InitializeDatabaseAsync<Pokemon>();

        NpgsqlDataSourceBuilder dataSourceBuilder = new("Host=localhost;Port=5433;Username=postgres;Password=admin;Database=postgres;");
        dataSourceBuilder.UseVector();
        var dataSource = dataSourceBuilder.Build();
        myVectorStoreRecordCollection = new PostgresVectorStoreRecordCollection<long, Pokemon>(dataSource, "pokemons");

        foreach (var record in data.RetrieveRecords())
        {
            await myTextEmbedding.GenerateEmbeddingAndUpsertAsync(ollamaEmbedding, myVectorStoreRecordCollection, record);
        }
    }

    public async Task Run(string userInput)
    {
        var ollamaEmbedding = myKernel.Services.GetService<ITextEmbeddingGenerationService>();
        string mostSimilarDocument = await myTextEmbedding.SearchAsync(ollamaEmbedding, myVectorStoreRecordCollection, userInput);

        var chatCompletionService = myKernel.Services.GetService<IChatCompletionService>();
        string prompt = $"You are a bot that makes Pokémon recommendations. Recommended Pokémon: {mostSimilarDocument}. User input: {userInput}.";

        var settings = new OllamaPromptExecutionSettings { MaxTokens = 4096 };
        var history = new ChatHistory(prompt);
        var responses = await chatCompletionService.GetChatMessageContentsAsync(history, settings, myKernel);

        foreach (var response in responses)
        {
            Console.WriteLine(response.Content);
        }
    }
}

Step 6: Run the Solution

In your Program.cs, initialize the LLMCommunicator and run the solution.

internal class Program
{
    static async Task Main(string[] args)
    {
        LLMCommunicator communicator = new LLMCommunicator();
        await communicator.AddRecords();

        Console.WriteLine("Enter your query:");
        string userInput = Console.ReadLine();
        await communicator.Run(userInput);
    }
}

Try it

Now try asking: Could you recommend me a Fire type Pokemon? (For the best results, try to create a Non Existing pokémon, e.g. NonExistant /w Description "A Non Existing Pokemon". Then try: Could you recommend me a Non Existing Pokemon?

Conclusion

This tutorial demonstrates how to build a simple RAG solution using Ollama LLM, mxbai embeddings, pgvector, and Pokémon data. You can extend this solution by adding more data, improving the prompt, or integrating additional features.

Sources & References

[1] Original sample used (Without Embeding model and vector database): https://learnbybuilding.ai/tutorial/rag-from-scratch/
[2] https://github.com/ollama/ollama/blob/main/docs/windows.md
[3] https://github.com/pgvector/pgvector
[4] https://learn.microsoft.com/en-us/semantic-kernel/concepts/vector-store-connectors/?pivots=programming-language-csharp
[5] https://learn.microsoft.com/en-us/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/?pivots=programming-language-csharp
[6] https://learn.microsoft.com/en-us/semantic-kernel/concepts/vector-store-connectors/defining-your-data-model?pivots=programming-language-csharp
[7] https://learn.microsoft.com/en-us/dotnet/ai/semantic-kernel-dotnet-overview
[8] https://learn.microsoft.com/en-us/semantic-kernel/concepts/ai-services/embedding-generation/?tabs=csharp-AzureOpenAI&pivots=programming-language-csharp
[9] https://devblogs.microsoft.com/semantic-kernel/managing-chat-history-for-large-language-models-llms/

DEV Community: Rudolf Šimo

Implementing Simple RAG in local environment /w .NET (C#).

Overview

Prerequisites

Step 1: Define Pokémon Data

Why?

Step 2: Initialize the Database

Why?

Step 3: Generate and Store Embeddings

Why?

Step 4: Perform Similarity Search

Why?

Step 5: Integrate Ollama LLM

Why?

Step 6: Run the Solution

Try it

Conclusion

Sources & References