How to Give Your .NET AI App a Memory: A Developer’s Guide to Persistent LLM Storage

I was recently working on a conversational AI project, building a chatbot for customer support. As I delved deeper, I realized a significant limitation: the chatbot couldn't remember past interactions. Each conversation started from scratch, making it impossible to provide personalized and context-aware responses. This got me thinking about the importance of memory for AI applications and how to implement it effectively.

In this article, we'll explore the concept of persistent storage for LLMs (Large Language Models) and how it can revolutionize your .NET AI applications. By giving your AI a memory, you can unlock new levels of intelligence, personalization, and user engagement.

Why Your .NET AI App Needs Memory

Traditional LLMs are stateless, meaning they treat each interaction as an isolated event. This lack of memory limits their ability to understand context, learn from past interactions, and provide truly personalized experiences.

Imagine a chatbot that remembers your past conversations, anticipates your needs, and provides tailored responses. Or a code assistant that learns your coding style and suggests relevant snippets based on your previous work. These are just a few examples of how memory can enhance AI applications.

Here are some key benefits of adding memory to your .NET AI app:

• Contextual Awareness: LLMs can understand the user's history and provide more relevant and personalized responses.

• Personalization: LLMs can adapt to individual user preferences and provide customized experiences.

• Improved Accuracy: LLMs can learn from past interactions and improve their performance over time.

• Learning from User Feedback: LLMs can incorporate user feedback to refine their responses and provide more accurate information.

• Building Personalized Models: LLMs can be fine-tuned on specific user data to create personalized models that cater to individual needs.

By adding memory to your .NET AI applications, you can create more intelligent, engaging, and user-centric experiences.

Choosing the Right Storage Strategy

Now that we understand the importance of memory for LLMs, let's explore the different ways to store LLM interactions. The optimal approach depends on your specific needs and the type of application you're building.

Here are three common strategies:

Raw Conversation Logs

This is the simplest approach, where you store the entire conversation history as plain text or in a structured format like JSON. Each interaction between the user and the LLM is recorded and stored sequentially.

Pros:

• Easy to implement

• Provides a complete record of the conversation

Cons:

• Can be inefficient for large-scale applications

• Difficult to search and analyze

Embeddings

Embeddings are dense vector representations of text that capture semantic meaning. By converting user queries and LLM responses into embeddings, you can store them more efficiently and enable semantic search.

Pros:

• More compact and efficient than raw text

• Enables semantic search and retrieval

Cons:

• Requires understanding of embedding models and techniques

• Can be computationally expensive to generate embeddings

Structured Data

This approach involves extracting key information from LLM interactions and storing it in a structured format, such as a database. This allows for more efficient querying and analysis of the data.

Pros:

• Enables efficient querying and analysis

• Flexible for storing different types of information

Cons:

• Requires careful design of the data schema

• Can be more complex to implement

Choosing the right storage strategy is crucial for building efficient and scalable LLM applications with memory. Consider the pros and cons of each approach and select the one that best suits your needs.

Implementing Persistent Storage Securely

Okay, let's dive into the practical side of giving your .NET AI app a memory. We need a secure and reliable way to store those LLM interactions. And remember, these conversations can contain sensitive information, so security is absolutely paramount.

For this, we'll use ByteHide's Object Storage service. Why? Because it checks all the boxes:

• Post-quantum cryptography: ByteHide is built with the future in mind, using encryption algorithms that are resistant to attacks even from quantum computers. Your data will stay safe, now and in the years to come.

• End-to-end encryption: Your data is encrypted before it even leaves your application, and stays encrypted until it's retrieved. No one can snoop on your LLM interactions, not even ByteHide themselves.

• High availability: ByteHide's infrastructure is designed for reliability, so you can be confident that your data will always be available when you need it. This is essential for AI applications that require quick access to memory.

Getting Started with ByteHide Storage

1. Create a Free Account: Head over to cloud.bytehide.com/register to sign up for a free ByteHide account. You'll get up to 40GB of storage for free – plenty to get started with building your AI's memory.

2. Create a .NET Project: Once you're logged in, create a new project in the ByteHide panel and select ".NET" as the project type.

3. Get Your API Token: In the "Storage" section of your project settings, you'll find your API token. Keep this safe, as you'll need it to access your storage from your .NET application.

Now that we have our secure storage solution set up, let's jump right into a practical example and see how to use it in a real-world scenario. We'll build a simple chatbot with memory in the next section.

Building a .NET Chatbot with Memory (Step-by-Step Guide)

We'll build a simple chatbot in .NET that remembers past interactions using an LLM and ByteHide Storage.

1. Create your project

Open your preferred IDE (like Visual Studio) and create a new .NET console application.

2. Install NuGet packages

Install the required packages for LLM integration and secure storage using your IDE's package manager or the .NET CLI:

dotnet add package Microsoft.Extensions.AI
dotnet add package Bytehide.Storage

3. Set up and integrate the LLM

• We’ll use Ollama, which allows you to run LLMs locally within Docker containers. This simplifies setup and gives you more control over your LLM environment.

• Install Docker: If you don’t have Docker Desktop installed on your machine, download and install it from the official Docker website.

• Install Ollama: Follow the instructions on the Ollama website to install Ollama. It’s typically a single command.

• Run Llama2 with Ollama:

  • Download the model: Use the following command in your terminal to download the Llama 2 model: ollama pull llama2
  • Start Ollama: Start the Ollama service with the Llama 2 model: ollama run llama2
  • Verify: You can verify that Ollama is running by visiting http://localhost:11434 in your web browser. You should see the Ollama web interface.

• Configure your .NET project: In your Program.cs file, configure your .NET application to use the Ollama endpoint for LLM interaction:
  
  

using Microsoft.Extensions.AI;

// ... other code ...

builder.Services.AddAi(options =>
{
    options.Provider = AiProvider.Ollama;
    options.Ollama.Endpoint = "http://localhost:11434";
});

4. Create the method to save the data

• Create a class to manage saving the conversation history to ByteHide Storage.

• Use the StorageManager class to interact with the storage service.

• You can configure additional options like compression and encryption to optimize storage and security.

• Here’s an example of how to save a conversation history in text format or as a structured JSON object, taking advantage of ByteHide Storage’s ability to auto-deserialize it:
  

using Bytehide.Storage;

public class ConversationStorage
{
    private readonly StorageManager _storage;

    public ConversationStorage(string projectToken, string? encryptionPhrase = null)
    {
        // Initialize ByteHide Storage with the project token and optional encryption phrase
        _storage = encryptionPhrase != null 
            ? new StorageManager(projectToken, encryptionPhrase) 
            : new StorageManager(projectToken);
    }

    public void SaveConversationHistory(string userId, string conversationHistory)
    {
        _storage
            .In("<your_bucket_name>/conversations")
            .Compress()
            .Encrypt()
            .Set($"conversation_{userId}.txt", conversationHistory);
    }

    public void SaveConversationHistory(string userId, List<ChatMessage> conversationHistory)
    {
        _storage
            .In("<your_bucket_name>/conversations")
            .Compress()
            .Encrypt()
            .Set($"conversation_{userId}.json", conversationHistory);
    }
}

5. Create the method to retrieve the data

• Create a class to manage loading the conversation history from ByteHide Storage.

• Use the StorageManager class to retrieve the data.

• Handle the case where there is no previous history for the user.

• Here’s an example of how to load the conversation history in both text and JSON formats:
  

using Bytehide.Storage;

public class ConversationStorage
{
    // ... (previous code for the constructor and SaveConversationHistory) ...

    public string LoadTextConversationHistory(string userId)
    {
        try
        {
            return _storage
                .In("<your_bucket_name>/conversations")
                .GetText($"conversation_{userId}.txt");
        }
        catch
        {
            return ""; // Return an empty string if no history is found
        }
    }

    public List<ChatMessage> LoadJsonConversationHistory(string userId)
    {
        try
        {
            return _storage
                .In("<your_bucket_name>/conversations")
                .Get<List<ChatMessage>>($"conversation_{userId}.json");
        }
        catch
        {
            return new List<ChatMessage>(); // Return an empty list if no history is found
        }
    }
}

Example ChatMessage class:

public class ChatMessage
{
    public string Role { get; set; } // "User" or "Assistant"
    public string Content { get; set; }

    public ChatMessage(string role, string content)
    {
        Role = role;
        Content = content;
    }
}

Useful ByteHide Storage options:

• .Compress(): Reduces the size of the stored data, saving space and improving performance.

• .Encrypt(): Protects your data with robust encryption. You can specify the encryption algorithm when initializing StorageManager by providing an optional QuantumAlgorithmType parameter.

• .Set(): This method can automatically serialize objects into JSON format when you provide an object as the second argument.

• .Get<T>(): This method can automatically deserialize JSON data into an object of type T.

6. Complete and functional example

using Bytehide.Storage;
using Microsoft.Extensions.AI;

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddAi(options =>
{
    options.Provider = AiProvider.Ollama;
    options.Ollama.Endpoint = "http://localhost:11434";
});

var app = builder.Build();

// Initialize StorageManager with your project token and optional encryption phrase
// For post-quantum encryption, use:
// var storage = new StorageManager("<your_project_token>", QuantumAlgorithmType.Kyber1024);
var storage = new StorageManager("<your_project_token>", "<your_encryption_phrase>"); 

app.MapGet("/", async (IAiService aiService) =>
{
    Console.WriteLine("Welcome to the chatbot!");
    string userId = Guid.NewGuid().ToString();

    var conversationStorage = new ConversationStorage(storage);
    List<ChatMessage> conversationHistory = conversationStorage.LoadJsonConversationHistory(userId);

    while (true)
    {
        Console.Write("You: ");
        string userInput = Console.ReadLine();
        conversationHistory.Add(new ChatMessage(ChatRole.User, userInput));

        var chatRequest = new ChatRequest(
            new ChatMessage(ChatRole.User, userInput),
            new ChatMessage(ChatRole.System, "You are a helpful AI assistant."),
            new ChatMessage(ChatRole.Assistant, string.Join("\n", conversationHistory.Select(m => $"{m.Role}: {m.Content}")))
        );

        var response = await aiService.ChatAsync(chatRequest);
        string assistantReply = response.Choices[0].Message.Content;
        conversationHistory.Add(new ChatMessage(ChatRole.Assistant, assistantReply));

        Console.WriteLine($"Bot: {assistantReply}");
        conversationStorage.SaveConversationHistory(userId, conversationHistory);
    }
});

app.Run();

Classes and methods

// ChatMessage class
public class ChatMessage
{
    public string Role { get; set; }
    public string Content { get; set; }

    public ChatMessage(string role, string content)
    {
        Role = role;
        Content = content;
    }
}

// ConversationStorage class
public class ConversationStorage
{
    private readonly StorageManager _storage;

    public ConversationStorage(StorageManager storage)
    {
        _storage = storage;
    }

    public void SaveConversationHistory(string userId, List<ChatMessage> conversationHistory)
    {
        _storage
            .In("<your_bucket_name>/conversations")
            .Compress()
            .Encrypt()
            .Set($"conversation_{userId}.json", conversationHistory);
    }

    public List<ChatMessage> LoadJsonConversationHistory(string userId)
    {
        try
        {
            return _storage
                .In("<your_bucket_name>/conversations")
                .Get<List<ChatMessage>>($"conversation_{userId}.json");
        }
        catch
        {
            return new List<ChatMessage>();
        }
    }
}

And there you have it! A complete and functional example of a .NET chatbot that remembers conversations. This example provides a solid foundation for building more complex chatbot applications with persistent memory. You can customize it further by incorporating additional features and integrating it with other .NET components

LLM Memory Management: Advanced Techniques and Best Practices

We’ve covered the basics of giving your .NET AI app a memory. Now, let’s explore some advanced techniques and best practices to optimize performance, security, and scalability.

Data Optimization

• Compression: Compress the data before storing it to reduce storage costs and improve retrieval speed. ByteHide Storage offers built-in compression capabilities with the .Compress() method.

• Indexing: For large datasets, consider implementing indexing techniques to speed up data retrieval. This is especially useful if you need to search or filter through the stored conversations.

Security and Privacy

• Encryption: Always encrypt sensitive data before storing it. ByteHide Storage provides robust encryption, including post-quantum cryptography.

• Access Control: Implement strict access control measures to prevent unauthorized access to your stored data. ByteHide Storage allows you to define granular access permissions for your storage buckets.

• Data Minimization: Only store the data that is absolutely necessary for your application. Avoid storing sensitive information that is not required for LLM memory.

• Regular Audits: Conduct regular security audits to identify and address potential vulnerabilities.

Building Knowledge Bases

• Extract Key Information: Extract key information from LLM interactions and store it in a structured format. This can be used to build a knowledge base that your LLM can access to provide more accurate and informative responses.

• Update the Knowledge Base: Regularly update the knowledge base with new information and insights gained from LLM interactions.

• Use a Graph Database: Consider using a graph database to store and manage the knowledge base. Graph databases are well-suited for representing relationships between different pieces of information.

By implementing these advanced techniques and best practices, you can ensure that your LLM memory is secure, efficient, and scalable.

Test Your Knowledge: Build Your Own AI with Memory!

Ready to put your newfound knowledge to the test? Building an AI application with memory might seem daunting, but with the right tools and guidance, it’s a rewarding endeavor. Here’s a quick recap of the key steps:

1. Choose your LLM: Select an LLM that suits your needs and integrate it into your .NET application. Ollama, with its diverse model library, is a great option for local development.

2. Set up secure storage: Choose a reliable and secure storage solution like ByteHide Storage. Remember to prioritize data protection with encryption and access control.

3. Implement storage logic: Write the code to store and retrieve LLM interactions. Leverage ByteHide Storage’s features for efficient data management.

4. Build your AI application: Integrate the LLM and storage logic into your application. Design your application to leverage the memory capabilities for enhanced user experiences.

Now it’s your turn to build something amazing! Experiment with different LLMs, explore various storage strategies, and unleash your creativity.