DEV Community

Cover image for Letting users teach an AI which memories matter
Aftab Bashir
Aftab Bashir

Posted on

Letting users teach an AI which memories matter

#dotnet #blazor #ai #openai

I built BlazorMemory to give Blazor AI assistants persistent memory. The library uses an LLM to extract facts from conversations, embeds them as vectors, and retrieves the relevant ones for each new query.
The system worked but it had a quiet failure mode. The LLM decided what was important. The user had no say. So when someone marked their birthday as a memory and the assistant kept surfacing "user enjoys coffee" instead, there was no way to fix it.
v0.5.0 fixes this.
How it works
Each memory now has an

ImportanceScore
Enter fullscreen mode Exit fullscreen mode

field. Default is 1.0. Users can mark a memory as important (1.5) or unimportant (0.3) via thumbs up and thumbs down buttons in the panel.
During search, the score acts as a multiplier on cosine similarity:

// In QueryAsync
results = results
    .Select(m => m.WithRelevanceScore((m.RelevanceScore ?? 0f) * m.ImportanceScore))
    .OrderByDescending(m => m.RelevanceScore)
    .ToList();
Enter fullscreen mode Exit fullscreen mode

That is the entire algorithm. A memory marked important with a 0.7 similarity gets boosted to 1.05 and beats a neutral memory at 0.9. A memory marked unimportant with a 0.9 similarity drops to 0.27 and falls off the list.

API

// Boost in future searches
await memory.MarkImportantAsync(memoryId);

// Down-rank but do not delete
await memory.MarkUnimportantAsync(memoryId);

// Back to neutral
await memory.ResetImportanceAsync(memoryId);

// Custom score
await memory.SetImportanceAsync(memoryId, 1.8f);
Enter fullscreen mode Exit fullscreen mode

In the UI
If you use the component, the thumbs buttons appear on every memory card automatically:

<MemoryPanel UserId="@userId" IsOpen="true" />
Enter fullscreen mode Exit fullscreen mode

You can hide them with AllowFeedback="false" if you do not want users to override the LLM.
Important memories get a green left border. Unimportant ones get dimmed and a red left border. Easy to scan at a glance.
Why down-rank instead of delete
The thumbs down button could have just deleted the memory. I decided against it for two reasons.
First, users click the wrong button. A delete that can not be undone is hostile UX.
Second, sometimes you want to suppress a memory without erasing it. Maybe the assistant keeps bringing up an old job title and you want to push it down without losing the context completely. Down-ranking with a score of 0.3 makes the memory available but unlikely to surface.
If users actually want to delete, the ✕ button right next to the thumbs is one click away.

Getting v0.5.0

dotnet add package BlazorMemory
dotnet add package BlazorMemory.Components
dotnet add package BlazorMemory.Storage.IndexedDb
dotnet add package BlazorMemory.Embeddings.OpenAi
dotnet add package BlazorMemory.Extractor.OpenAi
Enter fullscreen mode Exit fullscreen mode

All 8 packages on NuGet. Tests passing.
GitHub: https://github.com/aftabkh4n/BlazorMemory

Top comments (0)